Precision instrumentation specialists
network analysis, data flow latency monitoring
With faster trading, surging volumes and evolving regulations continuing to shape the marketplace, the pressure is on to ensure that operational performance is continually optimised.
The ability to rapidly access liquidity at the intended price, without slippage, the emergence of unknown risks or the fear of market data quality issues, is integral to trading success. That’s why leading capital market firms and some of the world’s largest stock exchanges trust TipOff® to effectively measure and manage the performance of their extremely fast moving and high volume environments.
Capturing, decoding and normalising 100% of data received, TipOff provides real-time and retrospective analysis with access to detailed network, middleware and market data insights. Read on to learn more:
High-speed network packet capture
TipOff delivers high-speed, passive, network packet capture to disk with sustained data rates in excess of 40Gbps. By capturing everything, TipOff eradicates the risks associated with sampling approaches, guaranteeing that 100% of raw data received will be available for future analysis.
Utilising hardware accelerated compression technology, TipOff can store more than 20 TB of raw capture files on a 2U appliance or 80 TB of raw capture files on a 4U appliance. This approach facilitates high capacity storage with a low footprint, presenting the potential for significant cost savings.
TipOff’s high degree of compatibility, and ease of integration, makes it the perfect partner to Application Tap®, which offers a patented, high precision approach to in-application instrumentation, with minimal performance overhead.
Combining Application Tap’s ability to gain a fine-grained view of the individual components operating inside an application with TipOff’s extensive level of network visibility, blind spots are eliminated. So regardless of where an issue occurs, TipOff can pinpoint its exact source, considerably reducing the problem’s time to resolution.
This level of understanding enables users to set benchmarks for future performance and be alerted when thresholds are breached. This insight enables service providers to understand if the latency reported by a client is occurring within their environment or that of the client’s. It also enables exchanges to identify if the matching engine is running slower than expected or if public updates are being distributed more quickly than private.
Additionally, by identifying latency hotspots, users can accurately determine how latency reduction budgets should be apportioned to achieve the greatest return on investment (ROI).
TipOff’s precise instrumentation enables performance issues to be forensically analysed, right down to the individual network packet and nanosecond level, supporting:
- 1/10/40 Gigabyte Ethernet interfaces
- Microburst analysis at OSI layers 1 to 4
- TCP re-transmission and transport quality analytics
- TCP latency and packet layer latency analytics
All of these measures are achieved with exact timings to 10 nanosecond resolution, alongside detailed bandwidth analysis.
TipOff supports performance root cause analysis by leveraging its thorough examination of the middleware stacks that comprise each hop in a data flow. As the most significant source of latency is often the retransmission latency that results from lost packets, TipOff enables middleware-messaging activities to be independently monitored and analysed.
TipOff monitors the internal operation of reliability layers across an extensive range of middleware messaging protocols. TipOff provides protocol specific statistics, identifying retransmission requests, and generating subscriber, publisher and topic statistics for the messages themselves.
In doing so, early warning signs of operational conditions that could result in increased latency can be detected, exposing potential load balancing and capacity challenges.
Market data analysis
TipOff supports high volume, market data sequence gap detection. By instantly identifying and alerting the business to missing sequences, and recognising the feeds impacted, damage limitation methods can be implemented to avoid inaccurate trading decisions based on unknowingly partial data streams.
TipOff also enables the real-time and historical analysis of market data message volumes, which can then be used to facilitate more effective capacity planning.
Event syndication into client specific analysis systems
TipOff guarantees 100% capture and enables data to be decoded through its extensive library of high performance protocol decoders. In addition to offering an extensive range of performance analytics of its own, TipOff is also able to stream normalised data into a client’s own choice of either in-house or third party analysis tools. In doing so, the syndicated events are then able to feed client specific systems, databases and messaging tools providing vital risk and regulatory insight.
In response to client demand, TipOff has developed a range of event syndication adapters facilitating seamless integration with many of the most widely adopted third-party analysis tools.
Precise timed retransmission
TipOff’s precise timed retransmission capability enables TipOff to retransmit previously captured data onto the network, reproducing the original packet timings with high precision, including every microburst with nanosecond accuracy. Enabling the idiosyncrasies of real-world market data, for example, to be accurately re-experienced at the original recorded rate or a multiple or fraction of the original rate.
This unique functionality enables real-world environments to be re-created, so new algorithms and system enhancements can be more robustly tested and users gain a stronger understanding of how their algorithms and systems are likely to perform under varying conditions.
TipOff offers equities, FX, fixed income and derivatives market data and transaction decoders for an extensive range of regional and globally adopted protocols.
TipOff can be rapidly deployed as both an Appliance and software only solution on physical and virtual machines.
In an industry where speed of execution is integral to trading success the ability to detect and consistently refine potential latency sources is fundamental.
However, as trading and exchange architectures increasingly consolidate into multi-core servers running client gateways, trading or matching engines, less inter-process communication is happening on the network. Unless developers are able to see inside these processes, they can very quickly become major blind spots.
By applying network monitoring techniques at the server’s ingress and egress points, users can identify if latency is occurring within the application. But, it is only by instrumenting inside the application that developers are able to gain precise visibility into the inner-processes and pinpoint latency sources between different components, for instance:
- Between two points in the code
- Across network/application boundaries enabling the latency of messages traveling from a point on the network to be correlated with a point in the code, inside the application
- Or vice-versa as messages are then transmitted from the application back onto the network
Having identified which component is causing the delay, developers can then more efficiently allocate engineering effort to the issue’s resolution.
Application Tap offers a patented, high precision approach to in-application instrumentation, with minimal performance overhead. Providing very accurate insight inside application processes, Application Tap offers a fine-grained view of the particular components introducing performance issues, enabling real-time system developers to become significantly more effective in quickly identifying the exact source of potential problems.
Introducing Application Tap
With Application Tap users can not only determine which component is causing the delay, but also independently validate whether the latency lies within the application, or by measuring across network boundaries, if the delay actually resides on the network stack or within the operating system.
This ability to gain a hop-by-hop understanding of the performance levels being achieved inside the application and across network/application boundaries would be impossible to comprehend using network only monitoring techniques.
Application Tap enables users to tap into an application and gain a deep, accurate understanding of what is going on, so the problem’s source can be isolated, in both testing and production environments. This enables burst and sustained performance throughput rates to be accurately calculated with minimal overhead.
Offering a very lightweight means of accessing data from within application processes, Application Tap also enables firms to achieve many additional goals as the instrumented data can then be fed into other centralised systems where it can be correlated and examined to meet client specific analysis needs
How it works
Application Tap delivers a view inside systems by placing minimally invasive instrumentation hooks into the application at customisable points of interest. For instance, developers may wish to precisely instrument the time of order matching or when a pre-trade risk check is passed. How Application Tap can instrument a process is demonstrated by the following diagram.
With Application Tap developers can signal events by simply using the user mode API to write to the Application Tap memory. When deployed as hardware, Application Tap is able to limit the overhead to a memory write operation avoiding CPU cycles being chewed up unnecessarily on instrumentation.
The code passes the instrumentation metadata to the Application Tap, which time stamps events to 10 nanoseconds resolution, synchronised to PTP, providing a wall clock time. The application tap then publishes the instrumented data onto the network, where TipOff or Velocimetrics can recreate a unified, correlated view of the complete stream of network and application data as demonstrated by the following diagram.
This is possible even across distributed environments, enabling users to rapidly detect performance issues impacting real-time financial systems and networks, or to feed the data to client specific analysis systems supporting for instance risk or regulatory needs.
Application Tap is a very easy to implement API that enables precisely timed software instrumentation with minimal performance overhead.
Developing to Application Tap follows the same process regardless of whether the user wishes to do so in software, where it is most commonly deployed in testing environments or on hardware where, due to its lightweight approach to instrumentation, it proves very popular in production.
Available for C/C++ and Java, supported on both Linux and Windows, once implemented Application Tap can be then deployed throughout an environment in software or on hardware multiple times. This is possible without the need to actually set up a server with a physical Application Tap for each developer requiring access to perform instrumentation tasks.