How XR Trading Fixed a Costly Exchange-Side Bug Before Competitors Caught On

In High-Frequency Trading (HFT), software errors don’t just cause inconvenience—they directly impact revenue. When unexpected exchange rejections started occurring on orders that had been working for years, the team at XR Trading faced a critical challenge: finding the root cause fast before missing out on high-value trades. Traditional debugging methods were slow and inefficient, putting them at risk of falling behind competitors. That’s when they turned to Undo.

The Challenge: Unexpected Order Rejections Costing $15,000/Day

XR Trading noticed that orders they had been successfully sending for years were suddenly being rejected—despite no software changes on their end. Each failed order meant missed opportunities, totaling an estimated $15,000 in daily losses.

To diagnose the issue, the software engineering team needed to trace how messages passed through their complex software stack, where components are arranged in a sequential chain. Tracking a message’s journey across 12 components and hundreds of method calls using traditional debugging tools like GDB was proving to be painfully slow.

To trace the message across 12 components using GDB, I would need to ‘step in’ to about 40 methods and ‘step over’ hundreds of lines of code to follow the path the message takes before the message arrives at component 12.

And if I accidentally step too far, I need to restart from the beginning—so painful and time-consuming!

Zephaniah Hill, Software Engineer, XR Trading

Unfortunately, simply looking up the call stack wasn’t an option to determine where the change had actually occurred, since the bad state was being passed through the system in a “context” object that was modified by function calls in multiple components.

Solution: From Symptom to Root Cause in a Single Pass with Undo

With Undo’s time travel debugging solution, the engineering team was able to diagnose the issue in just one hour instead of spending days manually stepping through code.

• Using the last command, engineers were able to instantly jump to the exact moment where the order type was modified—without manually tracing the message through thousands of lines of execution history.
• Using reverse execution methods such as reverse-next and reverse-continue, engineers can run up to the error, then step back until they see what causes the bad program state—without needing to know in advance which variables and functions are important for the bug they are looking into.

Using last orderType I was able to jump to the location where we set the value causing errors. That meant we could immediately work on testing different order types and find a suitable type the exchange would accept.

Zephaniah Hill, Software Engineer, XR Trading

Impact: A Competitive Advantage for XR Trading

The engineering team found that they had hard-coded an order type which the exchanges rejections indicated was now treated as unsupported.

Undo allowed the XR Trading team to quickly understand the bug, allowing them to get a workaround and resume trading this product before their competitors even realized there was a problem.

Because this bug was exchange-side, XR Trading benefited from having fewer competitors in this product while everyone else caught up. Undo didn’t just help XR Trading fix a bug—it gave them a strategic advantage.

In a market where every millisecond matters, Undo delivered real business value by enabling the XR Trading engineering team to diagnose and resolve the issue faster than its competitors. Quickly resolving the issue gave XR Trading a temporary advantage—showcasing how better tools impact business outcomes.

We experienced unexpected rejections from the exchange on orders that we have been sending for years—without any software changes on our end. With Undo, it took me just 1 hour to diagnose the issue, allowing us to get a quick workaround and resume trading this product before our competitors were able to act.

Zephaniah Hill, Software Engineer, XR Trading

A New Standard for Debugging in High-Frequency Trading

For HFT firms like XR Trading, debugging speed directly impacts revenue. Traditional tools like GDB are simply too slow for mission-critical issues like exchange order failures. Undo’s time travel debugging not only reduced debugging time from days to hours, but also provided a crucial competitive edge—allowing XR Trading to capitalize on an exchange-side issue before competitors even knew what was happening.