Real-time applications: deadlines, not speed

A real-time application is one whose correctness depends on when a result arrives as well as what it contains. A late answer is not a slow answer — it is a wrong one.

This is routinely confused with "fast". A system averaging 2ms but occasionally taking 400ms is fast and not real-time. A system that always responds within 50ms, never faster and never slower, is real-time and comparatively slow. Predictability is the property, not speed.

Hard, firm and soft

• Hard real-time. A missed deadline is a system failure, possibly a dangerous one. Anti-lock braking, flight control surfaces, pacemakers, airbag deployment. The deadline is part of the specification and is proven, not measured.
• Firm real-time. A late result is useless but not dangerous — it is discarded. A video frame that misses its display slot, a market data update overtaken by a newer one.
• Soft real-time. A late result still has value, degrading as it ages. Video calls, live dashboards, game servers. Most systems described as real-time are this.

Why average latency is the wrong measure

Averages hide exactly the behaviour that matters. Consider two systems handling a million requests:

• System A: every request takes 20ms.
• System B: 999,000 requests take 1ms, and 1,000 take 2 seconds.

System B has the better average — about 3ms against 20ms — and is unusable for anything with a deadline. The thousand slow requests are real users.

Measure percentiles instead: p50, p99, p99.9 and the maximum. The tail is the system's actual behaviour; the mean is a summary that discards it. For hard real-time, only the maximum matters, and it must be bounded by analysis rather than observed in testing.

What makes a system unpredictable

• Garbage collection. A stop-the-world pause can exceed the entire deadline. Languages with unmanaged memory, or carefully tuned collectors, dominate hard real-time.
• Dynamic memory allocation. malloc has no bounded worst case — it may need to consolidate free lists. Real-time code typically preallocates everything at startup.
• Caches and page faults. A cache miss costs hundreds of cycles; a page fault to disk costs millions. Locking memory with mlockall removes the second.
• Priority inversion. A low-priority task holds a lock a high-priority task needs, and a medium-priority task preempts the low one — so the highest-priority task waits on the lowest. This is what nearly lost the Mars Pathfinder mission in 1997. Priority inheritance protocols exist to prevent it.
• Interrupts and shared hardware. Anything the scheduler does not control.

Real-time operating systems

A general-purpose scheduler optimises throughput and fairness. An RTOS optimises predictability, guaranteeing that the highest-priority runnable task gets the processor within a bounded interval.

Common choices include FreeRTOS, Zephyr, VxWorks and QNX. Linux can be made suitable with the PREEMPT_RT patches, now largely merged upstream, which reduce worst-case scheduling latency to the tens of microseconds — enough for a great deal of industrial control, though not for the most demanding safety cases.

Designing for a deadline

1. State the deadline numerically. "Fast" is not a requirement; "within 10ms, always" is.
2. Decide the class. Hard, firm or soft. This determines how much rigour is justified.
3. Remove unbounded operations from the deadline path: allocation, I/O, locks held by lower-priority code, anything that can retry.
4. Measure the tail under sustained load, not the average under a benchmark.
5. Decide what happens on a miss. Every real-time system misses eventually. Whether it degrades, drops the result, or fails safe is a design decision, and leaving it undefined means it will be decided by accident.