Share on facebook
Share on twitter
Share on linkedin
Share on pinterest
Share on reddit


Critical system failures that don't trigger corrective action cause disasters
Dateline: 21 March 2016

Yet another airliner has fallen out of the sky without warning. And the reason is fail-safe systems that fail to alert the pilots to the real problem when they fail.

If that sounds a little confusing, that’s exactly what human operators experience, when autonomous systems default to manual control – the ultimate fallback when there are critical failures.

Even double-redundant systems can be completely taken out by say, a solar flare, or a massive electromagnetic pulse; something as simple as a direct lightning strike can do it. When that happens, the human backup needs to be completely in sync with the current status of the system, whether it’s a driverless car negotiating a tricky curve, or a jet on autopilot.

Part of the problem is, we’ve come to trust and rely on automated systems to such an extent, that we’re completely taken by surprise – and confused – when they don’t work as expected. After all, they hardly ever go wrong. But when they do, we’re often at a loss.

When things are running smoothly, as they do 99.9% of the time – that’s our service level guarantee – we get complacent. We know we can always take over in the event of a disaster, but if we never have to, how well are we prepared for that one-in-a-million failure? Our training needs to change.

In fact, the perfect automated system is one that employs fuzzy logic, and trips out occasionally, to keep us on our toes. This is one situation where zero defect is actually too little of a bad thing. We need a bit of unpredictability to keep the humans sharp.

Technology that never fails is not ideal – it creates the ultimate disaster when it does.

Links to related stories

Warning: Hazardous Thinking at Work

Despite appearances to the contrary, Futureworld cannot and does not predict the future. Our Mindbullets scenarios are fictitious and designed purely to explore possible futures, challenge and stimulate strategic thinking. Use these at your own risk. Any reference to actual people, entities or events is entirely allegorical. Copyright Futureworld International Limited. Reproduction or distribution permitted only with recognition of Copyright and the inclusion of this disclaimer. © Public domain image.

Like this article?

Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkedIn
Share on pinterest
Share on Pinterest

Read another Mindbullet

Investment in your future now depends on your elevator pitch!
Dateline: 19 December 2021
University education has become a game of loans. Student debt has exploded and increasingly, in the last few years, some of the best and brightest students got accepted into elite schools, only to fail to arrive on orientation day, because they couldn’t afford the tuition. There were only so many scholarships after all. Online courses...

Sign up to receive news from the future