For Mission-Critical Systems the Devil is in the Detail
30.07.2020Pareto’s Principle, or the “80/20 rule”, is another way of expressing the English language saying, “The Devil is in the detail”. Loosely stated it says that for 20% of the effort we get 80% of the results. When designing solutions for the most critical users Goodmill knows that 80% isn’t good enough. And that to achieve the remaining 20% of results - all the details which matter - takes 80% of the effort.
Goodmill pays attention to detail
A good example of how this shows up in Goodmill’s solution is in how we handle Over-The-Air (OTA) upgrades of our routers.
We aren’t alone in being able to upgrade the running firmware of our routers over-the-air. In fact, if you’re a professional fleet operator we would advise that you don’t even consider a solution which doesn’t give you OTA upgrades. Your vendor will produce upgraded firmware from time to time, with new features and bug fixes. You will want to upgrade your router fleet fairly regularly, and you don’t want to have to visit each router every time you need to do that. At Goodmill we have applied the mission-critical design ethos to every aspect of our solution. Including the Over-The-Air upgrade mechanism.
Poor OTA upgrading is a significant source of service downtime.
Upgrading a remote router over-the-air is an inherently risky operation. Slow and unreliable connections can make the downloading process tricky. Errors can be introduced into the new firmware image as it is downloaded over the radio network, since no radio network has a 0% error rate.
We can check for errors on reception and catch the vast majority of them. But even then, there are other things which can trip us. What if the new firmware has some problem or bug? It’s not impossible. And you should be suspicious of any vendor who looks you in the eye and tells you that they will never deliver you firmware with a bug. They will. All software is prone to bugs.
To anyone who thinks this admission signals something negative I would ask this question|:
Would you be happy to fly with an airline who told you that they don’t give safety briefings or supply oxygen masks, life-rafts and life-vests because they never crash?
Of course you would not.
Bad things happen. Sometimes just bad luck. Sometimes human error. That’s life. Not anticipating and preparing for them is negligent.
So, what happens when you download a new firmware image to your router fleet and some - or all - of your routers don’t come back online after reboot?
In many cases you have no choice but to go to those routers. Physically, wherever they are, and troubleshoot them.
With Goodmill routers we have practically eliminated this kind of problem.
Even in the extremely unlikely event that a newly upgraded Goodmill router doesn’t come back after reboot a “hardware watchdog” – a dedicated piece of circuitry, not dependant on the firmware – is monitoring whether the router is running. If the hardware watchdog senses that the router hasn’t properly booted-up after a couple of minutes, it takes control and reboots the router again. Only this time using the last known-good firmware. So, the worst which can happen in any practical circumstance, even if the new firmware won’t run, is that the router operation will be disturbed for a few minutes and then it will come back to its previous working state on the old firmware.
Goodmill Systems multi-network broadband is designed and built from the ground-up for the needs and expectations of the most demanding critical users.
We are already delivering uptime and reliability benefits to police, fire, ambulance, military and business-critical users in many countries. Eliminating downtime. One detail at a time.
Goodmill Systems can help you to rollout your mobile broadband services with confidence because we know that for critical users there is no substitute for being "Always Online".