A Good Leader Must also be a Good Manager
October 2000. I got into the office early that morning and already had a voicemail to get to an 8:30 meeting. I quickly found out that a distribution system that was running for about a year had maxed out the bank account that drew funds to make these payments. I was put in charge of a task force to find out what happened and get it fixed. It didn’t take long to determine that data from one system was being transferred to another system that was paying the money out. The problem: when the payment system took the records in, there was no confirmation to determine whether the output equaled the input in the number of records or amount of money…nor were there any other validations. While one might expect a check and balance that went to a more granular level, at the very least, there should have been a systematic check at the macro level. And, that was only the beginning. The distribution system was designed so badly, it was making duplicate payments – and sometimes to people who shouldn’t have been paid at all. I came to call it the Chernobyl System*, after the shoddy Soviet-era nuclear power plant that had no containment system. It blew up, contaminating millions of people with radioactivity and creating a forbidden zone with a radius of at least thirty miles that will remain uninhabitable for at least 180 years, and some scientists say 3,000 years.
Fast forward to Monday, January 13, 2018. Vern Miyagi, the man in charge of the Hawaii Emergency Management Agency (HI-EMA), standing side by side with the Governor of Hawaii, had the courage to say, “This was my fault.” The procedures in place allowed one single individual to “push the button” to issue a live alert that there was an incoming missile minutes away from impact to Hawaii. What!?!?!! Yes, you heard me right. No checks or balances or confirmations. One person can make this error, and it can go through to strike fear and panic in the hearts of 1.4 million Hawaiians and tens of thousands of tourists.
Hacked to Death
C’mon, stuff like that is one in a million. Really, let’s go back into the ancient history of
September 2017. Equifax, the giant credit agency, announces it has had a data breach of
143 million Americans – including their Social Security Numbers, driver’s license data, security questions and answers and more. Equifax CEO, Richard Smith, who was forced to resign, testified before the House of Representatives Energy and Commerce Committee and said, “The human error was that the individual who was responsible for communication in the organization to apply the patch, did not.” One person – no checks, no balances, only errors…one huge error.
These are examples of inexcusable malfeasance and incompetence, yet it seems to go on more often than we could imagine in the most important organizations. Most managers are competent, but when they are not and when processes are put in place that lacks the appropriate necessary checks and balances, there should be a leader at the top and leaders up and down the line who are asking the right questions.
Check, Check and Double-Check…Really?
So, let’s get a few things straight. Checks and balances cost money. You don’t need to check everything all the time. You need to have the appropriate checks and balances in place. So, how do you know when to check and how often? There is no black and white answer to that, but let’s set down some sound rational ground rules.
When the distribution system I described at the top, the so-dubbed Chernobyl distribution system, was reprogrammed to have the appropriate checks and balances programmed to work systematically, we went forward to use it for the first time. When we did, I put in place manual stops, where we downloaded data and checked it manually in Excel. Each step of the process was manually audited in this way. It was tedious, laborious and time-consuming, but that was the first time we were using the revamped system, and after all, we had already lost millions of dollars, most of which could not be recouped. Think of the financial and political exposure of repeating the mistakes that were already made. We conducted the same validations the second time we ran it, confirming each step of the process, and we were verifying that the system itself was performing the checks and balances correctly. Now, we could have a reasonable expectation that it would run right and pay accurately. We pulled back many of these laborious manual audits and, as we went forward, we did spot checks and a reasonability analysis at the end of a run.
What are the ground rules?
When a process, an operation or a system is in place, here are some criteria to follow:
Criticality, Newness, Reliability
How critical is this?
What are the stakes? The more that’s at stake, the more checks and balances you need.
Legal – Are there legal considerations that could make you, your organization or others legally liable if something goes wrong?
Financial – Is there significant financial risk to the organization or are you dealing with monies that amount to a virtual rounding error?
Political – Will there be fireworks in the boardroom? Will a key client, a key board member or the public be negatively impacted in a significant way?
You can add to the list, but these are the main events, and each of these have a scale. You have to balance the scale between the risks and costs.
How new is your process/operation/system?
When something is new, it needs the checks and balances to ensure that it is running right. This may mean a number of checks along the way. Over time, as you have ensured that it is running correctly and accurately, you can pull back on some, even many of these checks, but you always need some verifications. AND, you must always keep criticality in mind.
We’ve run this over and over again without a single problem.
Don’t get lulled to sleep by this. Things change. All of sudden, they’ve replaced the server, or someone new is on the job. If something has been humming along, that’s great, but you still need to beware of the unexpected and still need to have appropriate checks and balances in place. AND, you must always keep criticality in mind.
*Note: In the wisdom of the senior management of this organization, the software designer who built what I called the Chernobyl distribution system was later promoted and went on to build another system with equally disastrous results.