Several years ago I found myself helping an engineer facing a production issue. This was a P0 affecting an entire country. It was related to how iOS Push Notifications were set-up. Without notifications, phone calls wouldn’t be triggered and this meant that patients would miss their doctor’s appointment. It was bad. I spend four hours until late in the evening with him in the office. We were in constant communication with the Project Manager in the country that was affected. Eventually we got it done. Everyone was ecstatic - myself, my colleague, our PM and our customer.
I felt like a hero. I wasn’t part of that group of work, but had jumped at the opportunity and helped push it through. My manager looked at me as a hard working employee and commended me in private. But so did the PM I barely knew. I felt good about the situation.
Looking back, we lost the forest for the trees. There was little to be proud of. It was a silly mistake that could have been caught with some basic testing and a planned release. None of that was done. Corners were cut to get something out. Myself and the other engineer weren’t heroes at all. We were the villains. We both failed to do proper engineering, which led to a few hundred people missing their appointments. But still hailed as hard working, reliable employees.
These days I don’t look at who rises to the occasion - there’s a place and time for that. Rather how and why we find ourselves in a bad situation.