Who can declare an incident?Best Practices
”Can I declare an Incident?”
It’s one of the most common questions that new employees and new adopters of incident management ask themselves. Something has happened— it’s big. The site is going down, the infrastructure is falling over, the revenue is falling off a cliff… and you’re the first one to notice it.
What do you do?
The Old Way: Only a Few can Declare an Incident
If your company doesn’t practice modern incident management, incidents may still only be implicitly defined by senior staff members through traditional communication channels.
In this less than ideal world, incidents are often represented by email threads that start out with a series of managers, and grow to include everyone from executive stakeholders to individual contributors who are being added to “fix it!”.
Very often, the threads are started by senior employees and managers because only they have the social reach within the company to raise awareness of the fire and to declare its legitimacy. Further, because these threads are so disruptive, they’re considered negative events and employees also are not equipped to take on the risk of “wrongly” initiating a panic through incident declaration.
Worst of all, if you’re not on the thread or in the channel, you likely don’t even know the incident is happening.
This process tends to be self-reinforcing: there’s rarely any desire to expand the group of people who can declare an emergency because the email thread that results is painful for everyone who is on it and often spiders out into phone calls, side-chats, and other noisy communications that do a poor job of connecting to the progress towards resolution.
The New Way: Anyone can Declare an Incident
Modern incident management is more inclusive and states that anyone in an organization can declare an incident. Moreover, it is encouraged to declare incidents rather than frowned upon because incident management is considered a positive process.
This approach actually takes its cues from the longstanding physical practice of what companies often call “the big red button.”
You may have seen these large, red buttons, present on machinery and factory floors. In fact if you’ve ever pumped gas you’ve likely seen the big red “emergency shutoff” button.
You’ll notice these buttons are available to everyone who can reach them, not just management (or the person in the booth at the gas station). This is for one reason: it turns out that if anyone thinks there should be an emergency declared, the safest thing to do is to take their word for it and declare the emergency.
At Kintaba, we’re believers in the modern approach being applicable to all companies as much as it has been applicable to physical safety for decades.
How tooling helps lower the barrier to incident management
Of course, the barriers to entry that the “old way” present might make your company think it’s a bad idea to let anyone declare incidents. No one wants giant email threads flying around at all hours.
Tools like Kintaba allow you to:
Treat incident management as a positive set of processes rather than a negative set of interruptions
You can keep your progress towards resolution open within the company beyond just those subscribed to an email thread or slack channel, reducing status request emails and confusion as others are dropped into the conversation or seek to understand what's happening.
Declare IMOCs (Incident Manager On-Calls) and limit the scope of those contacted to the relevant parties.
The IMOC plays multiple roles of orchestration, both as a connecting entity to the rest of the organization if the incident owner doesn’t already have those relationships, as well as someone who can provide guidance on how the incident should be handled after it has been created.
The IMOC can be both a tempering force and an amplifier if needed. Read more about IMOCs here.
Manage responders and subscribers outside existing unwieldy comms channels
Rather than @‘ing a channel in slack or sending an email to a group of managers, Kintaba lets you bring the right people in based on their oncall rotation.
Provide a clear history and timeline for late-arrivals, as well as information on the current responders
Each incident has an individual owner who is responsible for the resolution, which can be passed from person to person but whose history is tracked for clarity.