102 lines
5.1 KiB
Markdown
102 lines
5.1 KiB
Markdown
+++
|
||
date = "2017-12-08"
|
||
title = "Bug severity explained"
|
||
tags = ["software", "engineering", "craftsmanship"]
|
||
description = "Classifying issues or bugs it never an easy task, but there are actually some very good guidelines out there you may have missed."
|
||
+++
|
||
|
||
Recently I got an email asking me about how I classify bugs and issues and how the different
|
||
categories for a bug's severity, like _Critical_ and _Major_ can be explained clearly.
|
||
|
||
I'm a software engineer and for the longest time have I approached everything in my
|
||
work as a _software engineering problem_. Bug reports are one of them. Any bug report
|
||
is a report on how software does not behave like it should.
|
||
|
||
> The overview page does not load < 2 seconds
|
||
|
||
or
|
||
|
||
> Fix the calculation of the thing to reflect real world exchange rates
|
||
|
||
Bug reports are not about your software. Bug reports are about users and stakeholders
|
||
_using_ your software to perform a task. And a bug report tells you that a user
|
||
cannot perform that task in an acceptable manner.
|
||
|
||
## How do _real_ engineers do it?
|
||
|
||
Computer programmers have a tendency to reinvent the wheel every few months. It's surprising to
|
||
see new ecosystems (looking at you, JavaScript) reinvent a lot of unix principles that have been
|
||
around for over forty years.
|
||
|
||
So, let's take a look at how the non-computer engineers tackle the categorization of bugs and issues.
|
||
I found that [ProQC](http://blog.proqc.com/classifying-defects-is-it-major-minor-or-critical/) has a
|
||
nice write up on the topic. Note that their guidelines are aimed at _physical_ products, but there's
|
||
a sensible parallel to software products.
|
||
|
||
### Critical
|
||
|
||
> Any condition found which poses the possibility of causing injury or harm to,
|
||
or otherwise endangering the life or safety of, the end user of the product or others in the
|
||
immediate vicinity of its use.
|
||
|
||
Luckily for most software engineers, nobody dies or gets injured from your software not working
|
||
correctly (although many stakeholders want you to think otherwise). This is probably true for
|
||
99% of all web applications. That does not mean that bugs cannot cause _harm_ in other ways.
|
||
|
||
Critical issues result in (most likely) financial damages for you or your customer.
|
||
|
||
* Missed business opportunities
|
||
* Waste due to product inavailablity (imagine an office of 100 people thumb twiddling because the website is down)
|
||
* Fines because of failure to comply with rules and regulations
|
||
* Customers leaving or seeking compensation for their damages
|
||
|
||
Basically any issue that results in real word damages should be classified as critical. Of course, this
|
||
is related to your terms of service or service level agreement.
|
||
|
||
### Major
|
||
|
||
> Any condition found adversely affecting the product’s marketability and sale-ability or adversely
|
||
affecting its required form, fit or function and which is likely to result in the end user returning
|
||
it to the source from which is was purchased for replacement or refund.
|
||
|
||
Imagine you buy a new iPhone and when you unpack it, it has a scratch across the screen. You don't want
|
||
that, although the phone probably works just fine.
|
||
|
||
In software I would categorize this as any issue that prevents a user from correctly or efficiently performing
|
||
a task. It's closely related to critical issues, but the financial impact is limited.
|
||
|
||
### Minor
|
||
|
||
> Any condition found which while possibly less than desirable to the end user of the product, does not
|
||
adversely affect its required marketability, sale-ability, form, fit or function and is unlikely
|
||
to result in its return to the source from which it was purchased.
|
||
|
||
This will include most software issues that you would call _annoyances_ or improvements.
|
||
|
||
### Trivial
|
||
|
||
Trivial is not mentioned in the ProQC list, which kind of makes sense. A physical product is impossible
|
||
to update without substantial cost. Software, however, can be updated frequently and strategies like _continuous
|
||
deployment_ make rolling out these changes almost painless.
|
||
|
||
I'm not a fan of the _trivial_ category. It's just too confusing for everyone involved. Trivial has too many
|
||
different meanings to different people.
|
||
|
||
* this is a huge issue and should be fixed asap
|
||
* this fix is so easy
|
||
* this issue is not a big deal, but I had to report it anyway
|
||
|
||
Any bug you'd classify as _trivial_ has either a critical, major or minor impact on the user, so classify it as such.
|
||
|
||
## Quantitative measurements
|
||
|
||
It's tempting to add metrics to each category's description. Of course you need to specify what you expect
|
||
your application to do, but a description like, "It's a major issue if the overview page takes longer than 10 seconds
|
||
to load," will not hold up. Aside from how and who measures this, what if the load time is a consistent 9.5 seconds?
|
||
|
||
The question should always be: in what way if the user impacted with this defect? Does it cost them money, will they
|
||
stop paying for your SaaS subscription or is it an annoyance that can be optimized later?
|
||
|
||
I hope this post was helpful to you to get a grip on how to categorize bugs and issues. This is by no means
|
||
a definitive list and I believe that it's a good practice to regularly evaluate which categories you use and
|
||
how you use them. |