Be on the same page with us
Subscribe to our latest news
By clicking the button Subscribe you give a permission for the automatic sending of e-mails

"Testing's job is to look all over the place for unexpected, surprising things". Interview with Michael Bolton

With this blog post Solvd starts series of interviews with world's most famous QA & Development specialists. We have the honor of interviewing Michael Bolton today. He has almost 30 years of experience in testing, developing, managing, and writing about software. Michael has delivered a lot of workshops, tutorials, and conference presentations on Rapid Software Testing and other aspects of testing methodology on all five continents. Many people in QA industry call him software tester #1
Posted by Slava Smirnov
April 20, 2020
SOLVD: Michael, thank you so much for being our guest today! First a bit of a philosophical question: do you think testing as a matter could predict or somehow prepare society to such dramatic threats as the coronavirus pandemic? Does better software support people in current fight with the virus and future challenges?

The answer to this philosophical question depends on your philosophical stance on testing. If you believe that testing's job is to confirm that everything is okay, then testing won't ever prepare you for anything much. I believe that the purpose of testing is to see the actual status of things, with a special focus on finding trouble. Testing's job is to look all over the place for unexpected, surprising things. The tester's job is to question assumptions that other people believe are safe.

Testing doesn't prevent very many kinds of problems or events, but it can help to prevent unpleasant surprises. Testing can help to prevent oblivion, or unawareness. Testing can help us to avoid being fooled by the idea that everything is okay. Testing can help us figure out how many people are actually infected with a virus — and testing can help us to find out if there are problems in our testing, too.

Software can help with all kinds of things, and better software can help better. It's important to remember, though, that "better" means different things to different people. Software amplifies our capabilities, whether that means liberating people or controlling people, helping people or hurting them. That's why ethical considerations need to be part of our evaluation of software. We must not limit ourselves to asking questions like "is there a coding error here?" or "does the software not do what the requirements document says it should do?" For testing to be relevant, we must ask broader questions: "What could possibly go wrong?" "Is there a problem here?" "Is everybody okay with all of this?".
SOLVD: How does the pandemic influence your training courses? Does it make sense to teach people software testing remotely? If so, what instruments you do use?

My preference is to be in the same room as the people I'm working with. That's not going to happen for a while. However, my colleague James Bach has lots of experience teaching Rapid Software Testing Applied online, and I've co-taught with him on a number of those. Recently we experimented with Rapid Software Testing Explored and Rapid Software Testing Management online, and those were surprisingly successful. We'll be doing lots more of those.

We've been using GoToMeeting for the presentation of webinars. Lately it seems to us that Zoom has better support for interaction with participants, and for some of the other things we'd like to do online. Of the remote meeting services we've looked at, support for group text chat is dreadful across the board, so we use generally a Slack-like chat product called Mattermost for discussions during the class.
SOLVD: Do you remember the time when you decided to become software tester? What in particular made you think of this profession?

I've been a tester all my life. Everyone is born a tester. Don't believe me? Find a very young kid, and observe how the kid apprehends the world: as an explorer, an experimenter, and as someone who is constantly learning and evaluating things through experience. Trouble is, lots of kids get that socialized out of them by various means, mostly prominently by many school systems.

My first technology-focused job was doing programming for a recruitment firm. In 1990, I started working as a technical support representative for the Canadian office of Quarterdeck Office Systems from 1990. In 1994, I was invited to Quarterdeck's head office to test and to co-ordinate testing work for the DESQview/X project. That quickly turned into a program management job, and the most interesting part of that work, for me, was the testing and test-related dimensions of it.

After I left Quarterdeck in 1998, I dabbled in programming and requirements work, but testing maintained a powerful pull. I started working with Ross Collard, which led to me working collegially with Cem Kaner, which led to me working with James Bach starting in 2003. Somewhere in there, I started my studies with Jerry Weinberg, in person, with his writing. Cem's support and vision, and then especially Jerry's and James', pretty much set the hook for life.

SOLVD: As a software tester, do you feel like a missionary, ambassador of users, quality warrior? Can you call QA an honorable role which comes as vocation or a calling?

I am a tester. As a tester, I do not assure quality. I wish people would stop talking about testing as "quality assurance". Testing isn't quality assurance, and it cannot be. Read more here.

If we think of testing as "quality awareness" or "quality assessment", we're on firmer ground.

I test in order to find problems that threaten value for users; sure. But I test in order to find problems that threaten value for everyone and anyone who might matter. There are some things that might be perfectly okay for the end users of a product, but that might be a big problem for in-house developers who use the product's API. There are some things that might be fine for both of those, but that might represent a problem for the operations people inside our company.

Is testing a vocation? I don't know about that. I know that if I find problems in a product — at any stage of development — and bring those problems to light, then the developers and managers have a chance to address those problems before they go any further. That's good enough for me.

SOLVD: As you train people with your progressive Rapid Software Testing methodology, does this mean that your methods could make every developer a good software tester?

We like to believe they'd help. Rapid Software Testing is about learning and finding problems under time pressure, and we can definitely help people to learn how to do that. But for programmers, there are two related challenges.

One is the problem of critical distance. I appeal to your own experience. I bet you've noticed this: it tends to be relatively hard to find problems in stuff that you've been working on yourself. We're all systematically blind to certain kinds of problems in things that we're building. That's because we're at close critical distance; we have models in our heads that are focused on getting something built, and it's really hard to switch to the mindset where we ask ourselves, "Okay… now what could go wrong?"

Shallow testing — the kind of testing you need to check whether the product you're building is the product that you think you're building — isn't too hard at close critical distance. Deeper testing — long-form, naturalistic, interactive testing that's farther from the builder's mindset — is harder at close critical distance. You need deep testing to find the hidden, rare, subtle, intermittent, emergent bugs. For that, you need someone at farther critical distance from the builder's mindset.

A related problem is that programmers, practically by definition, like to program. If they had wanted to be testers, they would have chosen to become testers; it's not like there are lots of barriers to entry into testing for skilled technical people! Programmers tend to be focused on programming stuff, and, in general, less interested in the customer's domain whatever that might be. Again, if programmers were interested in other stuff, like being testers, they'd be probably be doing that.

Put those things together, and you get this: while programmers might struggle to switch mindsets and find problems in their own code, they tend to be pretty good at finding programming-related problems in other programmers' code. But programmers tend to be less interested in the nuts and bolts of deeper testing. Programmers don't mind seting up programming environments; they're less interested in setting up test environments. They're good at fixing problems from the field, but (mostly) not so interested in processing problem reports from the field and trying to reproduce them in-house. And in order to maintain focus on programming, programmers don't always have the time or the inclination to make a deeper study of the ways that their clients do work. Testers don't always do that either, of course — but I think they should.

In Rapid Software Testing, we talk about the Tester Heuristic: if you want someone to focus on the skill set and mindset of testing, have someone in the testing role. Make deep testing someone's specific responsibility, and that person can then apply focus and commitment to it. If you want deep testing to disappear, make sure that no one person has responsibility for it. If you want to make sure that deep testing doesn't happen much, assign it to a programmer who's not interested in it or committed to it.

SOLVD: Many people now say that after pandemic this world will definitely face dramatic changes. Are you going to adapt your QA approaches as well?

There are two schools of thought about the pandemic: one says that we'll quickly get back to normal; the other is that everything will change. We all should have learned by now not to predict the future, but we can prepare for it. Rapid Software Testing is intended and designed to be adaptable to any context. "Designed" means that it deals with general principles and things that we can anticipate; adaptable means that the details can be changed to deal with changes in context. So: testing will always be testing, but our context shapes our choices, and all of those things evolve over time.

SOLVD: Thank you very much, Michael!

You can follow Michael on Twitter and LinkedIn