“Regression testing” is one of those phrases where you’d be forgiven for thinking, “…what testing?”
It’s an important testing type within QA (Quality Assurance), especially with software projects: when you’re testing newly-introduced code or services and they work, but you need to make sure they haven’t broken any other part of the site. It’s all too easy to only test that new thing you’ve built, without noticing that, oh, now your donation journey doesn’t work because New Thing has broken your buttons. (“Golden Path” testing, where you only test the things you’re building, is another topic altogether…)
Mark Bridgeman, Fat Beehive's Head of Technology, receiving a positive result when checking our automatic tests. Believe it or not, this makes him incredibly happy.
UI Regression Testing is for those times when, while the core functionality of your site may be unaffected by code changes, the User Experience (UX) or User Interface (UI) of your website could have altered without you knowing it. We set up automated tests that run whenever a change is made to your site, so we can pick these things up before your users do.
All of Fat Beehive’s Hosting & Support customers have at least one automated UI test set up. Generally, this will be a trip through the different journeys on their site: home page, navigation, event listings, and then a donation journey (up to, but not including, payment or being sent to a 3rd party site). The bread and butter elements of most charity websites. It might be helpful to maintain a regression testing suite to check repeatable functions just after a new build, code fix or in an additional change to the system.
So, Fat Beehive runs 100s of automated tests every day (at non-peak times) with the results posted to one of our internal boards, where developers keep an eye on the results. If a test fails, it’s flagged and we investigate. Fortunately, this largely occurs if a page layout has changed due to content being added, causing a screenshot comparison to fail. We review, compare, and then accept the new baseline.
It takes us 5 minutes, but gives us peace of mind.
Fat Beehive runs 100s of automated tests every day (at non-peak times) with the results posted to one of our internal boards
