A product team makes what appears to be a small change to its pricing logic.
The requirement is straightforward: enterprise customers in one market need a different discount calculation. Engineering changes a shared pricing function, tests the new calculation and confirms that the enterprise checkout produces the expected total.
The release goes live.
Several hours later, support notices something unrelated: subscription renewals are failing for customers in another market.
The new discount works perfectly.
The problem is that the same pricing function is also used by the renewal workflow, where a slightly different assumption applies. The team tested what it intended to change but not what the change could unintentionally affect.
That is the problem regression testing is designed to solve.
Regression testing checks whether software that worked before still works after the application has been changed. The change might be a new feature, bug fix, dependency update, refactor, database modification or configuration change. The important point is that testing does not stop at verifying the new behaviour; it also checks for unintended damage to existing behaviour.
DORA's guidance on continuous integration recommends running automated tests whenever changes are integrated specifically so teams can detect regression bugs quickly rather than discovering them after release.
For Levelworks, regression testing is central to Tested App Health. It also supports Seamlessly Updatable because an application becomes much easier to maintain when teams can make changes with evidence that existing functionality has not quietly been damaged.
The fastest way to make regression testing inefficient is to begin with:
Run every test we have.
Sometimes a full suite is appropriate, particularly before a high-risk release. But good regression testing begins with understanding the impact surface of the change.
Suppose engineering modifies customer-address validation.
Obvious tests cover:
Less obvious affected workflows might include:
The first regression-testing question is therefore:
What existing behaviour depends directly or indirectly on what we changed?
Code review can help identify this impact. Google's engineering practices recommend evaluating not only whether a change performs its intended function but also its interactions with the rest of the system, edge cases and appropriate unit, integration or end-to-end tests.
A useful impact review examines:
The more shared the component, the wider the likely regression surface.
Not every existing feature deserves equal testing after every change.
A CSS adjustment to a settings screen should not trigger the same regression effort as changing authorization logic shared across every API.
A practical regression strategy considers two dimensions:
Likelihood: How likely is this change to affect the feature?
Impact: How serious would it be if that feature broke?
That creates a useful priority model:
Existing behaviour
Likelihood of impact
Business impact
Regression priority
Code directly modified
High
High
Highest
Critical workflow using changed component
Medium–High
High
Highest
Adjacent functionality
Medium
Medium
High
Independent critical workflow
Low
High
Moderate
Unrelated low-value feature
Low
Low
Lower
This is more effective than giving every test equal importance.
For a checkout change, payment, pricing, inventory and order creation probably deserve strong regression coverage. An unrelated profile-avatar workflow may not require the same attention unless the underlying change genuinely touches it.
Risk-based selection keeps regression testing useful as the application grows instead of allowing every release to require an ever-expanding manual checklist.
Regression tests can exist at several levels.
A calculation defect might be protected with a unit test. A database interaction may require an integration test. A critical checkout journey may deserve an end-to-end test.
The mistake is attempting to protect everything through the user interface.
Martin Fowler's discussion of the Practical Test Pyramid recommends pushing tests toward lower levels where they can provide the same confidence, because lower-level tests generally execute faster and isolate failures more clearly. Higher-level tests remain useful where they verify behaviour that lower-level tests cannot establish.
A balanced regression suite might therefore look like this:
Test level
Best used for
Unit
calculations, validation, domain rules
Component/service
service behaviour in isolation
Integration
databases, APIs, queues, service interactions
End-to-end
a small number of important user journeys
Exploratory
unexpected interactions and newly changed areas
The point is not to follow a particular ratio mechanically.
It is to avoid making every regression check an expensive end-to-end test when a faster test can protect the same behaviour reliably.
One of the highest-value sources of regression tests is the application's own defect history.
If a production bug revealed that two simultaneous requests can create duplicate orders, the fix should usually be accompanied by a test reproducing that condition.
The process is straightforward:
The application has now converted a production failure into permanent test knowledge.
Google's code-review practices recommend keeping test changes alongside the behaviour they protect and ensuring that tests would actually fail if the underlying code were broken. (Google Engineering Practices)
This prevents teams from repeatedly learning the same lesson.
[Internal Link: “Why the Same Bugs Keep Coming Back and How to Stop Them”]
The highest regression risk often appears in code that looks small but is widely reused.
Examples include:
Changing one shared function may affect dozens of workflows.
Before modifying shared code, identify its consumers and select representative regression tests from the important paths.
This is one reason a change's size in lines of code is a poor measure of risk.
Changing three lines inside a central authorization function may deserve more regression testing than adding 500 lines to an isolated reporting feature.
Regression scope should follow dependency and impact, not simply code volume.
Application updates often change more than source code.
Suppose a release makes a database column optional.
The new feature works with newly created records, but older production records follow different assumptions and begin failing elsewhere.
Regression testing should therefore consider:
DORA's guidance on database change management recommends managing database changes through the same disciplined delivery process as application changes because database modifications are a significant source of deployment risk.
The relevant regression question is:
Will existing data still behave correctly after this update?
Testing only newly created data can miss exactly the condition existing customers will encounter.
A regression test is only as useful as the conditions it exercises.
Consider a search feature that performs correctly with 50 test records but fails after a change when working with millions of records.
Or an account migration that passes with perfectly formatted test users but fails on older accounts containing null or legacy fields.
DORA's test data management guidance emphasizes realistic test data for validating important user journeys, edge cases, defects and error conditions, while also cautioning against simply copying sensitive production information into test environments.
Regression datasets should therefore deliberately represent:
The objective is not to reproduce production exactly.
It is to reproduce the characteristics capable of exposing the regression.
Regression suites tend to grow.
Every bug adds a test. Every feature adds more scenarios. Eventually a suite that once took five minutes takes two hours.
Teams then begin avoiding it.
Tests run only before major releases, failures are tolerated because "that test is flaky anyway," and developers receive feedback long after they have mentally moved on from the change.
At that point, regression testing loses much of its preventive value.
DORA's test automation guidance recommends fast, reliable automated suites integrated throughout delivery because delayed feedback makes defects more expensive to diagnose and turns testing into a release bottleneck.
Maintain the suite like production code.
Remove redundant tests. Move scenarios to lower levels where possible. Investigate slow tests. Repair or remove tests that fail unpredictably.
A regression test that nobody trusts provides very little protection.
Imagine a regression suite contains 800 tests, but 15 fail randomly on most runs.
Developers learn to rerun failures until the pipeline turns green.
Then test number 16 fails.
Is that the regression introduced by this release or another unreliable test?
Flakiness reduces the signal of the entire suite because engineers gradually stop treating failure as evidence.
A healthy regression suite should make a failing test noteworthy.
If failure becomes normal, the suite can no longer function as a release gate with confidence.
Common sources of flakiness include:
Fixing those conditions is part of maintaining the regression system, not optional test-suite cleanup.
Regression testing is particularly well suited to automation because many checks need to be repeated after every relevant change.
DORA recommends automated testing throughout the delivery lifecycle and notes that manual regression testing becomes slow and expensive as systems and release frequency grow.
Useful candidates for automated regression coverage include:
Manual testing still has value, particularly for exploratory investigation, complex UX behaviour and situations where human judgement matters.
The goal is not to automate everything.
It is to avoid spending human attention repeatedly verifying behaviour a deterministic test can check reliably.
[Internal Link: “Automated vs Manual Testing for Application Maintenance”]
The longer feedback takes, the harder a regression is to investigate.
If a developer changes pricing code at 10 a.m. and a regression test fails five minutes later, the likely cause is relatively easy to identify.
If the same regression appears in a large release test three weeks later alongside fifty other changes, diagnosis becomes much more difficult.
DORA's continuous-integration guidance recommends running builds and automated tests on every integrated change and fixing broken builds promptly.
This does not mean every test must run on every commit.
A practical pipeline may use layers:
Commit: fast unit and targeted tests.
Integration: wider component and API regression suite.
Release candidate: broader critical-workflow tests.
Post-deployment: small smoke checks against the deployed system.
The faster tests provide early feedback, while broader suites increase confidence as the change approaches production.
[Internal Link: “CI/CD for App Maintenance: How Automated Releases Reduce Risk”]
An application changes continuously, so a regression suite cannot remain static.
Review it periodically.
Ask:
This last question is particularly useful.
If customers discover an important regression that the suite completely missed, do not stop at fixing the application.
Ask what the test strategy failed to understand.
A missed regression is information about both the product and its safety net.
Before releasing an application update, a useful workflow is:
The goal is not maximum test volume.
It is enough evidence to make a responsible release decision.
Every software change has two requirements.
The obvious one is:
The new behaviour must work.
The second is easier to forget:
Everything that should remain unchanged must still work too.
Regression testing protects that second requirement.
For Levelworks, this is what makes Tested App Health important beyond conventional QA. A strong regression strategy gives teams confidence to maintain, patch, refactor and improve an application without treating every update as a gamble.
It also supports Seamlessly Updatable App Health. When regression coverage is fast, trustworthy and aligned with the application's real risk, teams can change software more safely because they receive evidence about unintended consequences before customers do.
The aim is not to prove that an update changed nothing.
It is to prove that it changed what you intended—and not what you didn't.
