Skip to content
lemmaBY AJS, home

All work

Lemma 1

Insurance Billing QA Framework

End-to-end test automation for a simulated insurance billing platform: a Spring Boot application, and the seven levels of testing that hold it to its rules.

Context
Solo project
Period
September 2026
Repository
abdeljalilsennaoui/insurance-billing-qa-framework(opens in a new tab)
The invoice console of the billing application: a table of nine invoices with customer, policy, total, outstanding balance and a status of unpaid, cancelled, overdue, partially paid or paid.
Figure 1. The application under test. This screenshot is generated by the UI test suite itself, through the same page objects the tests use.

Overview

The repository has two halves that are deliberately kept apart. One is the system under test: a Spring Boot billing application with customers, policies, invoices, installment schedules and an append-only transaction ledger, exposed through a REST API, a contract-first SOAP service and a web console in English and French. The other is the automation that exercises it at seven levels — unit, integration, API, UI, BDD, smoke and performance — and the pipeline that runs it on every pull request.

I built both halves. The billing domain is simulated; the automation, the pipeline and the defects found along the way are real.

Problem

Test automation is easy to demonstrate against a toy page and hard to do well against a real system. The difficult parts are the ones a small demo hides: who owns the test data, how to wait for a browser without sleeping, how a pipeline starts and stops an actual application, and how the numbers in the documentation stay true as the suite grows.

I wanted a system whose rules have right answers. Money has to reconcile to the cent, invoices move through states, and a payment can be refused for a legitimate reason — so every test asserts something that matters.

Goals

  • Check the same billing rules at several levels, from plain unit tests to browser tests, and be able to say which level catches what.
  • Keep the automation independent of the application code, so it tests the published contract and not shared classes.
  • Run the suites on every pull request, against the exact artifact that was built.
  • Report only figures that were measured, and label anything that was never executed.

Technologies

Application under testJava 25, Spring Boot 3.5, Spring Data JPA, Thymeleaf, Bean Validation, H2
Test automationJUnit 5, TestNG, REST Assured, Selenium 4, Cucumber 7, Cypress 15, AssertJ, JMeter
Build and CIMaven, GitHub Actions, JaCoCo, SonarQube Cloud, Codecov

Architecture

A multi-module Maven build. billing-app is the application. qa-api-tests, qa-ui-tests and qa-bdd-tests are the automation modules, with a Cypress smoke suite and a JMeter plan beside them.

The automation modules have no dependency on the application code. They reach it only over HTTP and through the rendered page. If both sides shared an enum, renaming a constant would change the test and the application together, and the suite would stay green while the contract had quietly broken.

Framework code — API clients, page objects, test data builders — lives in each module’s src/main/java and the tests in src/test/java. That lets the BDD module reuse the clients and page objects as ordinary library classes, so its step definitions contain no automation logic at all.

  1. Automation

    • qa-api-testsREST Assured, TestNG
    • qa-ui-testsSelenium 4, page objects
    • qa-bdd-testsCucumber 7
    • cypressSmoke suite
    • perfJMeter plan

  2. Published contract

    • REST API
    • SOAPContract-first
    • Web consoleEnglish and French

  3. billing-app

    • DomainBilling rules on the entities
    • ServicesLookup and transactions
    • JPA

  4. Storage

    • H2In-memory, seeded at start-up
Figure 2. Module boundaries. The automation never imports application classes; it only sees what a client would see.

My responsibilities

This is a solo project. I designed and wrote the application, every test suite, the pipeline and the documentation.

  • Designed the billing domain and implemented it in Spring Boot, with a REST API, a SOAP service and a bilingual web console.
  • Built the API, UI and BDD automation frameworks, plus a Cypress smoke suite and a JMeter load plan.
  • Wrote the six-job GitHub Actions pipeline, including coverage that is merged across jobs.
  • Documented the test strategy, 34 manual test cases, a traceability matrix for 33 requirements and 14 defect reports.

Every commit in the repository is mine. As its documentation says plainly, that also means the code was self-reviewed, not peer-reviewed.

Engineering decisions

Billing rules live on the entity

A guard in a service is bypassed by any other caller that reaches the entity; a guard inside Invoice.applyPayment is not. It also makes the rules testable as plain JUnit with no Spring context. The rules are checked cheapest-first — amount, invoice state, policy state, then overpayment — and a test pins that order. The cost is an entity that is larger than an anaemic model.

A zero payment is 422, not 400

The request object validates only presence. Whether an amount is positive, correctly scaled or within the outstanding balance is a billing rule, reported as 422 with a reason code. Without that split, a test could not tell “the client sent nonsense” from “the platform applied a rule”, and a regression that turned one into the other would pass unnoticed.

Money never rounds silently

Amounts are BigDecimal at scale 2, normalised with RoundingMode.UNNECESSARY, so a value that would need rounding throws instead of losing a cent. Three installments of 33.33, 33.33 and 33.34 settle a 100.00 invoice exactly, and an over-precise amount such as 10.001 is refused.

Explicit waits only

The UI module has no implicit wait and no Thread.sleep. Mixing implicit and explicit waits produces timeouts that belong to neither. Each browser session is held in a ThreadLocal and removed as well as quit, because TestNG reuses threads and a stale entry would hand a dead session to the next test.

The pipeline tests the artifact it built

The first job builds the jar and publishes it; the API, UI, BDD and smoke jobs download that jar and run in parallel, so they test the validated artifact and not a second build of the same source. They start the application with the same script a developer runs locally, and poll its health endpoint instead of sleeping.

Test counts are generated, not typed

A script reads the counts out of the reports from the last run and writes a test inventory; another fails the build if a figure quoted in the documentation disagrees with it. This is the fix for a defect I logged against my own documentation, where eight hand-maintained figures had drifted apart.

Testing and quality

600 automated tests: 176 domain and service unit tests, 160 Spring integration tests, 129 API tests, 67 UI tests, 53 BDD scenarios and 15 Cypress smoke tests. 596 of them run on every pull request; the four that are excluded reset the database and so must run alone.

Every test that changes data creates its own customer, policy and invoice through the public API. Nothing in the mutating suites touches the seeded records, which is what makes parallel execution safe.

Coverage is measured full-stack. The API, UI and BDD suites drive the application in a separate process, so an ordinary in-process run sees nothing of what they exercise. The pipeline attaches the JaCoCo agent to the running application and merges the results in a final job. There is deliberately no coverage gate.

Alongside the automation: 34 manual test cases, each marked automated or not, and a matrix tracing 33 requirements to the test methods that cover them.

Challenges

A green run that tested the wrong server

A stale instance was still holding port 8080. The new jar failed to bind and exited, the health probe was answered by the old process, and the start script reported the application healthy. The SOAP suite then failed with 404s against a jar that predated the SOAP endpoint, which pointed the investigation at the servlet mapping instead of the port. The fix refuses to start when something already serves the port, and checks that the process is alive before it checks health.

Healthy before the data existed

Seeding ran after the web server had started, so the application accepted requests and reported healthy with an empty database — exactly the window every suite lands in: wait for health, read an invoice, get an intermittent 404. Seeding moved to bean initialisation, on a separate bean so that it still goes through the transactional proxy.

A defect that full coverage could not see

Whether an invoice was overdue depended on the host’s default time zone. No test could catch it, because every test evaluated the rule in the same zone the code did. Static analysis found it, inside code that reported as fully covered. The billing zone is now configuration, and the regression test freezes a clock at an instant where UTC and the business zone disagree about the date.

Two wrong ways to wait for a navigation

Waiting for a condition on the destination page returned immediately, because the same element was already on screen. Waiting for an old element to go stale was correct in principle and failed on the CI runner, where Chrome sometimes raises a different error mid-navigation. The version that holds sets a marker on window before the action and waits for it to be gone with the document fully loaded. At no point was the answer a sleep or a retry.

An invoice detail page showing a red banner: “That payment of $10,000.00 exceeds the outstanding balance of $360.00.” Below it are the invoice totals, the payment history and the payment form.
Figure 3. A payment refused by a billing rule. The API reports this as 422 with a reason code, not as a generic 400.
The agent console listing policy terms in a table, ordered by policy number, with their amounts and a total row.
Figure 4. The agent console. It renders the same template fragment as the policyholder screens, and tests at three levels check that the two agree.

Results

MeasureResultSource
Automated tests600, all passing; 596 on every pull requestRepository README, version 1.2.0
Coverage, full-stack98.4% line, 87.6% branchRepository README, version 1.2.0
Static analysisSonarQube Cloud quality gate passing: 0 bugs, 0 vulnerabilitiesRepository README, version 1.2.0
Defects found and fixed14, four of them in the test infrastructureDefect log and test inventory
Requirements traced to tests33Traceability matrix
Continuous integration6 jobs; latest run on main succeededGitHub Actions, checked 12 October 2026
Load test1,050 samples, 0% errorsJMeter run of 15 September 2026

Table 1. These figures are quoted from the repository, which generates its test counts from the last run and fails the build when the documentation disagrees with them.

What this does not show

  • The load-test numbers are not a capacity measurement: the load generator shared a machine with the application, the database was in memory and the run lasted seconds.
  • The repository includes a Jenkinsfile that has never been executed on a Jenkins controller. GitHub Actions is the pipeline that runs.
  • The database is in-memory H2, and concurrent payments against the same invoice are untested.

End of case study