Retest, Closure and the Evidence That Ends an Engagement

A penetration test report describes the state of a system on a particular day. On its own it is not evidence that anything is secure — it is evidence that somebody looked, and what they found.

What converts it into a closed engagement is the retest, and the documentation that comes out of it. This is the most frequently omitted part of a proposal and the most frequently assumed part of a purchase, which is an unfortunate combination.

Agree three things before the first test

All three are cheap to settle up front and expensive to negotiate afterwards, when your leverage is gone and the findings are already known.

• Is retesting included, or billed separately? Both are legitimate. Not knowing is not.
• How long does the window stay open? Thirty, sixty and ninety days are all common. Check it against how long your release process realistically takes — a thirty-day window against a quarterly release cycle will expire before the fix ships.
• Does the retest produce its own dated document? This is the one that matters most and is asked about least.

What a retest actually involves

The tester re-attempts each finding using the original reproduction steps and records the outcome. Three results are possible.

Closed. The original steps no longer produce the result. A competent retest goes a little further than that — trying obvious variations, because a fix that blocks one specific input while leaving the underlying flaw in place is common. If the fix for an injection was to filter one character, the retest should try the others.

Partially closed. The reported instance is fixed; the weakness class is not. This is the most useful result a retest produces and the one buyers most often misread as success. If a report listed three instances of one CWE, fixing three instances is not the same as fixing the habit that produced them.

Still open. Either not yet fixed, or the fix does not work. Worth distinguishing, because they lead to different conversations.

The closure document

The deliverable is a dated document listing every original finding with its current status, what changed, and how it was verified. It should stand on its own — a reader with no memory of the engagement should be able to follow it.

This is what an auditor asks for. Not the original report, which shows problems, but the closure document, which shows resolution. "We fixed it" is not evidence, and neither is a report with no retest attached; the question being asked is whether somebody independent confirmed the fix.

For anyone working to a framework with a testing requirement, the retest record is usually the artefact that satisfies it. Producing it a year later, from memory, is not possible.

Findings you are not going to fix

Every engagement produces some. The system is being decommissioned in four months. The fix requires a vendor who will not ship until next year. The remediation costs more than the risk.

These are legitimate outcomes, and the failure is not deciding against a fix — it is deciding informally. A finding that is quietly not fixed is indistinguishable, a year on, from one that was forgotten.

A recorded acceptance carries the finding and its severity, the reason, any compensating control, who accepted it, and when it will be reviewed. That last field is what stops an acceptance becoming permanent by default: a decision made because a system was being retired in four months should be revisited when it is still running eighteen months later.

Written down, this is a risk decision. Undocumented, it is an open finding with nobody's name on it.

Where the fix changes the system

Occasionally remediation is substantial enough that the retest is not the right instrument — a rewritten authentication flow, a re-architected permission model. Re-testing the original findings against a component that no longer works the same way tells you the old attacks fail, which is not the question.

New code deserves testing as new code. Worth raising explicitly when the remediation plan involves rebuilding rather than patching, because the scope of the retest was agreed against the original system.

The sequence, end to end

1. Test, and receive the report.
2. Re-rank the findings against your own context — see how to read a security finding.
3. Fix, in priority order, recording what was changed for each.
4. Decide and record acceptances for anything you are not fixing.
5. Retest within the window.
6. Receive the closure document.
7. Keep it with the original report. They are one artefact in two parts.

Then the question becomes when to test again, which depends on what changed and what you are subject to rather than on the calendar alone — covered in how often you should run a VAPT.