A web application firewall that is switched on tells you nothing about what it catches. Effectiveness is two numbers measured together: how much of each attack class the WAF blocks, and how much legitimate traffic it wrongly turns away. This article shows how to measure both, where an attacker repackages a payload to slip past a rule, and the tuning loop that moves both numbers in the right direction. It is the method behind our profile-level protection analyzer, APEA.
A WAF is not an on/off switch; it is a set of rules that has to be tuned to one application. Measuring its effectiveness is likewise not a yes/no — it is a measurement taken profile by profile, with evidence behind every number.
Why "on" is not enough
Every WAF ships with a default rule set. The OWASP Core Rule Set (CRS), version 4 (2024), is the open-source baseline most commercial WAFs build on. It is a strong starting point, but it does not know your application: which endpoint accepts free-form prose, which parameter always carries odd-looking data, which feature is never used at all. So a default install lands in one of two bad places: too strict, and it drives customers away; too loose, and it waves attacks through. Measurement replaces those guesses with numbers you can act on.
CRS scores each request: a matched rule adds to an anomaly score, and the request is blocked once the score crosses an inbound threshold (5 by default). That design is why a single clumsy rule can block real users.
Example — a false positive from the defaults. A content editor at an online shop saves a product note:
Use the SELECT switch to choose a size, then UNION both halves of the set.
At a higher paranoia level, the SQL keywords SELECT and UNION each trip a CRS rule, the score crosses 5, and the save is rejected with 403 Forbidden. Nothing was attacked. The rule simply does not know this endpoint accepts ordinary prose. Until someone measures it, that rejection is invisible on a green dashboard — the editor just sees "save failed".
Measure detection class by class
Effectiveness is never one overall figure, because a WAF can be excellent against one class of attack and helpless against another. We work by attack category, the way the OWASP Web Security Testing Guide (WSTG), version 4.2 (2020), groups them: injection, cross-site scripting, path traversal, OS command injection, file inclusion, and so on. For each category we send a set of known attack requests through the WAF and count how many were blocked and how many passed. The result is a map: where the WAF is solid, and where it has holes.
Example — one probe. Against a staging copy of the application, we send a textbook SQL injection to the login endpoint:
POST /api/login HTTP/1.1
Host: staging.shop.example
Content-Type: application/json
{"username":"admin","password":"' OR '1'='1"}
The WAF answers 403 and logs CRS rule 942100 (SQL injection). That single request is one data point. The class score comes from dozens of variants — different operators, comment styles, keyword cases and whitespace tricks — sent the same way, counting how many of each the WAF stops. A class that blocks 9 of 10 textbook payloads but 2 of 10 variants has a hole, not a score of 90%.
Each class is scored against a clear definition of what a result means:
| Attack class | Sample probe | A "block" means | A "pass" means |
|---|---|---|---|
| SQL injection | ' OR '1'='1 in a login field |
WAF returns 403 before the query runs | Payload reaches the database layer |
| Reflected XSS | <svg onload=alert(1)> in a search term |
WAF strips or rejects the request | Script is echoed into the page |
| Path traversal | ../../etc/passwd in a file parameter |
WAF rejects the traversal sequence | Request reaches the file handler |
| OS command injection | ; id appended to a hostname field |
WAF rejects the shell metacharacters | Command reaches the system shell |
| File inclusion | php://filter/... in an include parameter |
WAF rejects the wrapper | Wrapper reaches the include logic |
An important caveat: the attack testing must run against a controlled target you are authorised to test, not someone else's live system. The goal is to measure the WAF, not to exploit anyone.
Measure false positives on real traffic
The other half of the job is the customer. A WAF that blocks everything is 100% "secure" and 100% useless. So we measure the false-positive rate against a sample of the application's real traffic — in log-only mode, so nothing is actually blocked — and count how many healthy requests the rules would have stopped or slowed.
Example — a legitimate request wrongly blocked. Replaying a day of sampled production traffic, one request stands out:
POST /api/profile HTTP/1.1
Content-Type: application/json
{"bio":"I lead our \"ops\" team & mentor juniors; reach me at o'brien@example.com"}
It trips an XSS rule on the quotes and ampersand and a SQL-injection rule on the lone apostrophe, scoring 10 against a threshold of 5. A real customer saving that bio would receive a 403. This is a false positive, and because bio is a free-text field, it will hit many ordinary users, not one. Security and smoothness are two pans of one scale; without real traffic you cannot find the weight on either side.
How a payload slips past a rule
An attacker does not throw the raw attack at the rule; they repackage the same attack so it slips past, then rely on the application to unpack it again.
A large part of the assessment is exercising exactly these families:
| Family | What it does | Why it slips through |
|---|---|---|
| Encoding | Rewrites the payload with URL, Unicode or double encoding | The rule sees the raw pattern, not the encoded form |
| Content-type switching | Sends the data in a format the WAF inspects less thoroughly | Inspection for that content type is not fully configured |
| Parameter pollution | Repeats one parameter so different components read different values | The rule sees one copy; the app reads another |
| Oversized body | Places the payload past the size beyond which the body is no longer inspected | For performance, the WAF reads only the start of the body |
Example — an encoding bypass. The raw payload ' OR 1=1 -- is blocked, as above. We resend it double-URL-encoded:
username=admin&password=%2527%2520OR%25201%253D1%2520--
Now the WAF sees only percent-escaped bytes; its SQL pattern never matches, and the request passes. If the application decodes the value twice before using it — a common framework default — the injection runs against the database. The finding is not "the WAF is broken". It is precise: rule 942100 is not applied to the fully decoded form on this parameter. Every family that gets past the WAF points to one specific untuned rule, and the next step is to close that exact gap.
The tuning loop
An assessment is not the end point; it is the start of a loop.
First we run the WAF in monitoring (log-only) mode to see what it would have caught without blocking anyone. Then we narrow the noisy rules to the specific paths and parameters that need them, writing exceptions precisely and by target rather than turning a whole rule off. Then we review the anomaly-score threshold, and measure again. Each round raises the detection rate and lowers false positives, until both numbers sit in an acceptable place.
Example — one round. In monitoring mode, the WAF records what it would have blocked. A typical line:
[2026-09-20T10:14:02Z] would-block score=10 uri=/api/profile param=bio rules=942100,941130 client=203.0.113.45
Seeing param=bio light up across hundreds of distinct, legitimate clients tells us the fix exactly: scope rules 942100 and 941130 off the bio parameter on /api/profile only. We apply that exception, re-measure, and confirm the SQL-injection class still blocks on the login form where it belongs. Because the application and its traffic keep changing, this loop never fully closes; a periodic review keeps it alive.
What a finding looks like
A WAF-assessment finding names the gap, proves it, and says exactly what to change — never just "tune the WAF". A filled-in example:
| Field | Content |
|---|---|
| Title | SQL-injection rule not applied to the double-decoded password field |
| Probe | ' OR 1=1 --, sent double-URL-encoded in the login request body |
| Where it entered | POST /api/login, password parameter, on the staging copy |
| Blocked? | Raw: yes (403, rule 942100). Encoded: no — passed on 10 of 10 attempts |
| Impact | Authentication bypass on an endpoint the WAF was trusted to protect |
| Root cause | The rule matches the raw form only; the app URL-decodes the value twice |
| Fix | Normalise (URL-decode) before matching on this route, or add a decoded-form rule for password; fix the double-decode in the app |
How we run the measurement
- Test against a controlled staging target with realistic data, one you are authorised to test — never a live third party.
- Measure detection class by class, with many variants per class, and report each class separately rather than as one headline percentage.
- Exercise the bypass families — encoding, content-type switching, parameter pollution, oversized body — against the classes the WAF claims to block.
- Replay sampled real traffic in log-only mode to measure false positives before any rule is changed.
- Report each gap as a finding with its exact fix, then tune, re-measure and repeat.
Note: a WAF buys time; it is not a substitute for fixing the vulnerability in the application. Treat it as one layer of defence in depth, alongside input validation and output encoding in the code itself.
Common questions
How is a WAF assessment different from an application penetration test? A penetration test asks what weaknesses the application has; a WAF assessment asks how well the defensive layer in front of it catches those weaknesses. The two are complementary, and both belong in a report that names each finding and its fix (see what a penetration test report must contain).
Does a WAF replace patching the code? No. A WAF is a layer, not a final answer. It makes sense within defense in depth, alongside input validation in the application itself.
What does a "bypass" in the report actually mean? That a payload the WAF blocks in its raw form reached the application once it was repackaged — encoded, split across parameters, or hidden past the inspection limit. Each bypass maps to one untuned rule, so each has a precise fix.
How often should it be measured? After every significant change to the application or the rules, and periodically regardless — because both the application and the bypass techniques change over time.
Does this harm production traffic? No. The attack testing runs against a controlled target, and the false-positive measurement runs on a sample of traffic in log-only mode, without disrupting the service.
If you want to know what your WAF actually catches and what it lets slip, our WAF assessment, tuning and design work does exactly this, and APEA is the platform we run it on.
