Is penetration testing manual or automated? (Both.)

It is the question we get most often from growing companies: is your penetration testing manual or automated? The honest answer is both, with a third layer in between that has changed what the first two are for.

Getting this right matters commercially, not just technically. A report that a customer's security team does not trust is worse than no report, because now they have a reason to keep asking.

Three layers, not two

The layers are not interchangeable and none is optional. Skip automation and you miss the obvious. Skip people and you miss everything that gets exploited.

What automated scanning is for

Scanners are excellent at coverage. They sweep broadly, flag known vulnerabilities and common misconfigurations, and they do it consistently across an estate no team could check by hand.

What they cannot do is understand what your application is for. A scanner sees inputs, outputs and status codes. It has no model of what a user is supposed to be allowed to do, so it cannot tell a working feature from a working feature that should not exist.

That is not a criticism. It is the boundary of the tool, and knowing where it sits is the whole point.

What AI actually adds

This is the layer with the most marketing noise around it, so here is the plain version of where it earns its place on a real engagement.

Attack surface mapping. Correlating subdomains, exposed services, certificates, code repositories and cloud assets into a picture of what you actually have. Tedious, high-volume, pattern-heavy work, and this is exactly where machines beat people.

Triage and prioritisation. A raw scan of a mid-sized estate returns thousands of findings, many duplicated across hosts and many irrelevant. AI-assisted triage groups them, strips duplicates, and ranks by asset context, so tester hours go on the interesting 5% instead of the first 500 rows.

Reading code at volume. Flagging suspicious patterns across a large codebase far faster than a person can read it: unsafe deserialisation, hardcoded secrets, missing authorisation checks on new routes. It surfaces candidates. It does not confirm them.

Generating input variants. Producing payload variations for fuzzing, and adapting encodings when a filter blocks the obvious form.

Drafting the write-up. Turning a tester's notes and captured requests into a clear, reproducible finding, which a practitioner then checks and signs.

And what it does not

It does not understand your business. Whether a discount can be stacked, whether a support agent should see settlement data, whether an admin action should be reversible: none of that is in the code, so no amount of reading the code recovers it.

It does not chain. Combining minor issues into a real compromise means holding a goal in mind and reasoning backwards through a system. Automation, AI included, evaluates findings as items.

It invents findings. This is the one to actually worry about. AI tooling produces plausible, well-written, confident vulnerabilities that do not exist. Ship those to a customer and their security team will try to reproduce a finding, fail, and rightly discount the entire report. Every finding we report is reproduced by hand before it reaches you. That verification step is not optional overhead; it is what makes the rest of the AI-assisted work usable.

The reasonable summary: AI moved the boundary between the machine layer and the human layer upward. It did not remove the human layer, and any vendor telling you it did is selling a scan.

The human domain

The vulnerabilities that get exploited, and that enterprise buyers ask about, are rarely a missing patch.

Business-logic flaws

A business-logic flaw is an application working exactly as coded, where the logic itself is wrong. A checkout flow that lets a user alter the price parameter in transit. A scanner sees an HTTP POST returning 200 and records success. A person recognises that a customer just bought a high-value item for pennies, or approved their own refund.

There is no signature for this, and no model of your code can infer the rule that was supposed to exist.

Broken access control

Access control is about who may reach what. Distinguishing a legitimate query from horizontal privilege escalation, where one user retrieves another's billing records, means mapping the role model and then manipulating session state and tokens against it. The request looks valid in both cases. Only the intent differs.

Vulnerability chaining

Scanners score findings in isolation. Attackers combine them. Three low-severity issues:

• an error message that leaks an internal path
• a CSRF flaw on a profile page
• path traversal in a profile-picture upload handler

To a scanner, three minor items to schedule for next quarter. To a tester, a route to remote code execution.

Testing AI is a different job

Worth separating, because the words collide. Using AI to help test your systems is what this article is about. Testing an AI system is its own assessment: prompt injection, insecure output handling, training-data and model-supply-chain risk, excessive agency in tools the model can call. It works to the OWASP Top 10 for LLM Applications rather than the classic web list, and if you have shipped an LLM feature it needs scoping deliberately. See AI security.

Named standards, not a private checklist

A professional test maps to methodologies your auditors and customers recognise:

• OWASP Top 10 and the LLM Top 10 for application, API and model-integration flaws
• PTES for the workflow, from scoping and intelligence gathering through threat modelling, exploitation and post-exploitation
• MITRE ATT&CK for mapping post-compromise behaviour to what real adversary groups actually do

If a provider cannot tell you which standard they worked to, the answer is their own tooling.

How to spot a scan sold as a pen test

Compliance deadlines created demand for penetration tests, and a market grew to meet it with automated scans in a report cover. More recently the same product has reappeared branded as "AI-powered". The tells are unchanged:

The single most useful question to ask a prospective tester: "which of your findings did you have to reproduce by hand, and how?" Anyone running a scan cannot answer it.

What you should get at the end

A penetration test is an evidence asset, not a checkbox. A complete deliverable is a prioritised, reproducible report mapped to named standards, an evidence pack a customer's security team will accept, and a verification retest confirming the fixes hold.

Automation for breadth, AI for the volume work, people for the judgement. That is what AI-assisted and manual penetration testing means in practice, and it is why the findings that matter still come from someone who understood what your application was trying to do.

FAQ

Is penetration testing manual or automated?

Both, across three layers. Automated scanning gives breadth across everything you own. AI assistance handles the high-volume work in between: mapping attack surface, triaging and deduplicating findings, reading large codebases, generating input variants. Manual testing supplies the depth: business logic, authorisation, chained exploits, and judging what a finding is actually worth. A test using only one layer misses what the others catch.

Can AI replace a human penetration tester?

No, and the failure mode is specific. AI has no model of what your application is for, so it cannot identify a business-logic flaw, which is a feature working exactly as coded against a rule that was never written down. It also does not chain minor issues into a real compromise, because that needs goal-directed reasoning across a system. Most importantly, AI tooling generates confident findings that are not real, so every finding still has to be reproduced by hand before it is reported.

What is a business-logic flaw?

A vulnerability where the application behaves exactly as programmed but the logic itself is wrong: altering a price parameter in transit, approving your own refund, stacking a discount that was never meant to combine. There is no signature to match, because nothing is technically broken. Finding these requires someone who understands what the workflow is supposed to permit.

How do I tell a real penetration test from an automated scan?

Ask which findings the tester had to reproduce by hand, and how. A scan-based provider cannot answer. Other tells: turnaround in hours rather than one to three weeks, scoping that needs only a URL, findings that read like a tool export, and no business-logic findings at all. "AI-powered" is the current label for the same product.

Which standards should a penetration test map to?

OWASP Top 10 for application and API flaws, plus the OWASP Top 10 for LLM Applications where models are involved; PTES for the engagement workflow from scoping through post-exploitation; and MITRE ATT&CK for mapping post-compromise behaviour to real adversary tactics. If a provider cannot name the standard they worked to, they worked to their own tooling.

Is testing an AI system the same as AI-assisted testing?

No, they are separate things. AI-assisted testing means using AI to help test your systems. Testing an AI system is its own assessment covering prompt injection, insecure output handling, training-data and model-supply-chain risk, and excessive agency in the tools a model can call. If you have shipped an LLM feature, it needs scoping deliberately rather than being folded into a standard web application test.