AI Agents in SDLC Erase Audit Trails, Complicating Compliance and Governance

Organisations using artificial intelligence to build software face a growing compliance risk because autonomous agents leave no permanent record of their actions. Research published by JFrog indicates that while AI agents now handle pull requests, dependency selection, and infrastructure changes, their decision-making processes are ephemeral. Once a session ends, the specific reasoning, model versions, and tool calls used by the agent disappear, making it impossible to hold the process accountable or trace provenance.

Current software development life cycle standards were designed around human engineers, assuming that a person made every commit and verified every release. JFrog’s analysis shows this assumption no longer holds. Git logs capture the final state of code changes but fail to record which AI model generated them, which tools were invoked, or whether any human oversight occurred. This gap means that every existing governance framework, audit trail, and compliance control is fundamentally misaligned with the reality of agentic software development.

What we know

The shift to autonomous agents introduces significant operational and security challenges that traditional peer review cannot address. According to JFrog, human reviewers face volume overload when examining thousands of lines of agentic code, leading to fatigue and rubber-stamping approvals. Agents also make non-deterministic decisions that can introduce latent vulnerabilities or misalignments, even if functional tests pass. These issues are compounded by the fact that agents are transient entities; their internal logic and prompt iterations are not preserved in standard version control systems.

This lack of visibility opens new attack surfaces within the software supply chain. The research highlights that agents, harnesses, and large language models now act as new actors in the development process. This introduces vectors such as prompt injections, malicious Model Context Protocol tools, and jailbreaks from misaligned or unapproved models. Unlike human errors, these agentic failures occur without a traceable intent or identity, complicating incident response and forensic analysis for security teams.

Regulatory bodies are beginning to address these gaps, creating pressure for organisations to adapt their logging practices. The EU AI Act imposes strict record-keeping duties for high-risk AI systems, requiring automatic event logging and effective human oversight during use. Similarly, the OWASP AI Security Verification Standard mandates that AI interactions be logged with session context and that audit logs capture security-critical actions, including approver identity and decision outcomes. These requirements align with broader security-by-design expectations found in the EU Cyber Resilience Act and ISO/IEC 42001.

To meet these standards, JFrog suggests attaching provenance data directly to the software entities produced by agents, such as commits or artifacts. This approach enables automatic release gating through policy enforcement and supports post-release troubleshooting. By linking the ephemeral actions of AI agents to tangible SDLC outputs, organisations can create the audit trails necessary for compliance and security verification.

Defender's checklist

• Verify that AI agent sessions are logged with full context, including model versions and tool calls.
• Ensure audit trails capture security-critical actions, such as approver identity and decision outcomes.
• Implement automated policy enforcement to gate releases based on attached provenance data.
• Review current compliance frameworks to identify gaps in tracking non-human decision-makers.

Background: AI agents

AI agents are the software entities that perform specific tasks using instructions from large language models. Agentic AI is the architectural approach that grants these agents autonomy to plan, reason and act. You use agents as tools within agentic AI systems to achieve complex, multi-step outcomes without manual oversight at every stage.

Read the full guide: AI Agents vs Agentic AI: The Distinction That Matters

What to do and how to stay safe: JFrog

• Audit your current version control systems to determine if they capture AI agent reasoning and tool usage.
• Implement logging mechanisms that record session context and telemetry for all AI interactions in your SDLC.
• Establish clear policies for human oversight of high-risk AI systems, ensuring decisions are traceable to natural persons.
• Monitor for new regulatory requirements regarding AI system logging and provenance, such as those in the EU AI Act.

Step-by-step guide: AI Governance for Small Teams: Practical Controls and Cost

General security guidance from the TechWire Beat newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Why are traditional git logs insufficient for AI-driven development?

Git logs only show what code changed, not the AI agent's reasoning, the specific model used, or the tools it called during the process.

What regulations address AI logging requirements?

The EU AI Act and the OWASP AI Security Verification Standard set requirements for automatic event logging and audit trails for AI systems.

How do AI agents create new supply chain risks?

Agents introduce vulnerabilities through prompt injections, malicious tools, and non-deterministic decisions that may not be caught by functional tests.