umma.dev

Automating Vulnerabilities via Agents

Dependabot has flagged a vulnerable dependency in your GitHub repository. There might already be a pull request with an update but the tests are failing. Or the vulnerable package is several levels down the dependency tree and it isn’t obvious which dependency you need to change.

This is where an agent can be useful. Reading the alert, working out how the package enters the project, making the update and dealing with the application changes needed to get it working.

There are two ways to approach this. You can keep a skill in your repository and ask an agent to work through the alerts while you’re developing. Or you can connect an agent to GitHub automation, so it picks up alerts or Dependabot pull requests in the background.

What Dependabot already does

Dependabot alerts identify dependencies with known vulnerabilities using GitHub’s dependency graph and advisory data. An alert includes the affected dependency, the advisory and a patched version where one is available.

Dependabot security updates can then open pull requests to update vulnerable dependencies. These differ from version updates, which keep packages current even when there isn’t a known vulnerability.

For a straightforward update with passing tests, Dependabot and your existing review process may already be enough. An agent becomes more useful when the update needs investigation; a parent package must change, a new version breaks an API, or several alerts appear to come from the same dependency.

A Dependabot skill

A skill gives your coding agent a repeatable procedure for handling the alerts. It can describe the package manager, the checks to run, the constraints on upgrades and what evidence to include in the resulting pull request.

The Agent Skills format uses a SKILL.md file containing metadata and instructions, with optional scripts and references alongside it. The discovery directory depends on the agent you’re using.

For a JavaScript repository, the instructions might look like this:

---
name: fix-dependabot-alerts
description: Investigate and fix Dependabot alerts in this repository.
---

1. Fetch the requested open alerts using authenticated GitHub access,
   or use alert details supplied by the developer. Report access failures.
2. Read the advisory, affected manifest, installed version and patched range.
3. Check for an existing Dependabot pull request before creating duplicate work.
4. Trace whether the package is a direct or transitive dependency.
5. Choose a supported update that resolves the affected dependency paths.
   Read release notes for compatibility changes.
6. Update dependencies using the repository's package manager and regenerate
   the lockfile. Keep unrelated upgrades out of the change.
7. Run the relevant tests and build. Investigate failures introduced by the update.
8. Report the alert IDs, version changes, checks run and unresolved issues.
   Do not dismiss alerts or merge changes automatically.

You could then ask:

Use the Dependabot skill to investigate the open alerts, fix those with available updates and prepare the changes for review.

The skill supplies the procedure. The agent still needs permission to read the alerts and tools to edit the repository and run its checks. Checking in a markdown file doesn’t grant GitHub access or schedule a job.

The agent can connect through the GitHub MCP server or the GitHub CLI, depending on the tools available. With the relevant MCP tools enabled and suitable permissions, it can read Dependabot alerts, inspect existing pull requests and publish a proposed fix. The CLI offers another route, using gh api to fetch alerts and pull request commands to manage the resulting changes. Dependency installation and tests still run in the agent’s development environment. MCP or the CLI provides access to GitHub; the skill defines the procedure, and a workflow supplies the automation trigger.

A vulnerable transitive dependency

Let’s say an alert identifies a vulnerable HTTP client. Your application doesn’t list it in package.json; a service SDK brings it in as a transitive dependency.

In an npm project, the agent can use npm explain <package> or npm ls <package> to inspect why it is installed. It then checks whether a supported SDK release resolves to a patched HTTP client and whether upgrading that SDK requires application changes.

Suppose the newer SDK changes how requests are configured. The agent updates the SDK, regenerates the lockfile, adjusts the affected calls and runs the integration tests and build. It also checks whether another dependency still brings in a vulnerable copy of the HTTP client.

Updating one dependency path doesn’t necessarily remove every affected installation. The pull request should explain which paths changed and identify anything still unresolved.

Overrides can sometimes force a transitive dependency to a patched version, but compatibility needs checking. Where possible, prefer an upstream-supported dependency combination. If an override is necessary, explain why and record when it can be removed.

An agent working through GitHub automation

The same procedure can run without a developer starting a local session. GitHub automation supplies the trigger and environment; the agent investigates the dependency update and prepares a reviewable result.

There are two useful starting points.

Work on Dependabot pull requests

Dependabot pull requests can trigger GitHub Actions workflows. A workflow can identify a Dependabot update, run your normal checks and send the agent the diff, advisory details and any failing test output.

For a security-focused workflow, distinguish security updates from routine version updates using the associated alerts and update metadata. A Dependabot author alone doesn’t establish that a pull request fixes a vulnerability.

If the update passes everything, there may be little for the agent to do. If it breaks the application, the agent can inspect the release notes, reproduce the failure and prepare the compatibility changes. Keep that work attached to the existing update where your integration permits, or create a clearly linked follow-up pull request.

For the SDK example, Dependabot might have already made the version bump. The agent’s contribution is adapting the request configuration and verifying the resulting behaviour.

Work through the alert backlog

A scheduled or manually triggered workflow can fetch open alerts through the Dependabot alerts REST API, then pass selected alerts to the agent. For repository alert reads, use an appropriately authorised credential, such as a GitHub App installation token with Dependabot alerts read access. A normal checkout token should not be assumed to have that access.

An event-driven integration can instead receive a dependabot_alert webhook through a GitHub App or webhook receiver and dispatch work. That webhook is separate from a native Actions workflow trigger; adding on: dependabot_alert to a workflow isn’t the integration.

A typical run would:

  1. Fetch open alerts and check for existing remediation pull requests.
  2. Group alerts that share a dependency or can be resolved by the same update.
  3. Give the agent the advisory details, manifests, lockfiles and trusted repository instructions.
  4. Apply a bounded update and any necessary compatibility fixes in an isolated checkout.
  5. Run the relevant checks and open a pull request explaining what changed and what remains unresolved.

This also covers alerts for which Dependabot hasn’t opened a usable update. Deduplicate by alert and dependency so a daily run doesn’t keep creating the same pull request.