Skip to main content

Devin

How to connect Devin to Pensero, the read-only permissions it needs, setup steps, who gets Delivery credit for Devin’s pull requests, troubleshooting, and how estimated Devin cost is calculated.

Written by Wayne

Overview

Pensero’s Devin integration helps you understand what AI agents deliver and how their work impacts your engineering organization.

Once connected, Pensero imports Devin activity and aligns it with your engineering delivery data. This allows leaders to compare human and agent contributions in the same context, across code, reviews, delivery, and quality.

Devin is available only on the Enterprise tier.

What You Can Measure

With Devin connected, Pensero helps you understand how agents work alongside your teams, what each contributes, and how to optimize your hybrid workforce for greater delivery and quality.

You can track:

  • Code and review activity linked to Devin

  • Devin’s contribution to delivery

  • The impact of Devin’s work on quality

  • How Devin’s output compares with human engineering work

This helps leaders move beyond usage tracking and understand the real ROI of AI agents in the engineering organization.

Prerequisites

Before setting up the Devin integration, make sure you have:

  • Pensero Admin access

  • Access to your Devin organization

  • Enterprise tier enabled in Pensero

  • Permission in Devin to create a service user and assign it a role (Devin organization admin)

Devin Permissions Pensero Needs

Pensero connects to the Devin API with an organization service user that you create in Devin. Pensero only reads data: it never creates sessions, sends messages, or changes anything in your Devin organization.

Grant the service user’s role these two organization-level permissions:

  • ViewOrgSessions — reads the session list, session details, session insights, and session transcripts. This is what Pensero turns into agent delivery, review, and quality data.

  • ViewOrgMembership — reads the organization’s users. Sessions only carry a user ID, so this is the only way Pensero can attribute a session to the person who started it.

No write permissions are required. You do not need to grant UseDevinSessions, ManageOrgSessions, ViewOrgMetrics, or any enterprise-level role. Pensero reads each session’s own ACU value from the session data, so the Devin metrics and consumption endpoints are not used.

One optional convenience: if the service user also has the enterprise ManageOrganizations permission, Pensero lists the enterprise’s organizations for you and you pick which ones to sync. With an organization-scoped service user that listing is refused, which is expected — Pensero shows a notice and you add your organization by ID instead.

If your Devin organization assigns membership through an identity provider (IdP) group instead of direct role assignments, contact Pensero support: user attribution may need extra configuration.

How to Set Up the Devin Integration

Step 1: Create a Service User in Devin

In Devin, go to Settings → Devin API. You will see three tabs. Use Service users.

  • Service users: use this one. The key belongs to the integration, not to a person, so it keeps working when people change roles or leave.

  • PATs: do not use. A personal token works as you. Pensero only sees what you can see, and the sync stops if you leave or lose access.

  • API keys (legacy, start with apk_): do not use. Pensero reads the current Devin API, which does not accept these keys.

Then:

  1. Click Provision service user and give it a clear name (for example, Pensero read-only).

  2. Assign it a role that includes ViewOrgSessions and ViewOrgMembership.

  3. Copy the API key. It starts with cog_ and Devin shows it only once.

  4. Copy your Devin organization ID. It starts with org- and is shown at the top of the same page.

If you have several Devin organizations, create the service user in Enterprise settings → Devin API → Service users instead, so Pensero can list all of them.

Step 2: Open Devin Settings in Pensero

Go to: Integrations → Devin Settings

You will see the Devin connection page, where you can add or update the credentials for your Devin organization.

Step 3: Add the API Token

Paste the service user API key into API Token and click Save credentials. The connection page only asks for the token; the organization comes next.

Step 4: Add and Enable Your Devin Organization

After saving, Pensero tries to list the organizations the token reaches.

  • With an organization-scoped service user, Pensero reports that the token cannot list the enterprise. Enter your Devin organization ID (org-…) in the Organizations card and click Add organization.

  • With enterprise-wide permissions, the organizations are listed for you. Click Enable on each one you want Pensero to sync.

Syncing starts as soon as an organization is enabled. The health check on the same page confirms the token works for that organization.

Who Gets Credit for Devin’s Pull Requests

A pull request opened by Devin is scored like any other pull request, and its Delivery points go to the people behind it: first whoever committed to it, then anyone named as a co-author, and otherwise the people who steered the linked Devin session. Devin keeps only the share no person can claim, reported as Agentic delivery. Names Pensero cannot match to a contributor wait in Lost and Found → AI Tools until you bind them.

Reasoning and rules: How Pensero credits Devin’s pull requests.

Estimated Devin Cost

Devin does not report what a session cost. Dollar amounts live only in Devin’s own billing screens, and the usage values Pensero can read from the Devin API (ACUs) come back as zero unless your Devin organization is on an Enterprise plan. Pensero therefore estimates Devin spend rather than importing it, so you can still put agent cost next to agent delivery.

The estimate powers two cards on the Agents page:

  • DEVIN EST. $ — total estimated Devin spend in the selected period.

  • DEVIN $ / AGENT PT — that spend divided by the delivery points your agents produced in the same period.

Both are labelled as estimates and appear nowhere else: no other cost figure in Pensero includes them.

How the Estimate Is Calculated

Devin labels every session with a size — xs, s, m, l or xl — reflecting how much work it did. That label is the only cost-related signal available outside Enterprise plans, so Pensero prices each session from its size:

  • xs — $0.00

  • s — $9.89

  • m — $9.89

  • l — $31.94

  • xl — $73.88

Two other Devin products are added on top: code scan sessions are billed on a separate, cheaper line and count as $3.09 each, and every Devin Review submitted on one of your pull requests adds about $0.02.

These rates were derived by fitting session sizes against one real Devin billing cycle: the fitted total landed within 1% of the invoiced amount for that cycle, with sizes explaining roughly 90% of the daily variation. In that data, s and m sessions were indistinguishable in price and xs sessions were effectively free, which is why they share a rate and xs counts as zero.

What to Keep in Mind

  • It is accurate at period scale, not per session. Any individual session’s figure is a size-based average, so use the cards to see trends and cost per delivery point, not to audit one session.

  • The rates come from one organization’s billing cycle. If your Devin contract or usage mix differs, your real invoice will differ from the estimate.

  • Sessions with no size count as $0. Sessions synced before Pensero began storing Devin’s session insights have no size and do not contribute; contact support if historical periods look too cheap.

  • All estimated spend sits on the Devin agent identity. It is never charged to the person who started the session or to the author of the reviewed pull request, so it does not distort human cost or delivery figures.

  • Devin is treated as an agent, not an AI tool. The estimate never appears in AI tool spend, the AI Economics cost figures, or any per-tool usage table.

Requirements

  • The Devin integration is connected and sessions have synced.

  • Devin’s identity in Pensero is linked to an agent user. If it is unlinked, Pensero has no agent to attribute the spend to and the cards stay hidden. See Lost and Found.

  • Estimated Devin cost is released gradually: ask Pensero support to enable it for your organization.

  • Organizations on Pensero’s Enterprise tier keep their existing ACU-based Devin reporting and do not see the estimate.

Troubleshooting

  • Credentials rejected, or sync fails with an authentication error: the API key is wrong or expired, or the service user’s role is missing one of the two permissions above. Check the role in Devin, then re-enter the key in Pensero.

  • Sessions appear but are not attributed to people: the service user is missing ViewOrgMembership, or the Devin user’s email does not match the person in Pensero. See Diagnosing Missing or Incorrect AI Data.

  • “This token cannot list the enterprise organizations” after saving: expected for an organization-scoped service user. Add your organization by ID in the Organizations card; nothing is wrong with the key.

  • No sessions arrive at all: check that your organization is listed and shows Syncing in the Organizations card. An organization that is present but disabled is never synced.

  • Sessions stopped syncing after someone left the team: the integration was connected with a personal token (PAT). Create a service user key as in Step 1 and update the credentials in Pensero.

  • Key starts with apk_: it is a legacy key and Pensero cannot use it. Create a service user key as in Step 1.

Related Articles
Did this answer your question?