Troubleshoot registry authentication errors

A login or pull against cgr.dev failed and you have an error string. This page maps the errors Chainguard’s registry returns to their causes and tells you what to check for each one.

If your pull fails for a container you can see in the Chainguard Containers Directory, the container most likely hasn’t been added to your organization yet. That case has its own guide: refer to Troubleshoot container and version availability.

Where registry errors come from

Pulling a container from cgr.dev takes two separate requests, and each one fails differently:

1. Credential exchange. Your tool sends your credentials to the token endpoint at https://cgr.dev/token and gets back a bearer token scoped to the repository you asked for. docker login, helm registry login, podman login, and chainctl auth configure-docker all exercise this step.
2. The pull. Your tool presents that bearer token to the registry API at https://cgr.dev/v2/ and requests the tags, manifest, and layers.

The two steps reuse the same HTTP status codes for unrelated problems. A 403 from the token endpoint means something different from a 403 from the registry API, so read the error code and message in the response body rather than the status number on its own:

A bare Forbidden message doesn’t tell you which of those two it is. Chainguard’s registry returns the same 403 to a caller that sent no credentials, a caller whose credentials it can’t use, and a caller naming a repository it won’t grant a token for. Treat it as a prompt to work through the possible causes in order rather than as a single diagnosis, and in particular don’t read it as confirmation that your credentials were accepted.

Authentication required from the token endpoint

You’ll see a 401 like the following when a build or pull runs with no credentials configured:

failed to fetch anonymous token: unexpected status from GET request to https://cgr.dev/token?scope=repository%3A$ORGANIZATION%2Fpython%3Apull&service=cgr.dev: 401 Unauthorized

The word “anonymous” is the signal. Your tool found no credentials for cgr.dev, so it asked for a public token, and the repository you named isn’t public. Containers in the cgr.dev/chainguard/ namespace are public and need no authentication. Everything in your organization’s own namespace at cgr.dev/$ORGANIZATION/ requires credentials.

Configure the credential helper, then retry:

chainctl auth configure-docker

You don’t need to run chainctl auth login first. For headless machines, CI systems, and other login flows, refer to Authenticate to Chainguard’s Registry and Authentication options for chainctl.

If the failure happens inside docker build rather than docker pull, check that the builder can see your Docker configuration. A FROM line pointing at your organization’s namespace needs the same credentials a direct pull does.

If the message appears in a Kubernetes pod’s events, the credential helper won’t help, because the kubelet can’t run it. Refer to Pods stuck in ImagePullBackOff instead.

Forbidden from the token endpoint

A 403 from the token endpoint looks like this, often from helm registry login or another tool that logs in without naming a repository:

Error: authenticating to "cgr.dev": GET "https://cgr.dev/token?service=cgr.dev": response status code 403: forbidden

This response means the token endpoint rejected the request outright. It doesn’t tell you which part of the request it objected to, so check these in order:

1. The credential format. The username for a pull token is the Chainguard identity ID that owns the token, not your email address. Refer to Check your credential format.
2. The credential’s age. Pull tokens expire 30 days after creation by default. Create a replacement with chainctl auth configure-docker --pull-token.
3. The repository name. A scope the endpoint won’t grant returns this 403 rather than a 404, whether the repository doesn’t exist or isn’t in your organization’s catalog. Confirm the name against chainctl images repos list.

Bad request from the token endpoint

A 400 from the token endpoint has two causes, and the message text tells them apart.

The pull token expired

When a pull presents an expired pull token, the token endpoint returns a 400 rather than a 401 or 403:

{"errors":[{"code":"BAD_REQUEST","message":"rpc error: code = InvalidArgument desc = this identity's keys have expired"}]}

Create a replacement with chainctl auth configure-docker --pull-token and update every place that holds the old one. A login with an expired token, such as docker login or helm registry login, returns the bare 403 described in Forbidden from the token endpoint instead.

The organization didn’t resolve

When the message says unable to resolve, the organization portion of the image reference didn’t match any Chainguard organization:

{"errors":[{"code":"BAD_REQUEST","message":"rpc error: code = InvalidArgument desc = unable to resolve ..."}]}

This cause has nothing to do with your credentials. The organization name is wrong. List the organizations you belong to and compare:

chainctl iam organizations list

Organization names are usually domain names, such as example.com, and the full image reference is cgr.dev/$ORGANIZATION/$IMAGE:$TAG. Omitting the organization, or using your organization’s display name instead of its registry name, can produce this error.

Missing capabilities from the registry API

This 403 is the one that means your login worked and your authorization didn’t:

{"errors":[{"code":"FORBIDDEN","message":"caller does not have the required capabilities at \"...\""}]}

It also surfaces through the tool you’re running. A Helm install that logged in successfully and then failed reports it against the API path:

Error: INSTALLATION FAILED: ... /$ORGANIZATION/postgresql/tags/list ... response status code 403

Two different problems produce this error, and the fix differs:

• The repository isn’t in your organization’s catalog. This is the more common cause. Your credentials are valid, but the container hasn’t been added to your organization. Refer to Troubleshoot container and version availability for how to add it or request it, along with the roles that each path requires.

• Your identity lacks a role that grants pull access. Check which capabilities your current credentials carry:

  chainctl auth status

  Read the Capabilities field in the output. Pulling containers needs a role with the manifest (list) capability; registry.pull is the least privileged built-in role that has it. To grant it, refer to Overview of roles and role-bindings. Pulling APK packages needs apk (list) instead, which Private APK repositories covers.

An identity federated from a CI system carries only the role you assigned when you created it. An identity created with --role=registry.pull can pull containers and do nothing else, which is intentional, but it means the same credentials fail if your workflow later tries to list repositories or read entitlements.

Repository does not exist

Once you hold a valid token, a name that isn’t in the organization returns a 404 rather than a 403:

{"errors":[{"code":"NAME_UNKNOWN","message":"repository does not exist \"...\""}]}

Check the spelling against chainctl images repos list, then against the Chainguard Containers Directory. If the Directory doesn’t list the container either, Chainguard doesn’t build it yet and you can ask for it through Requesting new Chainguard resources. To work out which of those situations you’re in, refer to Troubleshoot container and version availability.

Check your credential format

When you authenticate with a pull token, the username is the Chainguard identity ID associated with that token, and the password is the token itself:

docker login "cgr.dev" \
  --username "<identity-id>" \
  --password "<pull-token>"

The email address you use to sign in to the Chainguard Console is not a valid registry username. Neither is your organization name. chainctl auth configure-docker --pull-token prints the correct pair, and chainctl auth pull-token create --output=env writes them to CHAINGUARD_IDENTITY_ID and CHAINGUARD_TOKEN for use in scripts.

The same pair works with any tool that logs in to an OCI registry:

helm registry login cgr.dev \
  --username "$CHAINGUARD_IDENTITY_ID" \
  --password "$CHAINGUARD_TOKEN"

Helm and Podman need chainctl installed only to create the token. Once you have the identity ID and token, the login itself doesn’t call chainctl, so you can run it on a machine that doesn’t have chainctl at all. Refer to pull token output formats and credential names for the other output formats.

Errors from a pull-through cache or mirror

A registry mirror reports Chainguard’s errors in its own wording, which can obscure which of the preceding cases you’re in. Artifactory, for example, reports a rejected credential as a configuration problem:

Invalid username/password configured for Remote Docker repository: $REPOSITORY_NAME ... Can't fetch token for repo ... realm: https://cgr.dev/token

The realm: https://cgr.dev/token fragment tells you the failure happened during credential exchange, so work through Forbidden from the token endpoint and Check your credential format. Mirrors are a common place for the username to be wrong, because their configuration forms label the field “username” and invite an email address.

Pull tokens expire, so a mirror that worked for a month and then stopped is likely holding an expired token. For the tool-specific settings each platform needs, refer to the pull-through guides.

Pods stuck in ImagePullBackOff

A Kubernetes pod that can’t pull its container reports ErrImagePull and then ImagePullBackOff. That often happens with credentials that pull the same container without trouble through docker or crane. When the credentials work everywhere except Kubernetes, the kubelet usually never sent them.

Read the pull failure in the pod’s events:

kubectl describe pod <pod-name> --namespace <namespace>

The Failed event repeats the token endpoint’s response:

Failed to pull image "cgr.dev/$ORGANIZATION/python:latest": ... failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://cgr.dev/token?scope=repository%3A$ORGANIZATION%2Fpython%3Apull&service=cgr.dev: 401 Unauthorized

The word before token tells you where to look:

• anonymous token: The kubelet found no credentials it could use for cgr.dev, so it pulled without any. The credentials themselves may be correct. Work through the checks in this section, in order.
• oauth token: The kubelet sent credentials and the token endpoint rejected them. A 400 means the pull token expired, as described in The pull token expired. A 401 or 403 means the username or token is wrong, so refer to Check your credential format.

Check the secret type

The kubelet reads pull credentials only from a secret of type kubernetes.io/dockerconfigjson. Print the type of the secret your pod references:

kubectl get secret <secret-name> --namespace <namespace> -o jsonpath='{.type}'

If the output is Opaque, the kubelet ignores the secret and emits no warning about it. kubectl create secret generic creates an Opaque secret unless you pass --type=kubernetes.io/dockerconfigjson, and so does a manifest that omits type:. Authenticating with Kubernetes covers this in more detail.

Replace the secret under the same name:

kubectl delete secret <secret-name> --namespace <namespace>
kubectl create secret docker-registry <secret-name> \
  --namespace <namespace> \
  --docker-server=cgr.dev \
  --docker-username="<identity-id>" \
  --docker-password="<pull-token>"

The kubelet reads the secret again on its next pull attempt, so the pod recovers without changes. A pod that has been failing for a while can wait many minutes between attempts, though. To retry now, delete the pod and let its controller recreate it, or restart its Deployment as shown in Check that the pod references the secret.

Check the secret’s namespace

A pod can use only pull secrets in its own namespace. When the secret it names isn’t there, the pod’s events include this warning alongside the anonymous token failure:

Unable to retrieve some image pull secrets (<secret-name>); attempting to pull the image may not succeed.

Create the secret in the namespace where the pod runs. Each namespace that pulls from cgr.dev needs its own copy.

Check that the pod references the secret

A secret in the right namespace has no effect until the pod names it. Print the pull secrets the pod received:

kubectl get pod <pod-name> --namespace <namespace> -o jsonpath='{.spec.imagePullSecrets}'

Empty output means the pod has none. Add the secret to the pod template’s imagePullSecrets, or to the service account the pod runs as. For Helm charts, the Chainguard Helm charts guide shows the global.imagePullSecrets value.

Kubernetes copies a service account’s imagePullSecrets into a pod only when the pod is created, and it won’t add them to a pod that already exists. After you update a service account, recreate the pods that use it. For a Deployment:

kubectl rollout restart deployment <deployment-name> --namespace <namespace>

Check the registry name in the secret

The kubelet uses a credential only when its key in the secret matches the image’s registry. List the keys without printing the credentials:

kubectl get secret <secret-name> --namespace <namespace> \
  -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d | jq '.auths // {} | keys'

The output needs to include "cgr.dev". The kubelet also accepts "https://cgr.dev", and a key that includes your organization’s path, such as "cgr.dev/$ORGANIZATION". Two outputs point to a specific mistake:

• ["https://index.docker.io/v1/"]: The secret was created with kubectl create secret docker-registry and no --docker-server flag, so the credentials are registered for Docker Hub. Re-create it with --docker-server=cgr.dev.
• [], or a list without cgr.dev: The secret was copied from a Docker configuration that reaches cgr.dev through the chainctl credential helper. That configuration stores no credentials for cgr.dev, only the name of a helper program, and the kubelet can’t run it. Create a pull token and build the secret from it, as described in Authenticating with Kubernetes.

A key that names a different organization, such as "cgr.dev/another-org", or that adds a port, such as "cgr.dev:443", also fails to match.

Isolate the failing step

To find out which of the two requests is failing, ask the token endpoint directly. This returns the HTTP status for a credential exchange with no credentials attached:

curl -s -o /dev/null -w '%{http_code}\n' \
  "https://cgr.dev/token?scope=repository%3A$ORGANIZATION%2F$IMAGE%3Apull&service=cgr.dev"

A 401 means the reference resolved and the endpoint wants credentials, so your tool’s configuration is the problem rather than the name you used. Drop the -o /dev/null to read the error code and message in the response body.

To check the second request, compare chainctl auth status against the capabilities listed in Missing capabilities from the registry API.

If the pull fails only in Kubernetes, start from the pod’s events instead, as described in Pods stuck in ImagePullBackOff.

Can’t create a pull token

If the Console won’t let you create a pull token, or chainctl auth pull-token create fails, the problem is your role rather than the registry. Creating a pull token creates a Chainguard identity and a role-binding for it, so it needs a role carrying the identity (create) and role_bindings (create) capabilities.

registry.pull_token_creator is the least privileged built-in role with both. Ask an administrator in your organization to bind it to your account, or refer to Overview of roles and role-bindings to do it yourself. The capabilities reference lists every capability each built-in role carries.

Learn more

• Authenticate to Chainguard’s Registry
• Troubleshoot container and version availability
• Onboard your teams
• Network requirements
• Get support