OIDC authentication for Grafana

Cozystack Grafana instances can authenticate users through OIDC instead of the shared admin_user / admin_password Secret. Each user then has their own identity, per-user audit, and a role that can be revoked by removing them from spec.oidc.users on the Monitoring CR, as long as the list keeps at least one other entry (see Assigning roles).

The identity model is deliberately per-instance rather than per-tenant: every Monitoring instance (one inner monitoring-system release per namespace — each tenant’s own, plus the platform’s in tenant-root) gets its own OIDC audience, and a token minted for instance A is rejected by instance B’s Grafana. Cross-tenant sign-in is additionally blocked by a allowed_groups gate on the release’s namespace-scoped <ns>-{view,use,admin,super-admin} groups (chart-owned in the tenant chart; the platform-managed groups scope in the cozy realm makes them visible on every token). The full rationale is in the design proposal. The tenant kube-apiserver’s Phase 1 ( cozystack/cozystack#3044) uses the same shape; this Grafana integration is the Phase-1 follow-up called out in that PR’s body.

Modes

spec.oidc.mode picks the identity source on the Monitoring CR:

• None — the default. No OIDC; only the admin_user / admin_password Secret works. Existing instances render identically to before.
• System — trust the platform cozy Keycloak realm via a per-instance confidential client and audience binding. Users are the ones a Cozystack platform admin already provisioned in cozy; the tenant does not manage a directory of its own.
• CustomConfig — trust a tenant-supplied issuer directly (BYO IdP: Okta, Auth0, a customer’s own Keycloak). cozy is not in the path.

Enable OIDC — System mode

apiVersion: apps.cozystack.io/v1alpha1
kind: Monitoring
metadata:
  name: monitoring
  namespace: tenant-acme
spec:
  oidc:
    mode: System
    users:
      - email: alice@acme.example
        role: Admin
      - email: bob@acme.example
        role: Editor
      - email: carol@acme.example
        role: Viewer
  # ...

Cozystack provisions the following. Every derived identifier below uses <release>, the name of the inner HelmRelease that actually carries the OIDC templates. A Monitoring CR’s release name is forced to monitoring (check-release-name.yaml rejects any other name), and its inner release is always <release-name>-system, so <release> is literally monitoring-system in every namespace — tenant-acme-monitoring-system for the CR above. Instances are told apart by namespace, not by release name:

• A per-instance KeycloakClient in the cozy realm with clientId set to <namespace>-<release> (for the CR above: tenant-acme-monitoring-system). Confidential (clientAuthenticatorType: client-secret), secret sourced from a chart-owned Kubernetes Secret. redirectUris locked to https://grafana.<host>/login/generic_oauth.

• A per-instance KeycloakClientScope whose audience mapper pins the token’s aud claim to that same clientId — the isolation primitive.

• A persistent Kubernetes Secret carrying the confidential client-secret (random on first install, preserved on upgrades).

• A chart-owned users-reconcile Job (<release>-oidc-users) that syncs spec.oidc.users[] into the Grafana instance on every reconcile: creates missing Grafana accounts, patches roles, and prunes stale org members. Runs as a post-install / post-upgrade hook when users[] is non-empty; omitted otherwise (BYO-IdP-friendly).

• An auth.generic_oauth entry under the Grafana CR’s spec.config (a single ini-section key literally named auth.generic_oauth, so it is addressed as spec.config["auth.generic_oauth"], not a two-level auth / generic_oauth nesting), wired to the cozy realm issuer, per-instance audience scope, and the tenant-membership gate:

  allowed_groups        = <namespace>-view <namespace>-use <namespace>-admin <namespace>-super-admin
  groups_attribute_path = groups
  

  groups_attribute_path is REQUIRED alongside allowed_groups on Grafana v11.x+: without it Grafana leaves the extracted user.Groups slice empty regardless of what the userinfo endpoint returns, and the gate rejects every login with “user not a member of one of the required groups” even when the token carries the correct claim. The JMESPath expression groups reads the top-level groups array from userinfo — the shape Keycloak’s oidc-group-membership-mapper emits.

  When spec.oidc.users[] is non-empty the chart additionally forces:

  skip_org_role_sync                = true
  oauth_allow_insecure_email_lookup = true
  allow_sign_up                     = false
  

  skip_org_role_sync=true keeps a login from overwriting the users-Job’s role assignments; oauth_allow_insecure_email_lookup=true lets Grafana attach the OIDC identity to the pre-provisioned local account by email; allow_sign_up=false is the isolation lever — without it, Grafana’s default allow_sign_up=true combined with skip_org_role_sync=true would mint a Viewer account for every cozy-realm identity that hits the login flow.

Prerequisite

System mode requires the platform-level OIDC feature (authentication.oidc.enabled at the Cozystack platform values). If the flag is off, the chart hard-fails the render with a clear message. Ask a Cozystack platform admin to enable it, or use CustomConfig.

Enable OIDC — CustomConfig mode

Bring your own issuer. Two supply paths, mutually exclusive:

spec:
  oidc:
    mode: CustomConfig
    customConfig:
      config:
        enabled: "true"
        client_id: my-grafana
        client_secret: xxxxxxxx
        auth_url: https://idp.acme.example/protocol/openid-connect/auth
        token_url: https://idp.acme.example/protocol/openid-connect/token
        api_url: https://idp.acme.example/protocol/openid-connect/userinfo
        scopes: openid email profile groups
    users:
      - email: alice@acme.example
        role: Admin

…or via a pre-existing Secret in the tenant namespace holding a ready-made [auth.generic_oauth] ini fragment in the auth.ini key:

spec:
  oidc:
    mode: CustomConfig
    customConfig:
      secretRef:
        name: acme-byo-grafana-auth

Setting both config and secretRef.name (or neither) fails the render. In CustomConfig mode no Keycloak objects are provisioned in cozy; the Grafana instance trusts the operator-supplied issuer directly.

The secretRef path is authoritative — the chart does not overlay any keys onto the operator-supplied ini fragment. If you use secretRef, spec.oidc.users[] MUST be empty (the chart fails the render otherwise) because the users-Job cannot reason about a config it cannot see.

On the inline config path, when users[] is non-empty the chart-forced users-map contract (skip_org_role_sync=true, oauth_allow_insecure_email_lookup=true, allow_sign_up=false) is merged over your map and wins every conflict: the chart renders merge $chartForced $yourConfig, and Helm’s merge keeps the left-hand (chart) values and only fills in keys your map is missing — so you cannot override those three. Unlike System mode, CustomConfig does not set enabled for you: your config MUST include enabled: "true" or Grafana leaves auth.generic_oauth disabled and no sign-in button appears. If your BYO IdP does not emit groups in the shape Grafana expects, add groups_attribute_path to your inline map to point at the right JMESPath.

Assigning roles

Roles are driven by spec.oidc.users[], not by group membership: each entry has an email (matched against the OIDC email claim on login) and a role (Grafana org-level: Admin, Editor, or Viewer). The users-Job creates or updates the local Grafana account on every reconcile and prunes stale members. Removing a user from the list revokes their access on the next chart apply — the users-Job removes them from the Grafana org (DELETE /api/orgs/1/users/<id>), an org-membership change rather than deletion of the underlying Grafana account; adding a user re-provisions them. This holds only while the list keeps at least one entry. The users-Job renders only for a non-empty users[], so removing the last entry skips the prune pass: the account keeps its org role and has to be removed from the Grafana org by hand. At the same point the chart stops forcing allow_sign_up=false and skip_org_role_sync=true, so Grafana’s default sign-up applies again and a member of the tenant groups can create a Viewer account on first login.

Login authorization is separate: the token owner MUST be a member of one of the release’s tenant-scoped cozy-realm groups (<namespace>-view, <namespace>-use, <namespace>-admin, <namespace>-super-admin) for allowed_groups to accept the login. That gate is unconditional in System mode, independent of users[].

Add a user to the appropriate tenant-scoped group in cozy — via the Keycloak UI or with a KeycloakRealmUser CR:

apiVersion: v1.edp.epam.com/v1
kind: KeycloakRealmUser
metadata:
  name: alice-acme
  namespace: cozy-keycloak
spec:
  realmRef:
    name: keycloakrealm-cozy
    kind: ClusterKeycloakRealm
  username: alice@acme.example
  email: alice@acme.example
  emailVerified: true
  passwordSecret:
    name: alice-acme-password
    key: password
  groups:
    - tenant-acme-admin

The cozy realm is a cluster-scoped ClusterKeycloakRealm named keycloakrealm-cozy, so spec.realmRef (not spec.realm) is required, with kind: ClusterKeycloakRealm. spec.passwordSecret points at a Secret in the same namespace holding the initial password (kubectl create secret generic alice-acme-password --namespace cozy-keycloak --from-literal=password=...); the CRD also accepts an inline spec.password, but it is deprecated in favour of passwordSecret.

Sign in

Open https://grafana.<host> and press “Sign in with Keycloak” under the login form. Grafana runs the OAuth Authorization Code flow against cozy, receives a token whose aud matches this Monitoring instance’s clientId, checks the groups claim against allowed_groups, then looks up the pre-provisioned Grafana account by email and grants the users-Job-assigned role.

The admin_user / admin_password field on the form stays wired to grafana-admin-password and continues to work.

Prerequisites and gotchas

• emailVerified: true on Keycloak users. email_verified is not enforced by Grafana: its auth.generic_oauth provider has no claim-validation option (claim handling is JMESPath only, and role_attribute_path is unused here because the users-Job drives roles). Enforce it on the Keycloak side instead: set emailVerified: true on the KeycloakRealmUser (or add the VERIFY_EMAIL required action / complete the email-verify flow in the Keycloak UI) so the identity holding a given email is guaranteed authentic. The cozy realm’s default duplicateEmails: false additionally prevents a second account from claiming an already-registered address.
• groups_attribute_path is not optional on Grafana v11.x+. The chart wires it automatically for System mode; in CustomConfig inline the operator must add it explicitly (groups_attribute_path: groups) if their IdP emits a top-level groups array. Otherwise allowed_groups becomes a silent no-op and every login fails.
• BYO issuer with a self-signed CA. In CustomConfig mode the secretRef path is the way to ship a CA bundle alongside the [auth.generic_oauth] block — you package auth.ini and any ca-cert files into the Secret and mount both under /etc/grafana/oidc. Placing the file is not enough on its own: point tls_client_ca at it from inside your auth.ini so Grafana actually trusts the issuer’s certificate.
• admin_user stays a break-glass path. Even under mode: System the login form and the grafana-admin-password Secret remain wired. Locking the form off is a follow-up hardening.
• Mode toggle is non-destructive. The users-Job renders only when spec.oidc.mode is not None and spec.oidc.users[] is non-empty. Flipping spec.oidc.mode from System to None therefore runs no reconcile and no prune pass: the Grafana accounts the Job provisioned survive untouched, and OIDC login is simply no longer offered on the form. Flipping back to System resumes reconciliation of spec.oidc.users[] against those existing accounts.

What’s out of scope for this feature

• Per-tenant Keycloak realms. Managed multi-tenant identity is a separate proposal, evaluated against Keycloak Organizations. Track it in the community proposal.
• Federating an external IdP into the platform cozy realm. BYO-for-Cozystack-itself is a distinct problem — this feature is BYO-for-a-managed-service.
• Full-logout through Keycloak’s end-session endpoint. Native auth.generic_oauth covers the OAuth part; backend-logout-url wiring is a follow-up.
• Server-level GrafanaAdmin promotion. All Grafana instances — platform and tenant — cap at org-level Admin; allow_assign_grafana_admin is not wired and the Monitoring CR exposes no field to opt in.
• Role granularity beyond Admin/Editor/Viewer. Grafana org roles are the assignment surface; team memberships / dashboard-level permissions stay out-of-band.