Google Cloud Credentials

SereneDB can read and write Google Cloud resources — most importantly Iceberg tables in a BigLake catalog and files in Cloud Storage. Google authenticates every request with an OAuth access token that expires after one hour, so the credential you configure is never the token itself: it is a long-lived credential from which SereneDB mints fresh tokens automatically, for as long as the server runs. You configure it once, through the Secrets manager.

This page explains the four kinds of Google credentials, how to obtain each one, and which one fits your situation.

Which credential should I use?

Your situation	Use
Production, SereneDB runs anywhere (on-prem, AWS, any datacenter)	Service account key
Production, SereneDB runs on Google Cloud (GCE, GKE)	Attached service account — no credential material at all
Personal use, development, trying things out from a laptop	Your own Google account (ADC)
Cloud Storage files only, no Iceberg catalog involved	HMAC keys — also usable as the data-plane half of a catalog setup

Service account key

A service account is a machine identity: it belongs to your project, not to a person, so it survives employee departures, ignores Workspace login policies, and can be provisioned by automation. Its key is a JSON file containing an RSA private key; SereneDB uses it to sign short-lived tokens (the JWT-bearer exchange) — the key itself never crosses the network. This is the credential to put in a production deployment.

Creating one

# 1. Create the service account
gcloud iam service-accounts create serenedb-engine \
    --project=my-project

# 2. Grant it access to the Iceberg catalog and the bucket
gcloud projects add-iam-policy-binding my-project \
    --member=serviceAccount:serenedb-engine@my-project.iam.gserviceaccount.com \
    --role=roles/biglake.editor
gcloud projects add-iam-policy-binding my-project \
    --member=serviceAccount:serenedb-engine@my-project.iam.gserviceaccount.com \
    --role=roles/serviceusage.serviceUsageConsumer
gcloud storage buckets add-iam-policy-binding gs://my-bucket \
    --member=serviceAccount:serenedb-engine@my-project.iam.gserviceaccount.com \
    --role=roles/storage.objectUser

# 3. Download the key (one-time download — store it safely)
gcloud iam service-accounts keys create key.json \
    --iam-account=serenedb-engine@my-project.iam.gserviceaccount.com

Attention

Organizations created after May 2024 have service-account key creation disabled by default. If step 3 fails with a policy error, an organization admin must lift iam.disableServiceAccountKeyCreation for the project: gcloud org-policies reset iam.disableServiceAccountKeyCreation --project=my-project

Using it

Open key.json and copy three of its fields into a secret. The private_key value can be pasted exactly as it appears in the file, \n sequences included:

Query

CREATE PERSISTENT SECRET gcp (    TYPE ICEBERG,    PROVIDER google,    CLIENT_EMAIL '⟨engine@my-project.iam.gserviceaccount.com⟩',    PRIVATE_KEY '⟨-----BEGIN PRIVATE KEY-----\nMIIE...\n-----END PRIVATE KEY-----\n⟩',    PRIVATE_KEY_ID '⟨a1b2c3d4...⟩',    EXTRA_HTTP_HEADERS MAP {'x-goog-user-project': '⟨my-project⟩'});

Attention

Paste the key inside plain '...' quotes. PostgreSQL's escape-string syntax (E'...') would consume the \n sequences and corrupt the key.

PRIVATE_KEY_ID is optional — it tells Google which of the account's keys (up to 10 can exist, e.g. during rotation) signed the request. TOKEN_URI (defaults to Google's token endpoint) and OAUTH2_SCOPE (defaults to https://www.googleapis.com/auth/cloud-platform) are also accepted. The private_key is stored redacted: it never appears in duckdb_secrets() output or catalog dumps.

Attached service account

When SereneDB runs on Google Cloud (a GCE VM, GKE), the machine itself has an identity — the attached service account — and tokens for it are served by a local metadata endpoint. There is no credential material at all: nothing to download, store, rotate, or leak. This is Google's recommended setup for production workloads on their cloud.

Grant the VM's service account the same three roles as above, make sure the VM has the cloud-platform access scope, and create the secret with no key fields:

Query

CREATE PERSISTENT SECRET gcp (    TYPE ICEBERG,    PROVIDER google,    EXTRA_HTTP_HEADERS MAP {'x-goog-user-project': '⟨my-project⟩'});

SereneDB detects that no key was given and fetches tokens from the metadata server instead.

Your own Google account (ADC)

For development and personal use, you can let SereneDB act as you. Run:

gcloud auth application-default login

A browser opens, you approve, and gcloud writes ~/.config/gcloud/application_default_credentials.json containing a refresh token — a long-lived credential tied to your Google account. Copy its three fields into a secret:

Query

CREATE PERSISTENT SECRET gcp (    TYPE ICEBERG,    OAUTH2_GRANT_TYPE 'refresh_token',    OAUTH2_SERVER_URI 'https://oauth2.googleapis.com/token',    CLIENT_ID '⟨client_id from the ADC file⟩',    CLIENT_SECRET '⟨client_secret from the ADC file⟩',    REFRESH_TOKEN '⟨refresh_token from the ADC file⟩',    EXTRA_HTTP_HEADERS MAP {'x-goog-user-project': '⟨my-project⟩'});

(client_id and client_secret in that file are gcloud's public application identifiers — the same for everyone; only the refresh_token is yours.)

Attention

This credential is your personal identity and is not suitable for shared or production servers: it stops working if your account is disabled, if your organization enforces a Google Cloud session-length policy (tokens then expire every few hours), or if it gets rotated out by Google's limit of 100 outstanding refresh tokens per account per OAuth client. Queries also appear in Cloud audit logs as you, not as the database.

HMAC keys

HMAC keys are an AWS-style permanent key pair for Cloud Storage's S3-compatible endpoint. They never expire and involve no tokens — every request is individually signed. Their limitation: they work only for Cloud Storage data files. The BigLake catalog endpoint accepts only OAuth tokens, so HMAC keys cannot attach an Iceberg catalog by themselves — but they can serve as the data-plane credential next to any of the catalog credentials above.

gcloud storage hmac create serenedb-engine@my-project.iam.gserviceaccount.com \
    --project=my-project

This prints an accessId (starts with GOOG1E...) and a one-time secret. Configure them as a gcs secret:

Query

CREATE PERSISTENT SECRET gcs_data (    TYPE GCS,    KEY_ID '⟨GOOG1E...⟩',    SECRET '⟨hmac secret⟩');

Attention

An HMAC key is a permanent secret stored in plain form. Prefer creating it for a service account (not a user), scope that account's access narrowly, and note that some organizations disable HMAC via the storage.restrictAuthTypes policy.

Putting a credential to work

The credential alone only proves identity. To actually connect to a BigLake Iceberg catalog — including the data-file half of the setup — follow the BigLake Iceberg cookbook. For the full menu of Iceberg catalog authentication methods across clouds, see Iceberg catalog authentication. To read plain files from Cloud Storage without a catalog, see GCS Import.