Installing the Grafbase Enterprise Platform

In this guide, we go through the steps on how to install the Grafbase Enterprise Platform on your Kubernetes cluster. For simplicity, in this guide we use kind to create a local Kubernetes cluster.

Prerequisites

Before you start the installation, make sure you have the following prerequisites:

Creating the Kind Cluster

First, create a kind cluster. Use the following configuration for the cluster:


kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
      # api
      - containerPort: 30080
        hostPort: 30080
        protocol: TCP
      # dashboard
      - containerPort: 30081
        hostPort: 30081
        protocol: TCP
      # postgres
      - containerPort: 30432
        hostPort: 30082
        protocol: TCP
      # clickhouse
      - containerPort: 30900
        hostPort: 30083
        protocol: TCP
      # minio api
      - containerPort: 30801
        hostPort: 30084
        protocol: TCP
      # minio console
      - containerPort: 30800
        hostPort: 30085
        protocol: TCP
      # gateway
      - containerPort: 30500
        hostPort: 30086
        protocol: TCP
      # telemetry sink
      - containerPort: 30431
        hostPort: 30087
        protocol: TCP

Save the configuration to a file, for example kind-config.yaml, and create the cluster:


kind create cluster --name grafbase-enterprise-platform --config kind-config.yaml
kubectl config use-context kind-grafbase-enterprise-platform

Installing the Grafbase Enterprise Platform

Before you start the installation, log in to the grafbase enterprise repository:


helm registry login helm.grafbase.com --username me@company.com --password changeme

When done, download the Grafbase Enterprise Platform Helm chart:


helm pull oci://helm.grafbase.com/enterprise-platform/stable/enterprise-platform

Create a new file values.yaml.gotmpl with the following values:


global:
  zitadel:
    enabled: true
  postgres:
    enabled: true
  clickhouse:
    enabled: false
  minio:
    enabled: true
  wreplicated:
    enabled: true
  kafka:
    enabled: false
  telemetrySink:
    enabled: false
  otelCollector:
    enabled: false
  dashboard:
    enabled: true

api:
  configmap:
    values:
      AWS_ACCESS_KEY_ID: changeme
  service:
    type: NodePort
    nodePort: 30080
  secrets:
    values:
      AWS_SECRET_ACCESS_KEY: changeme

dashboard:
  service:
    type: NodePort
    nodePort: 30081

minio:
  service:
    type: NodePort
    nodePorts:
      api: 30800
      console: 30801

Install the helm chart with the following command:


helm install grafbase-enterprise-platform enterprise-platform-*.tgz -f values.yaml

Wait a bit for the cluster to get ready, and you can see the API web worker crashing due to not being able to access the AWS S3 bucket:


> kubectl logs grafbase-enterprise-platform-api-web-69644fb98b-djgt2
{"timestamp":"2025-01-20T18:14:10.955429888+00:00","level":"INFO","fields":{"message":"API running at 0.0.0.0:8080"},"target":"api::runtime::std::serve","service":"grafbase-api","branch":"","environment":"local"}
{"timestamp":"2025-01-20T18:14:17.681903592+00:00","level":"ERROR","fields":{"message":"storage error: service error"},"target":"api::runtime::http::handlers","service":"grafbase-api","branch":"","environment":"local"}

In this example, use MiniIO as the S3 bucket, and create and set the access key and secret key to the secrets. Start first by opening MinIO console and log in with user grafbase and password grafbase. Click on "Access Keys" and create a new access key. Copy the access key and secret key to the values:


api:
  configmap:
    values:
      AWS_ACCESS_KEY_ID: ACCESS KEY FROM MINIO
  secrets:
    values:
      AWS_SECRET_ACCESS_KEY: SECRET KEY FROM MINIO

Upgrade the helm chart:


helmfile apply --environment=local -f helmfile/enterprise-platform.yaml

The environment variables are not reloaded automatically, so you need to restart the API pods:


kubectl rollout restart deployment grafbase-enterprise-platform-api-web

After waiting a few seconds, check if the API is running:


> curl localhost:30080/health
"Healthy"

In a real scenario, use a real S3 bucket. Create a bucket grafbase in your storage provider, and set the access key and secret key with read and write access to the bucket. You can set the RUNTIME_ASSETS_STORAGE configmap value to point to the bucket URL.

The next step involves creating a new Zitadel organization. The official Zitadel chart does not yet support creating a nodePort, so do a trick to be able to access the Zitadel UI from our kind cluster. These steps do not apply to a real cluster, where you have an ingress setup for Zitadel and can access it with a domain name.

Add the following line to your /etc/hosts file:


127.0.0.1 grafbase-enterprise-platform-zitadel

Then, add port forwarding to the Zitadel service:


kubectl port-forward service/grafbase-enterprise-platform-zitadel 30550:8080

Open the first-time admin UI by visiting http://localhost:30081/admin. Click "Run script" followed by "Manage single sign-on".

Log in to Zitadel with the username zitadel-admin@zitadel.grafbase-enterprise-platform-zitadel and password Password1!. Skip the 2-factor setup for now and enter a new password for the default user.

Change the organization to "Grafbase" from the top navigation.

Navigate to "Settings", click "Identity Providers", then "GitHub". To use GitHub as an authentication provider, create a new OAuth application in GitHub. Go to GitHub OAuth Apps, give the app a name and URL, and set the callback URL with the value from Zitadel. Now copy the client ID and the secret created in GitHub to the corresponding fields in Zitadel. Open optional settings, add the scope user:email, and check all the checkboxes. Click "Save", scroll up, and click "Activate".

To fetch the user email via GitHub, navigate to "Actions" and click "New". Name the action as "backfillEmail" and fill it with the following snippet:


let http = require("zitadel/http")

function backfillEmail(ctx, api) {
  const { human } = ctx.v1.externalUser

  // This is an example of how email is added with GitHub auth
  if (!human.email?.length) {
    const emails = http.fetch('https://api.github.com/user/emails', {
      method: 'GET',
      headers: {
        "Authorization": `Bearer ${ctx.accessToken}`
      }
    }).json()

    const primaryEmailAddress = emails.find(email => email.primary)
    if (primaryEmailAddress) {
      api.setEmail(primaryEmailAddress.email)
    }
  }
}

Click "Add", then click the "Add trigger" button. Set the trigger type to "Post Authentication" and actions to "backfillEmail" and "backfillName". Click "Save".

Now navigate to the Grafbase Dashboard and log in with your GitHub credentials. The dashboard should prompt you to create a new organization.

Installing the Telemetry Architecture

The Grafbase Enterprise Platform processes telemetry events from the Grafbase Gateway using a service called telemetry sink. To enable this service in your local installation, set the global values to true for both the sink and the opentelemetry collector in the values.yaml.gotmpl file:


global:
  telemetrySink:
    enabled: true
  otelCollector:
    enabled: true

Add a node port for the sink to access the sink locally:


telemetry-sink:
  service:
    type: NodePort
    nodePort: 30431
  configmap:
    values:
      GRAFBASE_KAFKA_BOOTSTRAP_SERVERS: "list_your_kafka_servers_here"
      GRAFBASE_KAFKA_USERNAME: "change me"
      GRAFBASE_KAFKA_MECHANISM: "SCRAM-SHA-256"
      GRAFBASE_KAFKA_SECURITY_PROTOCOL: "sasl_ssl"
  secrets:
    values:
      SYMMETRIC_ENCRYPTION_SECRET: "must be the same as in the API chart"
      PG_CONNECTION_STRING: "if not set, uses the local postgres"
      GRAFBASE_KAFKA_PASSWORD: "change me"
      CLICKHOUSE_CONNECTION_STRING: "if not set, uses the local clickhouse"

For the opentelemetry collector:


opentelemetry-collector:
  extraEnvs:
    - name: KAFKA_BROKER
      value: "address of the broker"
    - name: KAFKA_USER
      value: "change me"
    - name: KAFKA_PASSWORD
      value: "change me"
    - name: KAFKA_SASL_MECHANISM
      value: SCRAM-SHA-256
    - name: KAFKA_TLS_INSECURE
      value: "false"
    - name: CLICKHOUSE_ENDPOINT
      value: "must be the same as in the telemetry-sink"
    - name: CLICKHOUSE_DB
      value: "must be the same as in the telemetry-sink"
    - name: CLICKHOUSE_USER
      value: "change me"
    - name: CLICKHOUSE_PASSWORD
      value: "change me"

Ensure your Kafka server has a user with the corresponding password having access to the topics: otlp_metrics, otlp_spans, and otlp_logs. After updating these values with Helm, perform a rolling restart of both the opentelemetry collector and the telemetry sink. Finally, verify the logs for both pods to ensure there are no Kafka errors in the output.

Using Grafbase CLI with the Enterprise Platform

You can define your instance using the Grafbase CLI by setting the URL to your local installation in the CLI login command:


grafbase login --url http://localhost:30081/

Now, you are ready to use the CLI with your local Grafbase Enterprise Platform installation.

Using the Grafbase Gateway with the Enterprise Platform

To connect the Grafbase Gateway with the Grafbase Enterprise Platform, set the GRAFBASE_OTEL_URL environment variable to point to the telemetry sink URL.