CI/CD and OCP Deployment Guide

Reference doc for deploying NestJS/Node.js backends to OCP via Jenkins. Based on the feg_slot_be (Nx monorepo) setup. Adaptable for other backends.

Stack

OCP Namespaces

Two namespaces are used - this is intentional:

Helm deploys to gaming-studio-shared. The helm-gaming-studio-deployer SA in gaming-studio namespace has RBAC permissions to deploy to gaming-studio-shared.

Jenkins Pipeline Structure

Pod Agent

agent {
  kubernetes {
    inheritFrom 'node24 buildah'  // merges two base pod templates
    defaultContainer 'node24'     // node24 is default for most stages
  }
}

• node24 container: runs Node.js, pnpm, helm commands
• buildah container: runs image builds only (needs privileged access for OCI builds)
• container('buildah') directive required when running buildah commands

Stage Flow

Setup -> Install -> Detect Affected Packs -> Build & Push -> Deploy TST

Only Build & Push and Deploy TST run in container('buildah') and with Vault respectively. All other stages run in default node24 container.

Key Pattern: SKIP_BUILD

When no packs are affected by changes, pipeline exits gracefully:

def affectedPacks = readFile('affected-packs.txt').trim()
if (!affectedPacks) {
  env.SKIP_BUILD = 'true'
}

All subsequent stages check when { expression { env.SKIP_BUILD != 'true' } }. Without this, grep on empty input exits with code 1 and fails the build.

Detecting Affected Packs (Nx)

Uses git tags as deployment baselines:

• last-tst-deploy tag = last commit deployed to TST
• last-prod-deploy tag = last commit deployed to PROD

git fetch --depth=100 origin "+refs/tags/last-tst-deploy:refs/tags/last-tst-deploy" || true
FALLBACK=$(git rev-list --max-parents=0 HEAD)
NX_BASE=$(git rev-parse --verify last-tst-deploy 2>/dev/null || echo $FALLBACK)
pnpm nx show projects --affected --type=app --base=$NX_BASE \
  | grep -v '^>' | grep -v '^$' > affected-packs.txt || true

|| true on the grep pipeline prevents exit code 1 on empty input from failing the stage.

Proxy Configuration

Internal FEG network requires proxy for internet access:

environment {
  http_proxy  = 'http://10.0.68.20:3128'
  https_proxy = 'http://10.0.68.20:3128'
  no_proxy    = 'nexus.svc.ifortuna.cz,registry.svc.ifortuna.cz,sonatype-nexus-service.nexus-shared.svc,*.svc.cluster.local,*.svc'
}

Helm must bypass proxy to reach K8s API server (in-cluster):

HTTP_PROXY="" HTTPS_PROXY="" http_proxy="" https_proxy="" helm upgrade --install ...

Nexus npm Auth

Nexus uses Basic auth (_auth not _authToken):

NEXUS_HOST=$(echo "$NEXUS_URL" | sed 's|https://||;s|http://||')
npm config set //${NEXUS_HOST}/repository/gaming-studio-npm/:_auth $NEXUS_TOKEN

NEXUS_TOKEN is a Base64-encoded user:password string stored in Jenkins credentials as nexus-token.

Vault-Based OCP Auth

Pattern used by all FEG projects (from jenkins-shared-library):

• Vault path: jenkins/<project-name> (NOT <project>/kubeconfig_bm)
• Keys: kubeconfig and kubeconfig_bm

For gaming-studio:

withVault(vaultSecrets: [[path: 'jenkins/gaming-studio', secretValues: [
  [$class: 'VaultSecretValue', envVar: 'K8S_CONFIG_BM', vaultKey: 'kubeconfig_bm']
]]]) {
  sh '''
    dec="$(printf %s "$K8S_CONFIG_BM" | base64 -d 2>/dev/null || true)"
    if [ -n "${dec:-}" ] && printf %s "$dec" | grep -qE '^(apiVersion:|kind: Config|clusters:)'; then
      echo "$K8S_CONFIG_BM" | base64 -d > "$WORKSPACE/kubeconfig-tst"
    else
      echo "$K8S_CONFIG_BM" > "$WORKSPACE/kubeconfig-tst"
    fi
    chmod 600 "$WORKSPACE/kubeconfig-tst"
    KUBECONFIG="$WORKSPACE/kubeconfig-tst" helm upgrade --install ...
    rm -f "$WORKSPACE/kubeconfig-tst"
  '''
}

The base64 detection handles both base64-encoded and plain kubeconfig formats (different teams store them differently in Vault).

Fallback (if Vault access not granted yet): store kubeconfig as Jenkins file credential and use:

withCredentials([file(credentialsId: 'ocp-gaming-studio-kubeconfig-tst', variable: 'KUBECONFIG')]) { ... }

Tagging After Deploy

After successful deploy, tag the commit so next run uses it as baseline:

git tag -f last-tst-deploy
git push origin last-tst-deploy --force

Requires sshagent(credentials: ['jenkins-secret-bitbucket']) and setting:

git remote set-url origin $BB_SSH_URL  # SSH URL, not HTTPS
git config --global --add safe.directory '*'

Deploy PROD Input Gate

input directive at stage level runs BEFORE when condition is evaluated. Must be inside steps {} to respect when:

// WRONG - input runs even on main branch
stage('Deploy PROD') {
  when { branch pattern: 'release/.*', comparator: 'REGEXP' }
  input message: "Deploy to PROD?"   // this ignores when condition
  steps { ... }
}

// CORRECT
stage('Deploy PROD') {
  when { branch pattern: 'release/.*', comparator: 'REGEXP' }
  steps {
    input message: "Deploy to PROD?", ok: "Approve"  // inside steps
    sh '...'
  }
}

Dockerfile (Dockerfile.ocp)

Multi-Stage Build

pack-name-check  ->  validates PACK_NAME arg provided
node-base        ->  Node.js 24 + pnpm install (reused by builder and prod-deps)
builder          ->  full build (pnpm install + nx build <PACK_NAME>)
prod-deps        ->  production-only dependencies
runtime          ->  minimal runtime image (no build tools)

BuildKit Secrets for Nexus Auth

Avoids baking tokens into image layers:

RUN --mount=type=secret,id=nexus_token \
    npm config set //nexus.svc.ifortuna.cz/repository/gaming-studio-npm/:_auth "$(cat /run/secrets/nexus_token)" && \
    pnpm install --frozen-lockfile && \
    npm config delete //nexus.svc.ifortuna.cz/repository/gaming-studio-npm/:_auth

In pipeline, secret is passed via file:

echo -n "$NEXUS_TOKEN" > /tmp/nexus_token.txt
buildah build-using-dockerfile \
  --secret id=nexus_token,src=/tmp/nexus_token.txt \
  ...
rm -f /tmp/nexus_token.txt

OCI Format - No HEALTHCHECK

buildah uses OCI image format by default. OCI does not support HEALTHCHECK instruction. Do NOT add HEALTHCHECK to Dockerfile.ocp - buildah will warn and ignore it.

Health checks are handled by Helm probes (liveness + readiness) instead:

livenessProbe:
  httpGet:
    path: /games/health
    port: 8000
readinessProbe:
  httpGet:
    path: /games/health
    port: 8000

Security

RUN groupadd -r appgroup && useradd -r -g appgroup appuser
RUN chown -R appuser:appgroup /app
USER appuser

Container runs as non-root. OCP security context also enforces:

securityContext:
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  capabilities:
    drop: [ALL]

buildah vs docker

buildah is Red Hat/OCP standard. Key differences:

• No Docker daemon required (daemonless)
• buildah build-using-dockerfile instead of docker build
• buildah push instead of docker push
• buildah login instead of docker login
• Use --tls-verify=false for internal FEG registry (self-signed cert)

Helm Chart

Structure

infra/helm/game-pack/
  Chart.yaml
  values.yaml           # defaults
  values-tst.yaml       # TST overrides
  values-stg.yaml       # STG overrides
  values-prod.yaml      # PROD overrides
  templates/
    deployment.yaml
    service.yaml
    route.yaml

Single chart, parameterised by packName. Deployed once per pack.

Key Design Decisions

1. packName as primary parameter Everything derives from packName: deployment name, service name, route name, configmap reference. Helm release name also equals packName for simplicity.

2. ConfigMap for env vars (not baked into Helm values) Env vars (RGS_URL, NODE_ENV, etc.) live in a separately-managed ConfigMap <packName>-config. Reason: ConfigMap can be updated without a Helm release. Helm just references it. ConfigMap must exist before first helm upgrade --install.

3. Secret is optional

# values.yaml
secret:
  enabled: false

# deployment.yaml
{{- if .Values.secret.enabled }}
- secretRef:
    name: {{ .Values.packName }}-secret
{{- end }}

Not all packs need secrets. Avoids Helm failing on missing Secret resource.

4. OCP Route auto-created per pack Route is created by Helm automatically. URL pattern: <packName>-<namespace>.apps.<cluster-domain> This URL is the upstream for APIGW (see routing-apigw-architecture.md).

5. Helm 4 uses server-side apply If resources were previously created manually (kubectl apply), Helm will conflict on first install. Resolution: patch resources with Helm ownership labels, or delete and let Helm recreate.

oc patch deployment <name> --type=merge -p '{"metadata":{"annotations":{"meta.helm.sh/release-name":"<name>","meta.helm.sh/release-namespace":"<ns>"},"labels":{"app.kubernetes.io/managed-by":"Helm"}}}'

Helm Install Command

HTTP_PROXY="" HTTPS_PROXY="" http_proxy="" https_proxy="" \
helm upgrade --install $PACK_NAME ./infra/helm/game-pack \
  -f ./infra/helm/game-pack/values-tst.yaml \
  --set image.tag=${PACK_NAME}-${IMAGE_TAG} \
  --set packName=$PACK_NAME \
  --namespace gaming-studio-shared

Proxy variables cleared so Helm reaches K8s API directly (not through proxy).

Jenkins Credentials Required

Checklist: New Backend Deployment on OCP

• Dockerfile.ocp: multi-stage, no HEALTHCHECK, non-root user, BuildKit secret for Nexus
• Helm chart: deployment, service, route templates; values per env; secret.enabled=false default
• Create ConfigMap <packName>-config in target namespace before first deploy
• Jenkins credentials: nexus-token, quay-push-credentials, jenkins-secret-bitbucket
• Vault access confirmed: jenkins/<project> path with kubeconfig_bm key
• Proxy cleared for Helm commands (HTTP_PROXY="" ...)
• SKIP_BUILD pattern for empty affected list
• input gate for PROD inside steps {}, not at stage level
• git remote set-url to SSH before tag push
• || true on grep commands that may have empty input