Kustomize as the Main Entrance for Change Management in Kubernetes

image credit > https://www.instagram.com/p/B49q8GFFO06/ (my own photo)

Background

One of my jobs is to manage the mediate size data center, running Kubernetes. One kind of challenges quite often rises up: how to make the change and how could we remember when/ how the change is made after 3 months?

For “how to make the change” is easy. There are various options available to change configuration in Kubernetes: updating chart in helm, applying yaml, directly editing deployment or statefulset.

But after 3 months, how can we remember this in a team? I lean to find out a solution to centralize the change deployment in standard method. kustomize comes into my sight.

Reference >
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/

https://kustomize.io/

One of my customers wants to replace the default theme with a customized theme in Keycloak (Keycloak is running in one namespace in Kubernetes cluster). It’s possible to copy the customized theme in a docker image. Obviously you would have to keep update when base image is updated and maintain many images if you have 10 of this kind of customers.

TL;DR

I want to flexibly de-couple the customized theme and base image, when the change is applied into production environment. Here is one simple practice for example.

Apply a customized theme for KeyCloak by kustomize

Kustomize is Kubernetes native configuration management tool. Kustomize introduces a template-free way to customize application configuration that simplifies the use of off-the-shelf applications.

It becomes a kind of standard and the simplest way to manage configuration change in Kubernetes cluster.

Prepare customized theme property files for Keycloak

• Create a directory customized_theme and copy customized theme file to ~/kustomize/customized_theme

ubuntu@worker1:~/kustomize$ tree customized_theme
customized_theme
└── login
    ├── login.ftl
    ├── messages
    │   └── messages_en.properties
    ├── resources
    │   ├── css
    │   │   ├── login.css
    │   │   └── login.css.orig
    │   └── img
    │       ├── favicon.ico
    │       ├── feedback-error-arrow-down.png
    │       ├── feedback-error-sign.png
    │       ├── feedback-success-arrow-down.png
    │       ├── feedback-success-sign.png
    │       ├── feedback-warning-arrow-down.png
    │       ├── feedback-warning-sign.png
    │       ├── keycloak-bg.png
    │       ├── keycloak-logo.png
    │       ├── keycloak-logo-text.png
    │       ├── customized_Background.png
    │       └── customized_logo.png
    └── theme.properties
5 directories, 17 files

• Create a tarball for such directory (because configMap doesn’t like folder with files)

tar -cvf customized_theme.tar ./customized_theme

• Create a configMap from the above tarball

kubectl -n shared create configmap keycloak-cutomized-theme --from-file=./customized_theme.tar

Generate base resource for kustomize

• Extract keycloak statefulSet into keycloak-sts.yaml. Edit it and remove all runtime specification, such as creationTimestamp, softLink, uid, and status

kubectl -n shared get statefulsets.apps keycloak -o yaml | neat > keycloak-sts.yaml

• Another option to generate a clean output yaml by yq

kubectl -n shared get sts keycloak -o yaml | \
  yq eval 'del(.metadata.creationTimestamp, .metadata.uid, .metadata.selfLink, .metadata.managedFields, .status)' -

Generate overlaypatch YAML for kustomize

ubuntu@node1:~/kustomize$ cat keycloak-sts-patch.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: keycloak
  namespace: shared
spec:
  template:
    spec:
      volumes:
      - name: theme
        configMap:
          name: keycloak-cutomized-theme
      containers:
      - name: keycloak
        volumeMounts:
        - name: theme
          mountPath: "/mnt"
        lifecycle:
          postStart:
            exec:
              command: ["/bin/sh", "-c", "tar -xvf /mnt/customized_theme.tar -C /opt/jboss/keycloak/themes/"]

kustomize patch

• Generate kustomization.yaml for kustomize

ubuntu@node1:~/kustomize$ cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: shared
resources:
  - keycloak-sts.yaml
patchesStrategicMerge:
  - keycloak-sts-patch.yaml

• Apply patch

kubectl apply -k .

PS. The reasons that I chose the image for this article: when I search kustomize, the customized motorcycle popped up and I ride Ducati bike.