Helm Is Not Enough, You Also Need Kustomize
Customize the YAML’s to enforce policies from application operators, security operators, and cluster operators. ✊
Kubernetes let us to declaratively specify our intent on how the applications should be deployed in the underlying infrastructure through YAML. These YAML’s will have application definition, tags needed for governance, tags needed for cross-cutting concerns like logging, application security context definitions, application resource dependencies, etc. The moment we start scaling our Kubernetes to ten’s and hundred’s of pods, it will become a nightmare to manage all YAML’s.
To put us in perspective these declarative configs can be classified into three main categories
- Application Packaging
- Application Config
- Runtime Config
Can’t Helm help do this already with ease? why do we need a new tool? What is that the new tool can bring to the table? How Helm and Kustomize can together enable some of the powerful use-cases? Lets attempt to answer all these questions throughout this blog.
What Helm is good and not good?
First, let’s have a look into the different technical capabilities needed for deploying applications into Kubernetes.
Application Packaging & Description
A capability that will help us to describe the application and package relevant resources together eg., 1) Application Metadata definition like resource constraints, 2) Packing a pod with config and service
Dependency Management
A capability that will help us to define dependencies like, deploy the config resource before deploying pod resource.
Application Discovery & Dashboard
A capability that will help us to discover the application deployed in the cluster, and their deployment history, dependencies, etc.,
Life Cycle Management
A capability that will help us to execute different application deployment strategies like rollout, rollback, blue-green deployment.
Application Description Customization
A capability that will help us to add application/security/cluster tags into YAML for governance/security/cross-cutting concerns
Helm can do all of this very well but not a very good tool for use-cases falling under the category “Application Description Customization”. Let's look at some solid examples to understand this.
Case 1: Cost Of Maintenance — Git Fork or Copy
Let’s say we are using Nginx helm charts to deploy Nginx WAF into our cluster. As a part of the product team requirement, we should add an annotation to specify the team name responsible for the deployment. We will not be in a position to make the changes to the main Nginx git repository, as it’s only relevant to application operation governance policy in our organization. To achieve this we should fork or copy the chart template from the Nginx repository and modify it for our custom need. Forks / Copy are costly to maintain. Kustomize can help here.
Case 2: Leaky Abstraction
Let’s say we have some Network Policy to be applied to a set of pods. Security Operator is looking to add a label to all the relevant pods, then he can add a NetworkPolicy resource to all the pods with the given label. With the traditional helm way, we should add the label to the chart templates which developers work on. We are clearly leaking the security declarative tags to application developers.
Case 3: Coupling Between Different Cross-cutting Concerns
On Kubernetes resource YAML there are declarative tags concerning application developers, application operators, security operators, and cluster operators. Application, security and cluster operators' concerns are cross-cutting by nature. Coupling all these concerns will hinder the team's agility.
Case 4: Apply Patch in continuous deployment pipe
Let’s say that there is a security vulnerability in the Nginx pod, the security team is looking for a way to intercept all continuous deployment pipelines and update the Nginx image version with the latest. Kustomize will work very well in this scenario.
This is the sweet spot for Kustomize. It helps us to customize the YAML files leaving the original YAML untouched. Kustomize is a simple command-line CLI tool which can be added into any continuous deployment / integration workflow or even as an independent tool as needed
The below figure can give us a high-level perspective on how Helm and Kustomize can work together to create some power workflows.
How Kustomize works?
Its pretty simple three steps
- Add “kustomization.yaml” to the YAML’s that we are looking to customize
- Define your customizations into the YAML
- Then run “kustomize build”
Now, what goes into the kustomization.yaml? Let's take an example and look at the relevant “kustomization.yaml”
Example 1: We have a pod deployed by the shopping cart team, for the mobile app team to use. Also, there is a requirement from the security team to keep all the services used by the mobile app team in “backend-for-frontend” namespace. In addition to this, the application operator has a cross-cutting label that includes the team name for distributed tracing. The “kustomization.yaml” for the above example will look like this
The Pod YAML
The kustomization YAML
The output of running “kustomize build”
If you want to install directly into your kubernetes cluster use
kustomize build | kubectl apply -f -
Helm ➕ Kustomize
Using helm and kustomize together is a very simple three-step process. Let's do that for installing MariaDB.
Save a helm template after overriding values file into mariadb.yaml. In the below example values are replaced from config.yaml
helm template -f config.yaml stable/mariadb > mariadb.yaml
Then add kustomization YAML for mariadb.yaml
Finally, customize the changes and apply to the cluster
kustomize build | kubectl apply -f -
Read more about an end to end DevOps workflow Workflow for Kubernetes DevOps.
Hope the reading is useful. Let's create some powerful declarative workflows 🚀. See you in an upcoming article🏄
