Introduction to Secrets in Kubernetes

In the previous article, we discussed how we can store non-confidential data as key-value pairs using ConfigMaps. In this article, we will look into how we can use Secrets in Kubernetes to store confidential data.

What are Secrets in Kubernetes?

A Secret is a resource type in Kubernetes that is meant to hold a small amount of confidential data such as passwords, tokens or keys. Secrets allow us to decouple sensitive data from application code and container images. It also eliminates the need to put sensitive data in Pod specifications too, which in turn minimizes the risk of them being exposed when creating, editing or viewing Pods.

Like ConfigMaps, Secrets in Kubernetes are not designed to hold large chunks of data. In fact, we can only store up to 1MiB of data in a Secret. Similar to ConfigMaps Secrets can be used as environment variables in containers or can be mounted to containers as files in a volume. In addition, the Kubelet can use them as image pull secrets when pulling container images.

Secrets can be created either using imperative Kubectl commands or using manifest files, similar to how we created ConfigMaps.

Important!
Secrets are stored in the etcd database in the Kubernetes cluster in plain text. Therefore, anyone with API access to the cluster or access to etcd can view, modify Secrets. Furthermore, anyone who has access to create workload resources such as Pods and Deployments could also have indirect access to Secrets. To learn about how to manage and improve the security of Secrets in Kubernetes, you can refer to the Good Practices for Kubernetes Secrets documentation.

Write a Manifest for a Secret

Writing a manifest for a Secret is pretty easy. You would notice that it is pretty similar to a ConfigMap, but with a few differences. Following is an example of a very simple Secret. Eventually I would use this Secret to create a MySQL Pod.

apiVersion: v1
kind: Secret

metadata:
  name: mysql-secret

type: Opaque

data:
  MYSQL_ROOT_PASSWORD: Y2hhcml0aA==

Let us have a closer look at this Secret. You would notice that the first two sections are pretty much the same as a ConfigMap.

Version and Kind

Line 1 – apiVersion defines the Kubernetes API version. For a Secret, it is v1 at the time of writing this article.
Line 2 – kind defines the type of resource we are going to create. In this case the kind is Secret.

Metadata

Line 4 – the start of the metadata section.
Line 5 – here I have defined a name for my Secret as mysql-secret. You can give any name using alpha numeric characters and hyphens. Spaces are not allowed.

Type

Line 7 – This is a notable difference between the manifest of a ConfigMap and Secret. The type is defined here as Opaque. Opaque type is used for storing arbitrary user defined data. But there are many other types used for certain specific cases. You can read more about them in Kubernetes Documentation. We might also cover a few of them in future articles of this series.

Data

Line 9 – the start of the data section.
Line 10 – a key with the name MYSQL_ROOT_PASSWORD that has Y2hhcml0aAo= as the value has been defined. This is the actual piece of sensitive data we are storing in this Secret.

Now before you start applauding me for using such an impressive unintelligible password with upper/lower case letters, numbers and symbols, there is another important thing you must be aware of when writing a manifest for a Secret. That is, whatever the value you store here must be base64 encoded.

To base64 encode a string, we can use the following command and use the output string in the Secret.

$ echo -n charith | base64
Y2hhcml0aA==

Now you realize how inferior my password is. 😀 If you have any other data, you can define them too as key-value pairs under the data section.

Create the Secret Using the Manifest

Now that we have a manifest for the Secret object, we can run the kubectl apply command followed by the manifest file name to create the Secret.

$ kubectl apply -f mysql-secret.yaml
secret/mysql-secret created

If we need to view the Secret we just created, we can simply run the kubectl describe command, like for any other resource type. The output would be like below.

$ kubectl describe secret mysql-secret
Name:         mysql-secret
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
MYSQL_ROOT_PASSWORD:  7 bytes

We can see that the Secret has been successfully created. The type is Opaque and the MYSQL_ROOT_PASSWORD is there, but the value is not shown. Also notice that the namespace is default. This is because Secrets in Kubernetes are namespaced, which means they can only be accessed by other resources that are in the same namespace.

Create Secrets in Kubernetes Using Kubectl

As with ConfigMaps, it is possible to create Secrets in Kubernetes using imperative Kubectl Commands as well. If you are creating a simple Secret such as the one we just created in this article, it is in fact much easier with an imperative command. The command would look as below.

$ kubectl create secret generic mysql-secret --from-literal MYSQL_ROOT_PASSWORD=charith
secret/mysql-secret created

The command we have used is kubectl create secret generic.Here mysql-secret is the name of the Secret. Using the --from-literal flag, we pass the key-value pairs. When we create Secrets this way, we don’t have to base64 encode the values, because Kubectl will do that for us! Neat, right?

You can add more key-value pairs to the Secret by adding more --from-literal flags followed by a key-value pair as necessary. If you do a kubectl describe for this object, you would see that it is no different to the one we created using the manifest.

Now that we know how to create a Secret, let’s see how to use Secrets in our Pod configurations.

Configure a Pod to Use a Secret

In the documentation for MySQL official image in DockerHub, under the environment variables section, there is a mandatory environment variable called MYSQL_ROOT_PASSWORD. In earlier articles of this series, we directly defined the value for this in the Pod manifest. This time, let’s provide the value using the Secret we just created.

Create a MySQL Pod to Consume the Secret

We can write a simple manifest for the MySQL Pod as below.

apiVersion: v1
kind: Pod

metadata:
  name: mysql-pod
  labels:
    app: mysql

spec:
  containers:
  - name: mysql
    image: mysql
    ports:
    - containerPort: 3306
    env:
    - name: MYSQL_ROOT_PASSWORD
      valueFrom:
        secretKeyRef:
          name: mysql-secret
          key: MYSQL_ROOT_PASSWORD
    - name: MYSQL_DATABASE
      valueFrom:
        configMapKeyRef:
          name: mysql-config
          key: MYSQL_DATABASE

What you can see above is the full manifest for the MySQL Pod we are going to create. For the purpose of this article, let’s just have a closer look at the following part of the environment variable section.

    env:
    - name: MYSQL_ROOT_PASSWORD
      valueFrom:
        secretKeyRef:
          name: mysql-secret
          key: MYSQL_ROOT_PASSWORD

If you look at the environment variable MYSQL_ROOT_PASSWORD I have retrieved the value from the Secret I previously created.

Line 2 – the name of the environment variable.
Line 3 and 4 – basically says to fetch the value for MYSQL_ROOT_PASSWORD from a key defined in a Secret.
Line 5 – name is the name of the Secret which is mysql-secret.
Line 6 – key is the key in the Secret where the value is stored at. In this case the key is MYSQL_DATABASE.

So what this means is, when we start a MySQL Pod using this manifest, MySQL root password we should use to login to MySQL is ‘charith’ as we have defined in the Secret. You might have also noticed that there is another environment variable called MYSQL_DATABASE which fetches the value from a ConfigMap as we discussed in the article Introduction to ConfigMaps in Kubernetes. You can either follow that article to create the ConfigMap or just remove the whole environment variable from the manifest(last 5 lines) as it is not mandatory.

Now we can create the pod using kubectl apply command as before.

$ kubectl apply -f mysql-pod.yaml
pod/mysql-pod created

Once we see in the output that the Pod is created, if we run a kubectl describe command, we can see that the value for the environment variable is fetched from the Secret under the ‘Environment’ section.

 Environment:
      MYSQL_DATABASE:       <set to the key 'MYSQL_DATABASE' of config map 'mysql-config'>   Optional: false
      MYSQL_ROOT_PASSWORD:  <set to the key 'MYSQL_ROOT_PASSWORD' in secret 'mysql-secret'>  Optional: false

Login to the Database Using the Root Password

To check if the MySQL root password is properly set, let’s try to go inside the container and see if we can login to MySQL.

Run kubectl exec -it mysql-pod -- sh command to open an interactive terminal inside the Pod.
Run mysql -u root -p to login as root user and enter the root password when prompted. in the Secret, I set the value for MYSQL_ROOT_PASSWORD as charith, so that should be the password.

$ kubectl exec -it mysql-pod -- sh
sh-4.4# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 8.0.32 MySQL Community Server - GPL

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

Once the password is entered, I can see the mysql prompt, which means I have successfully logged in!

So this is it for this article! Now you know how to create simple Secrets in Kubernetes and how to use them in Pods. There are other types of Secrets and other ways you can use Secrets in Pods. We will look into them in future articles of this series! Hope you learned something in this article and drop a comment if it was helpful! 🙂