Gopal Gautam's Blog

Kubernetes Storage and Kubernetes Security

Kubernetes Storage and Kubernetes Security

Gopal Gautam's photo
Gopal Gautam
·

16 min read

Kubernetes is an open-source container orchestration platform that enables the deployment and management of containerized applications at scale. As such, it provides many features for managing containerized workloads, including storage and security.

Kubernetes provides robust storage and security features that can help to manage containerized workloads in a cluster efficiently and securely.

Kubernetes Storage

In Kubernetes, storage is used to provide persistent data storage for containers running in a cluster. Kubernetes provides several types of storage options, including local storage, network-attached storage (NAS), and cloud storage. Storage volumes can be mounted to a pod and accessed from multiple containers or pods within the same or different nodes. Kubernetes also provides built-in support for storage orchestration, which automates the creation, deletion, and resizing of storage volumes, and dynamic provisioning, which allows for the automatic creation of new storage volumes when needed. This makes it easier to manage persistent data in a containerized environment.

Persistent Volume (PV)

Persistent Volumes (PVs) in Kubernetes are a way to provide persistent storage to containers running in a Kubernetes cluster. PVs are networked storage volumes that can be attached to a container running in a Kubernetes cluster. They provide durable storage and are not tied to any particular node in the cluster. PVs are managed by the Kubernetes control plane and can be shared between multiple containers.

PVs are created by the cluster administrator and are available to all users of the cluster. Users can claim a portion of a PV for their application by creating a Persistent Volume Claim (PVC). Once a PVC is created, the Kubernetes control plane will find a suitable PV to satisfy the request.

Example of Creating a Persistent Volume

To create a PV, you need to define a PV manifest file in YAML format. Here is an example of a PV manifest file:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: myebsvol
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  awsElasticBlockStore:
    volumeID:           # here put the volume id
    fsType: ext4

To create the PV, save the manifest file to a file called pv.yaml, and then run the following command:

kubectl create -f pv.yaml

This will create the PV and make it available to the cluster.

Persistent Volume Claim (PVC)

A Persistent Volume Claim (PVC) is a Kubernetes resource that requests a certain amount of storage from a Persistent Volume (PV) resource. PVCs are used to provide a layer of abstraction between the storage requested by a pod and the storage actually provided by the underlying storage system. By using PVCs, a pod can request storage without having to know where that storage is coming from or how it is provisioned.

A PVC is created by a user of the Kubernetes cluster and specifies the desired storage capacity, access mode, and storage class. The Kubernetes control plane matches the PVC with an available PV that meets the PVC's requirements. Once the PVC is bound to a PV, the pod can use the PVC to access the storage.

Example of a Persistent Volume Claim

Here is an example of a PVC manifest file:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: myebsvolclaim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

The following manifest file can be used to attach the aforementioned Persistent Volume to a pod.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: pvdeploy
spec:
  replicas: 1
  selector:      # tells the controller which pods to watch/belong to
    matchLabels:
     app: mypv
  template:
    metadata:
      labels:
        app: mypv
    spec:
      containers:
      - name: shell
        image: centos
        command: ["bin/bash", "-c", "sleep 10000"]
        volumeMounts:
        - name: mypd
          mountPath: "/tmp/persistent"
      volumes:
        - name: mypd
          persistentVolumeClaim:
            claimName: myebsvolclaim

Storage Class

In Kubernetes, a StorageClass is an abstraction layer that allows users to define different classes of storage with different performance characteristics and then request storage resources using those classes. A StorageClass is a way to provision storage dynamically and automatically, without requiring manual intervention from the cluster administrator.

When a user requests storage by creating a PersistentVolumeClaim (PVC), they can specify the desired StorageClass. The Kubernetes control plane uses the StorageClass to provision a matching PersistentVolume (PV) with the requested capacity and other characteristics.

Here's an example of a StorageClass manifest file:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: fast
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2

In this example, we're defining a StorageClass called "fast" that uses the "kubernetes.io/aws-ebs" provisioner to provision storage on Amazon Elastic Block Store (EBS). The type parameter specifies that the storage should use the gp2 volume type, which provides fast and consistent performance.

Save the StorageClass manifest file to a file called storageclass.yaml, and then run the following command to create the StorageClass:

kubectl apply -f storageclass.yaml

Once this StorageClass is defined, a user can create a PVC that requests storage from this StorageClass, like so:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: fast

In this example, the storageClassName field is set to "fast", which tells Kubernetes to use the StorageClass we defined earlier to provision the requested storage.

Using StorageClasses makes it easy to dynamically provision storage resources that meet different performance and capacity requirements, and allows users to easily request and use the appropriate storage resources

Save the PVC manifest file to a file called pvc.yaml, and then run the following command to create the PVC:

kubectl apply -f pvc.yaml

This will create the PVC and bind it to a suitable PV in the cluster.

Now you can create a deployment that mounts the PVC as a volume in a container. Here's an example deployment manifest file that uses the PVC we created earlier:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        volumeMounts:
        - name: nginx-persistent-storage
          mountPath: /usr/share/nginx/html
      volumes:
      - name: nginx-persistent-storage
        persistentVolumeClaim:
          claimName: my-pvc

In this example, we're creating a deployment that runs a single replica of the nginx container image. We're also specifying a volumeMount that mounts the PVC at /usr/share/nginx/html inside the container.

To create the deployment, save this manifest file to a file called deployment.yaml, and then run the following command:

kubectl apply -f deployment.yaml

This will create the deployment and start the container. You can verify that the container is running by running the following command:

kubectl get pods

Statefullset

In Kubernetes, a StatefulSet is a higher-level abstraction over a Deployment that is designed to manage stateful applications, such as databases or other clustered applications that require stable network identifiers and persistent storage. A StatefulSet provides guarantees around the ordering and uniqueness of pod creation, scaling, and deletion.

Let's walk through an example of how to create a StatefulSet in Kubernetes.

Assuming you have a Kubernetes cluster up and running, you can follow these steps:

  1. Create a ConfigMap

Before creating a StatefulSet, you may want to create a ConfigMap to store configuration data for your application. Here's an example ConfigMap manifest file:

apiVersion: v1
kind: ConfigMap
metadata:
  name: myapp-config
data:
  app.properties: |
    database.host=mydb
    database.port=5432

In this example, we're creating a ConfigMap called "myapp-config" that contains some configuration data in the app.properties key.

To create this ConfigMap, save the manifest file to a file called configmap.yaml, and then run the following command:

kubectl apply -f configmap.yaml
  1. Create a StatefulSet

Next, you can create a StatefulSet to manage your application. Here's an example StatefulSet manifest file:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: myapp
spec:
  serviceName: myapp
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: myapp:latest
        ports:
        - containerPort: 8080
        envFrom:
        - configMapRef:
            name: myapp-config
        volumeMounts:
        - name: data
          mountPath: /data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: fast
      resources:
        requests:
          storage: 1Gi

In this example, we're creating a StatefulSet called "myapp" that uses the ConfigMap we created earlier. The StatefulSet specifies a replica count of 3, and defines a volumeClaimTemplate that creates a persistent volume for each replica.

To create this StatefulSet, save the manifest file to a file called statefulset.yaml, and then run the following command:

kubectl apply -f statefulset.yaml
  1. Verify the StatefulSet

You can verify that the StatefulSet has been created and is running correctly by running the following commands:

kubectl get statefulsets
kubectl get pods
kubectl describe statefulset myapp

The get statefulsets command will show you the StatefulSet that was created, the get pods command will show you the pods that were created by the StatefulSet, and the describe statefulset myapp command will give you detailed information about the StatefulSet.

That's it! Now you have a StatefulSet that is managing your application and providing stable network identifiers and persistent storage.

Kubernetes Security

Kubernetes security involves a range of practices and configurations aimed at securing the Kubernetes environment and the applications running on it. Here are some key components of Kubernetes security:

  1. Authentication and Authorization: Kubernetes supports various authentication mechanisms, such as client certificates, bearer tokens, and OpenID Connect tokens. Once authenticated, users are then authorized to access Kubernetes resources based on their RBAC (Role-Based Access Control) permissions.

  2. Network Security: Kubernetes provides various network security mechanisms, such as network policies, pod security policies, and encryption of network traffic using TLS.

  3. Secrets Management: Kubernetes provides a secrets API that allows sensitive information, such as passwords and access keys, to be stored securely and accessed only by authorized users and pods.

  4. Container Image Security: Container images used in Kubernetes should be scanned for vulnerabilities and signed with cryptographic keys to ensure their integrity. Kubernetes also provides admission controllers to enforce image security policies.

  5. Kubernetes API Security: The Kubernetes API can be secured using SSL/TLS, RBAC permissions, and API server authentication.

  6. Logging and Monitoring: Kubernetes provides various logging and monitoring mechanisms to help detect and respond to security incidents, such as audit logging, Prometheus metrics, and Kubernetes events.

  7. Compliance: Kubernetes supports compliance with various industry standards and regulations, such as HIPAA, PCI DSS, and GDPR.

Overall, Kubernetes security requires a comprehensive and layered approach, including best practices in network security, authentication and authorization, secrets management, container image security, Kubernetes API security, logging and monitoring, and compliance.

Role-Based Access Control (RBAC)

Role-Based Access Control (RBAC) is a method of restricting access to resources in a Kubernetes cluster based on roles assigned to individual users or groups. RBAC enables fine-grained control over what actions can be performed on Kubernetes resources, such as pods, services, and deployments.

Here's an example of how to use RBAC to control access to resources in a Kubernetes cluster:

  1. Create a Role

To create a role, you need to define a set of rules that specify what actions can be performed on which resources. Here's an example of a Role manifest file:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

In this example, we're creating a role called "pod-reader" that grants read access to pods.

To create this role, save the manifest file to a file called role.yaml, and then run the following command:

kubectl apply -f role.yaml
  1. Create a RoleBinding

A RoleBinding is used to associate a role with a user or a group. Here's an example of a RoleBinding manifest file:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pod-reader-binding
subjects:
- kind: User
  name: alice
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

In this example, we're creating a RoleBinding called "pod-reader-binding" that associates the "pod-reader" role with the user "alice".

To create this RoleBinding, save the manifest file to a file called rolebinding.yaml, and then run the following command:

kubectl apply -f rolebinding.yaml
  1. Verify the Role and RoleBinding

You can verify that the Role and RoleBinding have been created correctly by running the following commands:

kubectl get roles
kubectl get rolebindings
kubectl describe role pod-reader
kubectl describe rolebinding pod-reader-binding

The get roles command will show you the roles that were created, the get rolebindings command will show you the rolebindings that were created, and the describe role and describe rolebinding commands will give you detailed information about the role and rolebinding.

That's it! Now the user "alice" has read access to pods in the Kubernetes cluster.

Pod Security Policies (PSPs)

Pod Security Policies (PSPs) is a Kubernetes feature that allows cluster administrators to define and enforce security policies for pods. PSPs help ensure that pods are created with a minimum set of security requirements, such as running as non-root users, using read-only file systems, and using specific security contexts.

Here's an example of how to use Pod Security Policies to enforce security policies for pods in a Kubernetes cluster:

  1. Create a Pod Security Policy

To create a Pod Security Policy, you need to define a set of rules that specify what security requirements pods must meet. Here's an example of a Pod Security Policy manifest file:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
spec:
  privileged: false
  allowPrivilegeEscalation: false
  seLinux:
    rule: RunAsAny
  runAsUser:
    rule: MustRunAsNonRoot
  fsGroup:
    rule: MustRunAs
    ranges:
    - min: 1
      max: 65535
  volumes:
  - configMap
  - secret
  - emptyDir
  - downwardAPI
  - projected

In this example, we're creating a Pod Security Policy called "restricted" that requires pods to run as non-root users, disallows privilege escalation, and allows only specific types of volumes to be used.

To create this policy, save the manifest file to a file called podsecuritypolicy.yaml, and then run the following command:

kubectl apply -f podsecuritypolicy.yaml
  1. Create a RoleBinding for the PSP

To associate the Pod Security Policy with a specific user or group, you need to create a RoleBinding. Here's an example of a RoleBinding manifest file:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp-restricted
subjects:
- kind: Group
  name: system:authenticated
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: PodSecurityPolicy
  name: restricted
  apiGroup: policy

In this example, we're creating a RoleBinding called "psp-restricted" that associates the "restricted" Pod Security Policy with the group "system:authenticated".

To create this RoleBinding, save the manifest file to a file called rolebinding.yaml, and then run the following command:

kubectl apply -f rolebinding.yaml
  1. Verify the Pod Security Policy

You can verify that the Pod Security Policy has been created correctly by running the following command:

kubectl get psp

This command will show you a list of Pod Security Policies in the cluster, including the "restricted" policy that we just created.

  1. Create a Pod that Meets the Policy

Now that we have a Pod Security Policy in place, let's create a pod that meets the policy requirements. Here's an example of a Pod manifest file that meets the "restricted" policy requirements:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:latest
    securityContext:
      runAsUser: 1000
      runAsNonRoot: true
    volumeMounts:
    - name: nginx-config
      mountPath: /etc/nginx/nginx.conf
      readOnly: true
  volumes:
  - name: nginx-config
    configMap:
      name: nginx-config

In this example, we're creating a pod called "nginx" that runs as a non-root user with UID 1000,

Secrets

In Kubernetes, a secret is an object that contains sensitive information, such as passwords, API keys, or certificates. Secrets are stored in the Kubernetes API server and are used to pass sensitive information to a container running in a pod.

Secrets can be created and managed using the kubectl command-line tool or through manifest files.

Let's explore three types of secrets along with examples:

Generic Secrets:

  • Generic secrets are the most basic type in Kubernetes, allowing you to store key-value pairs.

  • Example: Let's say you have an application that requires a database password. You can create a generic secret named db-secret with the password as the value:

kubectl create secret generic mysecret --from-literal=username=Gopa123 --from-literal=password=MySecurePassword123
kubectl get secrets

Once the secret is created, you can use it in a pod by referencing it in the pod's manifest file. For example, here's a pod manifest file that uses the my-secret secret:

# if we want to access specific variable from secret then we can use.
apiVersion: v1
kind: Pod
metadata:
  name: secret-demo-1
spec:
  containers:
  - name: demo-container
    image: nginx
    env:
    - name: Username
      valueFrom:   
        secretKeyRef:   
          name: db-secret
          key: username

or
# if we want to access all variable from the secret then we can use.
apiVersion: v1
kind: Pod
metadata:
  name: secret-demo-1
spec:
  containers:
  - name: demo-container
    image: nginx
    envFrom:
    - secretRef:
       name: docker-secret

kubectl apply -f my-secret-pod.yml
kubectl exec secret-demo-1 -it -- printenv

Docker-Registry Secrets:

  • Docker-registry secrets are used when pulling images from private Docker registries that require authentication.

  • Example: If your application uses a private Docker registry, you can create a Docker-registry secret named docker-secret with the registry credentials:

kubectl create secret docker-registry docker-secret --docker-email=example@gmail.com --docker-username=dev --docker-password=pass1234 --docker-server=my-registry.example:5000

apiVersion: v1
kind: Pod
metadata:
  name: secret-demo-2
spec:
  containers:
  - name: demo-container
    image: nginx
    envFrom:
    - secretRef:
       name: docker-secret

kubectl apply -f <file-name.yaml>
kubectl exec secret-demo-2 -it -- printenv

TLS Secrets:

  • TLS secrets are used to secure communication over HTTPS or other TLS-enabled protocols.

  • Example: Suppose you want to secure communication for your application with an SSL/TLS certificate. You can create a TLS secret named tls-secret by providing the certificate and key files:

kubectl create secret tls my-tls-secret --cert=/root/data/serverca.crt --key=/root/data/servercakey.pem

apiVersion: v1
kind: Pod
metadata:
  name: secret-demo-3
spec:
  containers:
  - name: demo-container
    image: nginx
    volumeMounts:
      - name: data
        mountPath: /etc/cert-data
  volumes:
  - name: data
    secret:
      secretName: my-tls-secret

kubectl apply -f <file-name.yaml>
kubectl exec -it <pod-name> -- bash
cd /etc/cert-data
ls

Network policies

Here's an example manifest file to create two pods, one with the label app: web and another without the label:

kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80
kubectl get svc,pod
kubectl run busybox --rm -ti --image=busybox:1.28 -- /bin/sh
wget --spider --timeout=1 nginx

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: access-nginx
spec:
  podSelector:
    matchLabels:
      app: nginx
  ingress:
  - from:
    - podSelector:
        matchLabels:
          access: "true"

kubectl apply -f network-policy.yaml
kubectl run busybox --rm -ti --image=busybox:1.28 -- /bin/sh
wget --spider --timeout=1 nginx

This creates a new Pod running the BusyBox image. Once inside the shell of the Pod, run the wget command to access the nginx service. Since the Pod does not have the required access: true label, the output should indicate a download timeout.

kubectl run busybox --rm -it --labels="access=true" --image=busybox:1.28 -- /bin/sh
wget --spider --timeout=1 nginx

This creates a new Pod running the BusyBox image with the required access: true label, and tests the connection to the nginx service again. The output should confirm a successful connection to the service.

TLS(Transport Layer Security)

Transport Layer Security (TLS) is a protocol used to secure communication over a network. It is commonly used to secure web traffic, email, and other types of network communication. In Kubernetes, TLS can be used to secure communication between clients and servers, as well as between different components of a cluster.

Here's an example of how to use TLS in Kubernetes:

  1. Create a TLS Certificate

The first step in using TLS in Kubernetes is to create a TLS certificate. You can use the openssl command to create a self-signed certificate for testing purposes. For example, to create a certificate with a key length of 2048 bits, you can run the following command:

kubectl create secret tls tls-secret --key tls.key --cert tls.crt
  1. Use the Kubernetes Secret

Once you have created the Kubernetes secret, you can use it in your Kubernetes manifests to enable TLS. For example, here's a sample deployment manifest that uses the tls-secret to secure communication between the container and the client:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: tls-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tls
  template:
    metadata:
      labels:
        app: tls
    spec:
      containers:
        - name: tls-container
          image: nginx
          ports:
            - containerPort: 443
          volumeMounts:
            - name: tls-secret
              mountPath: /etc/tls
              readOnly: true
      volumes:
        - name: tls-secret
          secret:
            secretName: tls-secret

In this example, we're creating a deployment with one replica and a container named tls-container. We're using the nginx image as an example. We're also creating a volume named tls-secret and mounting it to the container at the path /etc/tls. This will make the certificate and key available to the container.

  1. Test the TLS Connection

To test the TLS connection, you can use the curl command to send a request to the container. For example, to send a request to the container at the IP address 10.0.0.1 over HTTPS, you can run the following command:

curl --cacert tls.crt https://10.0.0.1

In this example, we're using the --cacert option to specify the certificate authority file, which is the same as the certificate file in our case since it's self-signed. The https:// prefix is used to indicate that we want to use HTTPS, which is the protocol that uses TLS to secure the communication.

Overall, using TLS in Kubernetes is an important security measure to protect communication between different components of a cluster, and between clients and servers in the cluster.

#KubeWeekChallenge Day5#kubeweekTrainWithShubham#kubernetes #container