Tutorial - Clean up MicroK8s Cluster and Redeploy with Helm

In this tutorial, you will learn how to clean up the MicroK8s cluster removing all the containers, deployments, configmaps, secrets, etc from the previous tutorials. MariaDB and EJBCA are deployed using a Helm chart on the cleaned-up cluster. The database is backed up and restored that way all the configuration that has been done from the previous tutorials is not lost.

Over the last year, all the EJBCA tutorials have been demonstrated on a MicroK8s virtual machine. EJBCA was deployed with YAML files and it would be nice to redeploy EJBCA using Helm. It’s time to clean up and clear out the cluster starting with a fresh deployment of EJBCA using a backup of the MariaDB database.

If you want to clean house, start fresh, and use Helm then this tutorial is for you.

The tutorial covers these steps:

Review what is deployed in the MicroK8s cluster
Backup the MariaDB database
Reset the MicroK8s cluster
Install required services on the MicroK8s cluster
Deploy MariaDB using Helm and restore the MariaDB backup
Deploy EJBCA using Helm

Prerequisites

For this tutorial, EJBCA Community Docker container version 8.2.0.1 was used.

Before you begin, you need:

A virtual machine running MicroK8s.
To have completed the tutorial deploy EJBCA and MariaDB containers in MicroK8s.

Step 1 - Review what is deployed in the cluster

If you have been trying out the previous tutorials, there are a lot of pieces and parts deployed and configured in MicroK8s from following the tutorials. Follow the steps below to review what is currently deployed in the cluster:

Review all of the deployments and items deployed:

$ kubectl get all,ingress,secret,pv,pvc,svc,no --all-namespaces
- The output is similar to the following:
  
  NAMESPACE NAME READY STATUS RESTARTS AGE
  
  cert-manager pod/cert-manager-55cf8685cb-tztzk 1/1 Running 15 (2d21h ago) 151d
  
  cert-manager pod/cert-manager-cainjector-fbd548cb8-nrr7l 1/1 Running 20 (2d21h ago) 151d
  
  cert-manager pod/cert-manager-webhook-655b4d58fb-46zzh 1/1 Running 12 (2d21h ago) 151d
  
  ejbca-cert-manager pod/ejbca-cert-manager-issuer-6c5549d575-qd52x 1/1 Running 30 (2d21h ago) 137d
  
  ejbca-k8s pod/ejbca-deployment-78fdbbb5c4-n5v2p 1/1 Running 12 (2d21h ago) 167d
  
  ejbca-k8s pod/ejbca-httpd-7d646c85f9-2n68j 1/1 Running 3 (2d21h ago) 48d
  
  ejbca-k8s pod/mariadb-deployment-5665d96759-sdg24 1/1 Running 3 (2d21h ago) 48d
  
  ingress pod/nginx-ingress-microk8s-controller-sqk8c 1/1 Running 2 (2d21h ago) 24d
  
  istio-system pod/istio-egressgateway-66fc798fb8-6cmc4 0/1 Running 22 (2d21h ago) 442d
  
  istio-system pod/istio-ingressgateway-668c8c965b-s7725 0/1 Running 22 (2d21h ago) 442d
  
  istio-system pod/istiod-7fc54f9bd-qhwxg 1/1 Running 27 (2d21h ago) 442d
  
  kube-system pod/calico-kube-controllers-7c8c9d85c9-htp24 1/1 Running 3 (2d21h ago) 48d
  
  kube-system pod/calico-node-c4ffk 1/1 Running 0 2d21h
  
  kube-system pod/coredns-6f5f9b5d74-wxbmz 1/1 Running 28 (2d21h ago) 449d
  
  kube-system pod/hostpath-provisioner-69cd9ff5b8-fx8ps 1/1 Running 45 (2d21h ago) 449d
  
  metallb-system pod/controller-9556c586f-gzbm6 1/1 Running 28 (2d21h ago) 449d
  
  metallb-system pod/speaker-58lm5 1/1 Running 39 (2d21h ago) 449d
  
  pkirules pod/ejbca-cm-issuer-helloworld-one-6c886556f9-xth78 1/1 Terminating 8 (2d21h ago) 136d
  
  NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  
  cert-manager service/cert-manager ClusterIP 10.152.183.223 <none> 9402/TCP 151d
  
  cert-manager service/cert-manager-webhook ClusterIP 10.152.183.154 <none> 443/TCP 151d
  
  default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 449d
  
  ejbca-k8s service/ejbca-database-service ClusterIP 10.152.183.44 <none> 3306/TCP 449d
  
  ejbca-k8s service/ejbca-internal ClusterIP 10.152.183.133 <none> 443/TCP,80/TCP 449d
  
  ejbca-k8s service/ejbca-service ClusterIP 10.152.183.46 <none> 8009/TCP 449d
  
  ingress service/ejbca-node1 LoadBalancer 10.152.183.171 172.16.170.240 80:31205/TCP,443:30622/TCP 449d
  
  istio-system service/istio-egressgateway ClusterIP 10.152.183.125 <none> 80/TCP,443/TCP 442d
  
  istio-system service/istio-ingressgateway LoadBalancer 10.152.183.92 172.16.170.241 15021:30350/TCP,80:31533/TCP,443:30430/TCP,31400:32547/TCP,15443:31322/TCP 442d
  
  istio-system service/istiod ClusterIP 10.152.183.76 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 442d
  
  kube-system service/kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 449d
  
  metallb-system service/webhook-service ClusterIP 10.152.183.52 <none> 443/TCP 449d
  
  NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
  
  ingress daemonset.apps/nginx-ingress-microk8s-controller 1 1 1 1 1 <none> 449d
  
  kube-system daemonset.apps/calico-node 1 1 1 1 1 kubernetes.io/os=linux 449d
  
  metallb-system daemonset.apps/speaker 1 1 1 1 1 kubernetes.io/os=linux 449d
  
  NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
  
  cert-manager deployment.apps/cert-manager 1/1 1 1 151d
  
  cert-manager deployment.apps/cert-manager-cainjector 1/1 1 1 151d
  
  cert-manager deployment.apps/cert-manager-webhook 1/1 1 1 151d
  
  ejbca-cert-manager deployment.apps/ejbca-cert-manager-issuer 1/1 1 1 137d
  
  ejbca-k8s deployment.apps/ejbca-deployment 1/1 1 1 167d
  
  ejbca-k8s deployment.apps/ejbca-httpd 1/1 1 1 449d
  
  ejbca-k8s deployment.apps/mariadb-deployment 1/1 1 1 449d
  
  istio-system deployment.apps/istio-egressgateway 0/1 1 0 442d
  
  istio-system deployment.apps/istio-ingressgateway 0/1 1 0 442d
  
  istio-system deployment.apps/istiod 1/1 1 1 442d
  
  kube-system deployment.apps/calico-kube-controllers 1/1 1 1 449d
  
  kube-system deployment.apps/coredns 1/1 1 1 449d
  
  kube-system deployment.apps/hostpath-provisioner 1/1 1 1 449d
  
  metallb-system deployment.apps/controller 1/1 1 1 449d
  
  NAMESPACE NAME DESIRED CURRENT READY AGE
  
  cert-manager replicaset.apps/cert-manager-55cf8685cb 1 1 1 151d
  
  cert-manager replicaset.apps/cert-manager-cainjector-fbd548cb8 1 1 1 151d
  
  cert-manager replicaset.apps/cert-manager-webhook-655b4d58fb 1 1 1 151d
  
  ejbca-cert-manager replicaset.apps/ejbca-cert-manager-issuer-6c5549d575 1 1 1 137d
  
  ejbca-k8s replicaset.apps/ejbca-deployment-78fdbbb5c4 1 1 1 167d
  
  ejbca-k8s replicaset.apps/ejbca-httpd-7d646c85f9 1 1 1 449d
  
  ejbca-k8s replicaset.apps/mariadb-deployment-5665d96759 1 1 1 449d
  
  istio-system replicaset.apps/istio-egressgateway-66fc798fb8 1 1 0 442d
  
  istio-system replicaset.apps/istio-egressgateway-7bf6457867 0 0 0 442d
  
  istio-system replicaset.apps/istio-ingressgateway-668c8c965b 1 1 0 442d
  
  istio-system replicaset.apps/istio-ingressgateway-8576d6c55f 0 0 0 442d
  
  istio-system replicaset.apps/istiod-77f9c8bf68 0 0 0 442d
  
  istio-system replicaset.apps/istiod-7fc54f9bd 1 1 1 442d
  
  kube-system replicaset.apps/calico-kube-controllers-5cdbddfc9 0 0 0 449d
  
  kube-system replicaset.apps/calico-kube-controllers-75b6ddb7b8 0 0 0 408d
  
  kube-system replicaset.apps/calico-kube-controllers-79568db7f8 0 0 0 449d
  
  kube-system replicaset.apps/calico-kube-controllers-7c8c9d85c9 1 1 1 48d
  
  kube-system replicaset.apps/calico-kube-controllers-7c9cdf4bfc 0 0 0 396d
  
  kube-system replicaset.apps/calico-kube-controllers-7fcc7d8496 0 0 0 167d
  
  kube-system replicaset.apps/calico-kube-controllers-b5fcf779f 0 0 0 60d
  
  kube-system replicaset.apps/calico-kube-controllers-df49bb54 0 0 0 364d
  
  kube-system replicaset.apps/coredns-6f5f9b5d74 1 1 1 449d
  
  kube-system replicaset.apps/hostpath-provisioner-69cd9ff5b8 1 1 1 449d
  
  metallb-system replicaset.apps/controller-9556c586f 1 1 1 449d
  
  NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
  
  ejbca-k8s ingress.networking.k8s.io/ejbca-node1 public ejbca-node1.ejbca-k8s 127.0.0.1 80, 443 24d
  
  NAMESPACE NAME TYPE DATA AGE
  
  cert-manager secret/cert-manager-webhook-ca Opaque 3 151d
  
  cert-manager secret/sh.helm.release.v1.cert-manager.v1 helm.sh/release.v1 1 151d
  
  ejbca-cert-manager secret/ejbca-ca-secret Opaque 1 151d
  
  ejbca-cert-manager secret/ejbca-secret kubernetes.io/tls 2 151d
  
  ejbca-cert-manager secret/sh.helm.release.v1.ejbca-cert-manager-issuer.v1 helm.sh/release.v1 1 137d
  
  ejbca-csr-signer secret/ejbca-client-cert kubernetes.io/tls 2 442d
  
  ejbca-csr-signer secret/ejbca-credentials Opaque 1 442d
  
  ejbca-k8s secret/ejbca-mariadb-secret Opaque 1 449d
  
  ejbca-k8s secret/mariadb-secret Opaque 1 449d
  
  ejbca-k8s secret/mtls-trust-chain Opaque 1 47d
  
  ejbca-k8s secret/tls-ejbca-node1 kubernetes.io/tls 2 60d
  
  istio-system secret/external-ca-cert Opaque 1 442d
  
  istio-system secret/istio-ca-secret istio.io/ca-root 5 442d
  
  metallb-system secret/memberlist Opaque 1 449d
  
  metallb-system secret/webhook-server-cert Opaque 4 449d
  
  vault secret/tls-api-vault kubernetes.io/tls 2 160d
  
  vault secret/vault-ha-tls Opaque 3 160d
  
  NAMESPACE NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE
  
  persistentvolume/pvc-031c7c9d-63dd-42b7-8c20-83e9e57782d7 100M RWO Delete Bound ejbca-k8s/mariadb-pvc microk8s-hostpath <unset> 449d
  
  persistentvolume/pvc-230714e3-ed56-498f-8386-c84b941e9b00 1Gi RWO Delete Bound vault/data-vault-1 microk8s-hostpath <unset> 160d
  
  persistentvolume/pvc-3d0cc22e-3c9a-4e8a-a867-9f99b9f97424 1Gi RWO Delete Bound vault/audit-vault-2 microk8s-hostpath <unset> 160d
  
  persistentvolume/pvc-4cacc08e-2802-4b54-9af5-b386750fa715 1Gi RWO Delete Bound vault/audit-vault-0 microk8s-hostpath <unset> 160d
  
  persistentvolume/pvc-5be3658f-abda-4512-873a-38b4e9e46556 1Gi RWO Delete Bound vault/audit-vault-1 microk8s-hostpath <unset> 160d
  
  persistentvolume/pvc-6d665f7a-c89e-4536-8b3a-f8f5a79a4b00 1Gi RWO Delete Bound vault/data-vault-0 microk8s-hostpath <unset> 160d
  
  persistentvolume/pvc-903289f8-db05-44c2-8ded-fdb93a34fb21 1Gi RWO Delete Bound vault/data-vault-2 microk8s-hostpath <unset> 160d
  
  persistentvolume/pvc-b1d34ec9-10db-4b8d-ab3d-d8ba7810be14 30M RWO Delete Bound ejbca-k8s/ejbca-pvc microk8s-hostpath <unset> 449d
  
  NAMESPACE NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
  
  ejbca-k8s persistentvolumeclaim/ejbca-pvc Bound pvc-b1d34ec9-10db-4b8d-ab3d-d8ba7810be14 30M RWO microk8s-hostpath <unset> 449d
  
  ejbca-k8s persistentvolumeclaim/mariadb-pvc Bound pvc-031c7c9d-63dd-42b7-8c20-83e9e57782d7 100M RWO microk8s-hostpath <unset> 449d
  
  vault persistentvolumeclaim/audit-vault-0 Bound pvc-4cacc08e-2802-4b54-9af5-b386750fa715 1Gi RWO microk8s-hostpath <unset> 160d
  
  vault persistentvolumeclaim/audit-vault-1 Bound pvc-5be3658f-abda-4512-873a-38b4e9e46556 1Gi RWO microk8s-hostpath <unset> 160d
  
  vault persistentvolumeclaim/audit-vault-2 Bound pvc-3d0cc22e-3c9a-4e8a-a867-9f99b9f97424 1Gi RWO microk8s-hostpath <unset> 160d
  
  vault persistentvolumeclaim/data-vault-0 Bound pvc-6d665f7a-c89e-4536-8b3a-f8f5a79a4b00 1Gi RWO microk8s-hostpath <unset> 160d
  
  vault persistentvolumeclaim/data-vault-1 Bound pvc-230714e3-ed56-498f-8386-c84b941e9b00 1Gi RWO microk8s-hostpath <unset> 160d
  
  vault persistentvolumeclaim/data-vault-2 Bound pvc-903289f8-db05-44c2-8ded-fdb93a34fb21 1Gi RWO microk8s-hostpath <unset> 160d
  
  NAMESPACE NAME STATUS ROLES AGE VERSION
  
  node/microk8-01 Ready <none> 449d v1.29.4 microk8s-hostpath <unset> 159d

You have now reviewed what has been deployed in MicroK8s, continue to the next step to backup the EJBCA database.

Step 2 - Backup the EJBCA database

Time was invested to configure EJBCA following the tutorials and it would be great to not lose the EJBCA configuration. Follow these steps to backup the EJBCA database before the MicroK8s cluster is reset:

Review what directories are present in the VM host:

$ ls -l
- The output is similar to the following:
  
  total 20
  
  -rw-rw-r--. 1 user user 3612 Apr 24 07:56 ca.crt
  
  drwxrwxr-x. 3 user user 4096 Jan 26 06:30 cert-manager
  
  drwxrwxr-x. 3 user user 88 Mar 25 2023 csr-api
  
  drwxrwxr-x. 2 user user 4096 May 17 11:58 k81
  
  drwx------. 3 user microk8s 22 Mar 18 2023 snap
  
  drwxrwxr-x. 2 user user 4096 Jun 9 15:34 sql
  
  drwxrwxr-x. 2 user user 4096 Jan 2 12:01 vault
Install the MariaDB client using the DNF package manager:

$ sudo dnf install -y <https://dlm.mariadb.com/2690116/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-client-10.10.2-1.el8.x86_64.rpm> \

<https://dlm.mariadb.com/2690045/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-backup-10.10.2-1.el8.x86_64.rpm> \

<https://dlm.mariadb.com/2690100/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-common-10.10.2-1.el8.x86_64.rpm> \

<https://dlm.mariadb.com/2690079/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-shared-10.10.2-1.el8.x86_64.rpm>
Backup the database to the SQL directory:

$ mariadb-dump -h 10.152.183.44 -u root -p --all-databases --quick --lock-tables=false --system=users --insert-ignore --force > sql/20240610_BackupToMigrate.sql
- The output is similar to the following:
  
  # mariadb-dump: Couldn't execute 'SELECT CONCAT(QUOTE(u.user), '@', QUOTE(u.Host)) AS u FROM mysql.user u /*!80001 LEFT JOIN mysql.role_edges e ON u.user=e.from_user AND u.host=e.from_host WHERE e.from_user IS NULL */ /*M!100005 WHERE is_role='N' */': Illegal mix of collations (utf8mb4_general_ci,COERCIBLE) and (utf8mb4_uca1400_ai_ci,COERCIBLE) for operation '=' (1267)

You have now backed up the EJBCA MariaDB database using the mariadb-dump utility. Continue to the next step to reset the MicroK8s cluster.

Step 3 - Reset the MicroK8s cluster

When MicroK8s is reset, the cluster is cleaned up by removing the entire configuration and restoring the cluster to the default install from when MicroK8s was first installed. To reset the cluster follow these steps:

Uninstall the ejbca cert-manager Issuer Helm deployment:

$ helm uninstall -n ejbca-cert-manager ejbca-cert-manager-issuer
- The output is similar to the following:
  
  release "ejbca-cert-manager-issuer" uninstalled
Uninstall the cert-manager Helm deployment:

$ helm uninstall -n cert-manager cert-manager
- The output is similar to the following:
  
  release "cert-manager" uninstalled
Uninstall istio using istioctl:

$ csr-api/ejbca-csr-signer/istio-1.16.3/bin/istioctl uninstall --purge --force
- The output is similar to the following:
  
  failed to get proxy infos: failure running port forward process: failure running port forward process: error upgrading connection: error dialing backend: dial tcp 172.16.170.187:10250: connect: connection refused
  
  All Istio resources will be pruned from the cluster
  
  Proceed? (y/N)
Enter y to continue.
- The output is similar to the following:
  
  Removed IstioOperator:istio-system:installed-state.
  
  Removed Deployment:istio-system:istio-egressgateway.
  
  Removed Deployment:istio-system:istio-ingressgateway.
  
  Removed Deployment:istio-system:istiod.
  
  Removed Service:istio-system:istio-egressgateway.
  
  Removed Service:istio-system:istio-ingressgateway.
  
  Removed Service:istio-system:istiod.
  
  Removed ConfigMap:istio-system:istio.
  
  Removed ConfigMap:istio-system:istio-sidecar-injector.
  
  Removed Pod:istio-system:istio-egressgateway-66fc798fb8-6cmc4.
  
  Removed Pod:istio-system:istio-ingressgateway-668c8c965b-s7725.
  
  Removed Pod:istio-system:istiod-7fc54f9bd-qhwxg.
  
  Removed ServiceAccount:istio-system:istio-egressgateway-service-account.
  
  Removed ServiceAccount:istio-system:istio-ingressgateway-service-account.
  
  Removed ServiceAccount:istio-system:istio-reader-service-account.
  
  Removed ServiceAccount:istio-system:istiod.
  
  Removed ServiceAccount:istio-system:istiod-service-account.
  
  Removed RoleBinding:istio-system:istio-egressgateway-sds.
  
  Removed RoleBinding:istio-system:istio-ingressgateway-sds.
  
  Removed RoleBinding:istio-system:istiod.
  
  Removed RoleBinding:istio-system:istiod-istio-system.
  
  Removed Role:istio-system:istio-egressgateway-sds.
  
  Removed Role:istio-system:istio-ingressgateway-sds.
  
  Removed Role:istio-system:istiod.
  
  Removed Role:istio-system:istiod-istio-system.
  
  Removed EnvoyFilter:istio-system:stats-filter-1.13.
  
  Removed EnvoyFilter:istio-system:stats-filter-1.14.
  
  Removed EnvoyFilter:istio-system:stats-filter-1.15.
  
  Removed EnvoyFilter:istio-system:stats-filter-1.16.
  
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.13.
  
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.14.
  
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.15.
  
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.16.
  
  Removed PodDisruptionBudget:istio-system:istio-egressgateway.
  
  Removed PodDisruptionBudget:istio-system:istio-ingressgateway.
  
  Removed PodDisruptionBudget:istio-system:istiod.
  
  Removed MutatingWebhookConfiguration::istio-revision-tag-default.
  
  Removed MutatingWebhookConfiguration::istio-sidecar-injector.
  
  Removed ValidatingWebhookConfiguration::istio-validator-istio-system.
  
  Removed ValidatingWebhookConfiguration::istiod-default-validator.
  
  Removed ClusterRole::istio-reader-clusterrole-istio-system.
  
  Removed ClusterRole::istio-reader-istio-system.
  
  Removed ClusterRole::istiod-clusterrole-istio-system.
  
  Removed ClusterRole::istiod-gateway-controller-istio-system.
  
  Removed ClusterRole::istiod-istio-system.
  
  Removed ClusterRoleBinding::istio-reader-clusterrole-istio-system.
  
  Removed ClusterRoleBinding::istio-reader-istio-system.
  
  Removed ClusterRoleBinding::istiod-clusterrole-istio-system.
  
  Removed ClusterRoleBinding::istiod-gateway-controller-istio-system.
  
  Removed ClusterRoleBinding::istiod-istio-system.
  
  Removed CustomResourceDefinition::authorizationpolicies.security.istio.io.
  
  Removed CustomResourceDefinition::destinationrules.networking.istio.io.
  
  Removed CustomResourceDefinition::envoyfilters.networking.istio.io.
  
  Removed CustomResourceDefinition::gateways.networking.istio.io.
  
  Removed CustomResourceDefinition::istiooperators.install.istio.io.
  
  Removed CustomResourceDefinition::peerauthentications.security.istio.io.
  
  Removed CustomResourceDefinition::proxyconfigs.networking.istio.io.
  
  Removed CustomResourceDefinition::requestauthentications.security.istio.io.
  
  Removed CustomResourceDefinition::serviceentries.networking.istio.io.
  
  Removed CustomResourceDefinition::sidecars.networking.istio.io.
  
  Removed CustomResourceDefinition::telemetries.telemetry.istio.io.
Reset microk8s to restore MicroK8s to the default install:

$ sudo /var/lib/snapd/snap/bin/microk8s reset
- The output is similar to the following:
  
  Disabling all addons
  
  Disabling addon : core/cert-manager
  
  Disabling addon : core/dashboard
  
  Disabling addon : core/dns
  
  Disabling addon : core/gpu
  
  Disabling addon : core/helm
  
  Disabling addon : core/helm3
  
  Disabling addon : core/host-access
  
  Disabling addon : core/hostpath-storage
  
  Disabling addon : core/ingress
  
  Disabling addon : core/kube-ovn
  
  Disabling addon : core/mayastor
  
  Disabling addon : core/metallb
  
  Disabling addon : core/metrics-server
  
  Disabling addon : core/minio
  
  Disabling addon : core/observability
  
  Disabling addon : core/prometheus
  
  Disabling addon : core/rbac
  
  Disabling addon : core/registry
  
  Disabling addon : core/storage
  
  All addons are disabled.
  
  Deleting the CNI
  
  Cleaning resources in namespace bookinfo
  
  Cleaning resources in namespace cert-manager
  
  Cleaning resources in namespace default
  
  Cleaning resources in namespace ejbca-cert-manager
  
  Cleaning resources in namespace ejbca-csr-signer
  
  Cleaning resources in namespace ejbca-k8s
  
  Cleaning resources in namespace ingress
  
  Cleaning resources in namespace istio-system
  
  Cleaning resources in namespace kube-node-lease
  
  Cleaning resources in namespace kube-public
  
  Cleaning resources in namespace kube-system
  
  Cleaning resources in namespace metallb-system
  
  Cleaning resources in namespace pkirules
  
  Cleaning resources in namespace vault
  
  Removing CRDs
  
  Removing PriorityClasses
  
  Removing StorageClasses
  
  Removing namespace/bookinfo
  
  Removing namespace/cert-manager
  
  Removing namespace/ejbca-cert-manager
  
  Removing namespace/ejbca-csr-signer
  
  Removing namespace/ejbca-k8s
  
  Removing namespace/ingress
  
  Removing namespace/istio-system
  
  Removing namespace/metallb-system
  
  Removing namespace/pkirules
  
  Removing namespace/vault
  
  Restarting cluster
  
  Stopped.
  
  Setting up the CNI

You have now reset MicroK8s back to the default. Continue to the next step to enable the required services in MicroK8s for deploying MariaDB and EJBCA.

Step 4 - Enable required services on MicroK8s

Enable DNS, Helm, and hostpath storage on the MicroK8s cluster which are used to deploy EJBCA and MariaDB and provide persistent storage for MariaDB. Follow the steps below to prepare the cluster:

Enable the DNS, Helm, and hostpath-storage addons on the cluster:

$ microk8s enable dns hostpath-storage helm
- The output is similar to the following:
  
  Infer repository core for addon dns
  
  Infer repository core for addon hostpath-storage
  
  Infer repository core for addon helm
  
  WARNING: Do not enable or disable multiple addons in one command.
  
  This form of chained operations on addons will be DEPRECATED in the future.
  
  Please, enable one addon at a time: 'microk8s enable <addon>'
  
  Addon core/dns is already enabled
  
  Enabling default storage class.
  
  WARNING: Hostpath storage is not suitable for production environments.
  
  A hostpath volume can grow beyond the size limit set in the volume claim manifest.
  
  deployment.apps/hostpath-provisioner created
  
  storageclass.storage.k8s.io/microk8s-hostpath created
  
  serviceaccount/microk8s-hostpath created
  
  clusterrole.rbac.authorization.k8s.io/microk8s-hostpath created
  
  clusterrolebinding.rbac.authorization.k8s.io/microk8s-hostpath created
  
  Storage will be available soon.
  
  Addon core/helm is already enabled
Enable the Metal LB addon on the cluster using the IP range of the VM network the VM is on:

$ microk8s enable metallb:172.16.170.240-172.16.170.245
- The output is similar to the following:
  
  Infer repository core for addon metallb
  
  Enabling MetalLB
  
  Applying Metallb manifest
  
  customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created
  
  customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
  
  customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
  
  customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
  
  customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
  
  customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
  
  customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
  
  namespace/metallb-system created
  
  serviceaccount/controller created
  
  serviceaccount/speaker created
  
  clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
  
  clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
  
  role.rbac.authorization.k8s.io/controller created
  
  role.rbac.authorization.k8s.io/pod-lister created
  
  clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
  
  clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
  
  rolebinding.rbac.authorization.k8s.io/controller created
  
  secret/webhook-server-cert created
  
  service/webhook-service created
  
  rolebinding.rbac.authorization.k8s.io/pod-lister created
  
  daemonset.apps/speaker created
  
  deployment.apps/controller created
  
  validatingwebhookconfiguration.admissionregistration.k8s.io/validating-webhook-configuration created
  
  Waiting for Metallb controller to be ready.
  
  deployment.apps/controller condition met
  
  ipaddresspool.metallb.io/default-addresspool created
  
  l2advertisement.metallb.io/default-advertise-all-pools created
  
  MetalLB is enabled
Create an alias for helm command:

$ sudo snap alias microk8s.helm helm
- The output is similar to the following:
  
  #Added:
  
  # - microk8s.helm as helm
Review what is deployed in the cluster after the microk8s reset:

$ kubectl get all,secret,pv,pvc,svc,no --all-namespaces
- The output is similar to the following:
  
  NAMESPACE NAME READY STATUS RESTARTS AGE
  
  kube-system pod/calico-kube-controllers-7cb4fd5784-xs4db 1/1 Running 0 3m39s
  
  kube-system pod/calico-node-s42fj 1/1 Running 0 3m39s
  
  kube-system pod/coredns-85db789d58-2pfxz 1/1 Running 0 2m3s
  
  kube-system pod/hostpath-provisioner-7df77bc496-xcdbm 1/1 Running 0 50s
  
  metallb-system pod/controller-5f7bb57799-wxcvx 1/1 Running 0 50s
  
  metallb-system pod/speaker-rdn59 1/1 Running 0 50s
  
  NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  
  default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 6m31s
  
  kube-system service/kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 2m3s
  
  metallb-system service/webhook-service ClusterIP 10.152.183.225 <none> 443/TCP 102s
  
  NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
  
  kube-system daemonset.apps/calico-node 1 1 1 1 1 kubernetes.io/os=linux 4m44s
  
  metallb-system daemonset.apps/speaker 1 1 1 1 1 kubernetes.io/os=linux 102s
  
  NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
  
  kube-system deployment.apps/calico-kube-controllers 1/1 1 1 4m44s
  
  kube-system deployment.apps/coredns 1/1 1 1 2m3s
  
  kube-system deployment.apps/hostpath-provisioner 1/1 1 1 114s
  
  metallb-system deployment.apps/controller 1/1 1 1 102s
  
  NAMESPACE NAME DESIRED CURRENT READY AGE
  
  kube-system replicaset.apps/calico-kube-controllers-7cb4fd5784 1 1 1 3m39s
  
  kube-system replicaset.apps/coredns-85db789d58 1 1 1 2m3s
  
  kube-system replicaset.apps/hostpath-provisioner-7df77bc496 1 1 1 50s
  
  metallb-system replicaset.apps/controller-5f7bb57799 1 1 1 50s
  
  NAMESPACE NAME TYPE DATA AGE
  
  metallb-system secret/memberlist Opaque 1 49s
  
  metallb-system secret/webhook-server-cert Opaque 4 102s
  
  NAMESPACE NAME STATUS ROLES AGE VERSION
  
  node/microk8-01 Ready <none> 449d v1.29.4

You have now enabled the required services to deploy EJBCA and MariaDB using Helm. Continue to the next step to deploy MariaDB using Helm.

Step 5 - Deploy MariaDB with Helm & restore the EJBCA SQL backup

Before EJBCA is deployed you need to have the database (MariaDB) deployed and the SQL backup restored. That way when EJBCA is deployed you begin right where you left off from the last backup taken. Follow these steps to deploy MariaDB with Helm and restore the EJBCA database backup:

Install Helm repository for MariaDB:

$ helm repo add groundhog2k https://groundhog2k.github.io/helm-charts
- The output is similar to the following:
  
  "groundhog2k" has been added to your repositories
Create namespace for the EJBCA MariaDB deployment:

$ kubectl create namespace ejbca-k8s
- The output is similar to the following:
  
  namespace/ejbca-k8s created
Create the MariaDB root password secret:

$ kubectl -n ejbca-k8s create secret generic mariadb-secret --from-literal=MARIADB_ROOT_PASSWORD=ejbca
- The output is similar to the following:
  
  secret/mariadb-secret created
Change to the k81 directory:

$ cd k81
Create the mariadb-override.yaml file with settings to deploy MariaDB:

$ cat > mariadb-override.yaml << EOF

## Default values for MariaDB deployment

## MariaDB docker image

image:

registry: "docker.io"

repository: "mariadb"

pullPolicy: IfNotPresent

# Overrides the image tag whose default is the chart appVersion.

tag: "latest"

## Optional service account

serviceAccount:

# Specifies whether a service account should be created

create: true

# Annotations to add to the service account

annotations: {}

# The name of the service account to use.

# If not set and create is true, a name is generated using the fullname template

name: ""

# Resource limits and requests

resources:

limits:

cpu: 2

memory: 2048Mi

requests:

cpu: 1000m

memory: 1024Mi

## Use Kubernetes Deployment instead of StatefulSet

useDeployment: false

## Database configuration

settings:

## Arguments for the container entrypoint process

arguments:

- --character-set-server=utf8mb4

- --collation-server=utf8mb4_unicode_ci

## Optional existing secret for the MariaDB root password

existingSecret: mariadb-secret

## Set true to skip loading timezone data during init

skipTZInfo: false

## Storage parameters

storage:

## Alternative set requestedSize to define a size for a dynamically created PVC

requestedSize: 1Gi

EOF
Deploy MariaDB using helm chart with override:

$ helm install -n ejbca-k8s ejbca-mariadb groundhog2k/mariadb -f mariadb-override.yaml
- The output is similar to the following:
  
  NAME: ejbca-mariadb
  
  LAST DEPLOYED: Sun Jun 9 13:31:31 2024
  
  NAMESPACE: ejbca-k8s
  
  STATUS: deployed
  
  REVISION: 1
  
  TEST SUITE: None
Check on the deployment after waiting for 1 min:

$ kubectl -n ejbca-k8s get all
- The output is similar to the following:
  
  NAME READY STATUS RESTARTS AGE
  
  pod/ejbca-mariadb-0 1/1 Running 0 85s
  
  NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  
  service/ejbca-mariadb ClusterIP 10.152.183.95 <none> 3306/TCP 85s
  
  NAME READY AGE
  
  statefulset.apps/ejbca-mariadb 1/1 85s
Connect to the database using the service/ejbca-mariadb to make sure the DB is accessible, enter the password for the root account when prompted:

$ mariadb -h 10.152.183.234 -u root -p
Enter the password for the root account.
- The output is similar to the following:
  
  Enter password:
  
  Welcome to the MariaDB monitor. Commands end with ; or \g.
  
  Your MariaDB connection id is 51
  
  Server version: 11.4.2-MariaDB-ubu2404 mariadb.org binary distribution
  
  Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
  
  Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
  
  MariaDB [(none)]>
Enter exit to exit the database shell:

MariaDB [(none)]> exit
- The output is similar to the following:
  
  MariaDB [(none)]> exit
  
  Bye
Import the EJBCA SQL backup into MariaDB, enter the password for the root account when prompted:

$ mariadb -h 10.152.183.234 -u root -p < ../sql/20240610_BackupToMigrate.sql
Enter the password for the root account.
Connect to the ejbca database as the ejbca user:

$ mariadb -h 10.152.183.234 -u ejbca -p ejbca
Enter the password for the ejbca account.
- The output is similar to the following:
  
  Enter password:
  
  Reading table information for completion of table and column names
  
  You can turn off this feature to get a quicker startup with -A
  
  Welcome to the MariaDB monitor. Commands end with ; or \g.
  
  Your MariaDB connection id is 185
  
  Server version: 11.4.2-MariaDB-ubu2404 mariadb.org binary distribution
  
  Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
  
  Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
  
  MariaDB [ejbca]>
Verify the tables are present:

MariaDB [ejbca]> show tables;
- The output is similar to the following:
  
  MariaDB [ejbca]> show tables;
  
  # Output similar to the following:
  
  +-------------------------------+
  
  | Tables_in_ejbca |
  
  +-------------------------------+
  
  | AccessRulesData |
  
  | AcmeAccountData |
  
  | AcmeAuthorizationData |
  
  | AcmeChallengeData |
  
  | AcmeNonceData |
  
  | AcmeOrderData |
  
  | AdminEntityData |
  
  | AdminGroupData |
  
  | AdminPreferencesData |
  
  | ApprovalData |
  
  | AuditRecordData |
  
  | AuthorizationTreeUpdateData |
  
  | Base64CertData |
  
  | BlacklistData |
  
  | CAData |
  
  | CRLData |
  
  | CertReqHistoryData |
  
  | CertificateData |
  
  | CertificateProfileData |
  
  | CryptoTokenData |
  
  | EndEntityProfileData |
  
  | GlobalConfigurationData |
  
  | IncompleteIssuanceJournalData |
  
  | InternalKeyBindingData |
  
  | KeyRecoveryData |
  
  | NoConflictCertificateData |
  
  | OcspResponseData |
  
  | PeerData |
  
  | ProfileData |
  
  | PublisherData |
  
  | PublisherQueueData |
  
  | RoleData |
  
  | RoleMemberData |
  
  | SctData |
  
  | ServiceData |
  
  | UserData |
  
  | UserDataSourceData |
  
  +-------------------------------+
  
  37 rows in set (0.001 sec)
Enter exit to exit the database:

MariaDB [ejbca]> exit
- The output is similar to the following:
  
  MariaDB [ejbca]> exit
  
  Bye

You have now deployed MariaDB using Helm and restored the EJBCA database backup. You then verified the tables created from restoring the backup. Continue to the next step to deploy EJBCA using Helm.

Step 6 - Deploy EJBCA with Helm

EJBCA is deployed after MariaDB is deployed and the database backup is restored. Follow the steps below to deploy EJBCA:

Install the Helm repository for EJBCA:

$ helm repo add ejbca-ce https://keyfactor.github.io/ejbca-community-helm
- The output is similar to the following:
  
  "ejbca-ce" has been added to your repositories
Create the ejbca-override.yaml file:

$ cat > ejbca-overrides.yaml << EOF

ejbca:

useEphemeralH2Database: false

useH2Persistence: false

importExternalCas: false

importAppserverKeystore: false

importAppserverTruststore: false

importEjbcaConfFiles: false

env:

TLS_SETUP_ENABLED: "simple"

DATABASE_JDBC_URL: "jdbc:mariadb://ejbca-mariadb.ejbca-k8s:3306/ejbca?characterEncoding=utf8"

LOG_AUDIT_TO_DB: true

LOG_LEVEL_APP: "INFO"

LOG_LEVEL_SERVER: "INFO"

OBSERVABLE_BIND: "127.0.0.1"

METRICS_ENABLED: false

OCSP_CHECK_SIGN_CERT_VALIDITY: false

OCSP_NON_EXISTING_IS_GOOD: true

envRaw:

- name: DATABASE_USER

valueFrom:

secretKeyRef:

name: ejbcadbcredentials

key: database_user

- name: DATABASE_PASSWORD

valueFrom:

secretKeyRef:

name: ejbcadbcredentials

key: database_password

# - name: PASSWORD_ENCRYPTION_KEY

# valueFrom:

# secretKeyRef:

# name: ejbca-confidentiality-secrets

# key: password_encrpytion_key

envFrom: []

initContainers: []

sidecarContainers: []

volumes: []

volumeMounts: []

services:

directHttp:

enabled: false

proxyAJP:

enabled: false

proxyHttp:

enabled: true

type: LoadBalancer

bindIP: 0.0.0.0

httpPort: 80

httpsPort: 443

sidecarPorts: []

nginx:

enabled: true

host: "ejbca-node1.ejbca-k8s"

proxy_url_host: localhost

mountInternalNginxCert: true

secretInternalNginxCert: "internal-nginx-credential-secret-ca"

service:

enabled: false

type: NodePort

httpPort: 30080

httpsPort: 30443

ingress:

enabled: false

replicaCount: 1

updateStrategy: {}

image:

repository: keyfactor/ejbca-ce

pullPolicy: IfNotPresent

tag: latest

imagePullSecrets: []

nameOverride: "ejbca-ca"

fullnameOverride: "ejbca-ca"

serviceAccount:

create: true

annotations: {}

name: ""

podLabels: {}

podAnnotations: {}

podSecurityContext: {}

securityContext: {}

resources:

limits:

cpu: 4

memory: 4096Mi

requests:

cpu: 1000m

memory: 2048Mi

autoscaling:

enabled: false

minReplicas: 1

maxReplicas: 5

targetCPUUtilizationPercentage: 80

behavior: {}

podDisruptionBudget:

create: false

minAvailable: 1

maxUnavailable: ""

nodeSelector: {}

affinity: {}

tolerations: []

topologySpreadConstraints: []

priorityClassName: ""

schedulerName: ""

terminationGracePeriodSeconds: ""

EOF
Create the secret for the EJBCA Database username and password:

$ kubectl -n ejbca-k8s create secret generic ejbcadbcredentials --from-literal=database_user=ejbca --from-literal=database_password=ejbca
- The output is similar to the following:
  
  secret/ejbcadbcredentials created
Create the TLS secret for the certificate, key, and CA cert used by NGINX that is deployed in the EJBCA pod:

$ kubectl -n ejbca-k8s create secret generic internal-nginx-credential-secret-ca \

--from-file=ejbca-node1.ejbca-k8s.pem=ejbca-node1.ejbca-k8s.crt \

--from-file=ejbca-node1.ejbca-k8s-Key.pem=ejbca-node1.ejbca-k8s.key \

--from-file=ejbca-node1.ejbca-k8s-CA.pem=../ca.crt
- The output is similar to the following:
  
  secret/internal-nginx-credential-secret-ca created
Deploy EJBCA using the Helm chart and the override values:

$ helm install -n ejbca-k8s ejbca-ca ejbca-ce/ejbca-community-helm -f ejbca-overrides.yaml
- The output is similar to the following:
  
  NAME: ejbca-ca
  
  LAST DEPLOYED: Sun Jun 9 15:29:16 2024
  
  NAMESPACE: ejbca-k8s
  
  STATUS: deployed
  
  REVISION: 1
  
  TEST SUITE: None
  
  NOTES:
  
  Thank you for installing EJBCA CE!
  
  Now that you have deployed EJBCA, you should look over the docs on using
  
  EJBCA available here:
  
  <https://doc.primekey.com/>
  
  Your release is named ejbca-ca. To learn more about the release, try:
  
  $ helm status ejbca-ca
  
  $ helm get manifest ejbca-ca
Review what has been deployed:

kubectl -n ejbca-k8s get all
- The output is similar to the following:
  
  NAME READY STATUS RESTARTS AGE
  
  pod/ejbca-ca-84649f7f7b-ck5bn 1/2 Running 0 15s
  
  pod/ejbca-mariadb-0 1/1 Running 0 83m
  
  NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  
  service/ejbca-ca LoadBalancer 10.152.183.78 172.16.170.240 80:31313/TCP,443:30725/TCP 15s
  
  service/ejbca-mariadb ClusterIP 10.152.183.95 <none> 3306/TCP 83m
  
  NAME READY UP-TO-DATE AVAILABLE AGE
  
  deployment.apps/ejbca-ca 0/1 1 0 15s
  
  NAME DESIRED CURRENT READY AGE
  
  replicaset.apps/ejbca-ca-84649f7f7b 1 1 0 15s
  
  NAME READY AGE
  
  statefulset.apps/ejbca-mariadb 1/1 83m
Check the EJBCA container logs and verify the container has properly started:

$ kubectl -n ejbca-k8s logs pod/ejbca-ca-84649f7f7b-spwh7 -c ejbca-ca
- The output is similar to the following:
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) #######################################################################################################
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # Whenever you are ready for production: #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # Try out the commercially supported EJBCA Enterprise Cloud on AWS or Azure, fully featured with: #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - Additional enrollment APIs such as a complete REST API, EST, ACME and more. #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - Support for external Registration Authority and OCSP responders #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - Hardware Security Module support #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - Application updates #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - ...and more! #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # Feel free to contact us directly for a cloud, on-prem, SaaS, embedded or hybrid solution that fits #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # your specific needs. #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # <https://aws.amazon.com/marketplace/seller-profile?id=7edf9048-58e6-4086-9d98-b8e0c1d78fce> #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # <https://azuremarketplace.microsoft.com/en-us/marketplace/apps/primekey.ejbca_enterprise_cloud_2> #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # <https://www.keyfactor.com> #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # sales@keyfactor.com #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # #
  
  2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) #######################################################################################################
  
  2024-06-09 19:29:50,015+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) Health check now reports application status at /ejbca/publicweb/healthcheck/ejbcahealth
Open a web browser and access EJBCA Administration.

You have now deployed EJBCA using Helm and logged into the AdminWeb to verify that all your configuration is present from the database restore. At this point, you can continue using EJBCA and try out some of the the other EJBCA integrations.

Next steps

In this tutorial, you learned how to clean up/reset the MicroK8s cluster and deploy MariaDB along with EJBCA using Helm.

Here are some next steps we recommend:

Unable to render include or excerpt-include. Could not retrieve page.