CloudNatix logo

User Guide

Installing CloudNatix to Kubernetes Clusters



  • Cluster Controller pods running in K8s clusters need to be able to send HTTPS requests to the Internet (e.g.,
  • Support Kubernetes version 1.21 and above.

AKS only:

EKS only:

GKE only:

  • The cluster-watcher deployment in the cloudnatix namespace needs to be associated with the service account that has a role of Kubernetes Engine Cluster Viewer and Compute Viewer. See Appendix E for the details.

Please follow and enable Workload Identity if you have not.

KIND only:

  • The minimum KIND cluster size is 1 node with 2 cores and 16GB.

Installation Procedure

This section describes the procedure for installing Cluster Controller to a target Kubernetes cluster. Cluster Controller components are deployed to the cloudnatix namespace and connect to Global Controller to provide features such as workload visibility and optimization.


Step 1. Download the CLI

Please obtain the latest version of the CLI (cnatix) from either the CloudNatix web UI, or by following the instructions in Appendix I. Note that the download link expires in 1 hour. There's a button named "Install CLI" on the side menu bar. Select version 21.2 and channel stable.

Step 2. Set up CSP credentials (for AKS, EKS and GKE)

Please set environment variables to be able to issue (readonly) API calls to CSP. It is necessary to obtain the configuration of K8s clusters during the installation process.

For AKS, run the following command to verify:

az aks list

For EKS, run the following command to verify:

aws eks list-clusters

For GKE, run the following command to verify:

gcloud container clusters list

Step 3. Set the current kubeconfig context

Please also set the context of the kubeconfig to an K8s cluster where Cluster Controller is installed.

For AKS, run the following command to generate a kubeconfig:

az aks get-credentials --resource-group <resource group> --name <cluster name>

For EKS, run the following command to generate a kubeconfig

aws eks update-kubeconfig --region <region> --name <cluster name>

For GKE, run the following command to generate a kubeconfig:

gcloud container clusters get-credentials <cluster name>

Step 4. Generate a YAML file and a Helm chart for installing Cluster Controller

First type cnatix login to login CloudNatix.

$ cnatix login

Type cnatix clusters configure and answer questions. When all questions are answered, a YAML file and a Helm chart for installing Cluster Controller is generated under ~/.cnatix/cluster-configs/<cluster-name>.

$ cnatix clusters configure
Welcome! This command will guide you through configuring install parameters.
Initializing... done

? Use current context(my-cluster) to install? Yes
- configuring AKS my-cluster cluster...
? Choose CloudNatix distribution version: 21.2
? Choose CloudNatix channel: stable
? Enable CloudNatix components auto-upgrade? Yes
? Send Kubernetes container arguments and environment variables to global controller? Yes

Writing the configuration file... done
Successfully configured install parameters!

Please run ANY of the following command to install CloudNatix Cluster Controller:
[Use cnatix CLI]: cnatix clusters install --name=my-cluster
[Use YAML file ]: kubectl apply -f /home/user/.cnatix/cluster-configs/my-cluster/ccoperator.yaml
[Use Helm Chart]: helm install -n cloudnatix --create-namespace ccoperator /home/user/.cnatix/cluster-configs/my-cluster/ccoperator

Please set the distribution to 21.2 and the channel to stable.

Note: Please note that this Helm chart includes a secret that contains an API key. If you want to avoid embedding the API key in the chart, please remove the following from the Helm chart:

  • apiKey and dockerconfigjson from values.yaml
  • secret.yaml

Please see Appendix F for the details.

Note: We will install Prometheus Operator if the target cluster hasn’t it yet. If the target cluster already has Prometheus, we will not install an additional instance. You will see a message like below:

? Choose Prometheus to use for CloudNatix:  [Use arrows to move, type to filter]
> cloudnatix/kube-prometheus-stack-prometheus - existing
  cloudnatix/prometheus-operated - existing

Note: If you want to run Cluster Controller pods to specific nodes, please answer the following questions and set a target node label and/or a toleration.

? Input a node selector in key=value format: (end if empty)

? Input a toleration key: (end if empty)

Note: CloudNatix Cluster Controller sends the spec of Kubernetes resources to Global Controller to display the information in Dashboard. Please type n to the following question if you want to remove commandline arguments or environment variables in spec information sent to Global Controller:

? Send Kubernetes container arguments and environment variables to global controller? [? for help] (y/n)

Step 5. Apply the YAML file or install the Helm chart

Apply the generated YAML file by running kubectl apply -f or install the Helm chart.

You will see pods created in the cloudnatix namespace.

$ kubectl get pods -n cloudnatix

NAME                                        READY   STATUS    RESTARTS   AGE
authtokenrefresher-654654bfdb-q875k         1/1     Running   0          65s
ccoperator-6896f487c6-xqdrh                 1/1     Running   0          87s
cluster-watcher-69ddbdb7d6-qd2xr            1/1     Running   0          54s
clusteragent-7bc457c548-449xh               1/1     Running   0          63s
connect-agent-69585c4995-dlq8s              2/2     Running   0          67s
infra-operator-7d7b768c7d-9lvm2             1/1     Running   0          49s
vpa-admission-controller-6489c786d7-24xcc   1/1     Running   0          56s
vpa-recommender-67f6f6f998-hgthh            1/1     Running   0          56s
vpa-updater-55496cc648-p48vj                1/1     Running   0          56s

The Workloads page will show a list of workloads in the cluster.

Uninstallation Procedure

Please type cnatix clusters uninstall --name <cluster-name>.

When this command runs, the ccoperator pod must be running as the pod orchestrates the uninstallation process (e.g., uninstall Cluster Controller components installed by ccoperator, delete CRDs).

Please note that the context of the kubeconfig must not be set to the one that is generated by CloudNatix. If the context is set to the CloudNatix generated one, the uninstallation won't successfully complete (as k8s API server becomes unreachable from the uninstallation process in the middle of uninstallation).

Please avoid just deleting the cloudnatix namespace as it will not unregister the cluster from Global Controller.

Appendix A: Enabling Azure Active Directory Pod Identity

Please follow Getting Started to install AAD Pod Identity if you haven’t.

Here are example commands for the installation and role assignment.

kubectl apply -f
kubectl apply -f

export CLUSTER_RESOURCE_GROUP=<resource group of the cluster>
export CLUSTER_NAME=<cluster name>

export KUBELET_IDENTITY_ID=$(az aks show -g ${CLUSTER_RESOURCE_GROUP} -n ${CLUSTER_NAME} --query identityProfile.kubeletidentity.clientId -o tsv)
export NODE_RESOURCE_GROUP="$(az aks show -g ${CLUSTER_RESOURCE_GROUP} -n ${CLUSTER_NAME} --query nodeResourceGroup -o tsv)"
export NODES_RESOURCE_ID=$(az group show -n ${NODE_RESOURCE_GROUP} -o tsv --query "id")

# Assign the roles to the Kubelet identity.
az role assignment create --role "Managed Identity Operator" --assignee ${KUBELET_IDENTITY_ID} --scope ${NODES_RESOURCE_ID}
az role assignment create --role "Virtual Machine Contributor" --assignee ${KUBELET_IDENTITY_ID} --scope ${NODES_RESOURCE_ID}

Then create a user-assigned identity and assign the Monitoring Reader role to the identity.

Step 1. Create a new identity

export IDENTITY_NAME=<identity name>
export NODE_RESOURCE_GROUP="$(az aks show -g ${CLUSTER_RESOURCE_GROUP} -n ${CLUSTER_NAME} --query nodeResourceGroup -o tsv)"

az identity create -g ${NODE_RESOURCE_GROUP} -n ${IDENTITY_NAME}

Step 2. Create a new role assignment for read-only access

export IDENTITY_CLIENT_ID=$(az identity show -g ${NODE_RESOURCE_GROUP} -n ${IDENTITY_NAME} --query clientId -otsv)
export NODES_RESOURCE_ID=$(az group show -n ${NODE_RESOURCE_GROUP} -o tsv --query "id")

az role assignment create --role "Monitoring Reader" --assignee ${IDENTITY_CLIENT_ID} --scope ${NODES_RESOURCE_ID}

Appendix B: Enabling Azure Active Directory Pod-managed Identities (preview version)

Please follow Use Azure Active Directory pod-managed identities in Azure Kubernetes Service (Preview) and create a pod identity that has the "Monitoring Reader" role in the “cloudnatix” namespace.

Step 1. Register the EnablePodIdentityPreview

az feature register --name EnablePodIdentityPreview --namespace Microsoft.ContainerService
az provider register -n Microsoft.ContainerService

Step 2. Install the aks-preview Azure CLI

az extension add --name aks-preview
az extension update --name aks-preview

Step 3. Update an existing AKS cluster with Azure CNI to include pod-managed identity.

export AKS_CLUSTER_RESOURCE_GROUP=<resource group name of your AKS cluster>
export AKS_CLUSTER_NAME=<your AKS cluster name>
az aks update -g ${AKS_CLUSTER_RESOURCE_GROUP} -n ${AKS_CLUSTER_NAME} --enable-pod-identity

Or create a new AKS cluster if you haven’t.

az group create --name ${AKS_CLUSTER_RESOURCE_GROUP} --location eastus
az aks create -g ${AKS_CLUSTER_RESOURCE_GROUP} -n ${AKS_CLUSTER_NAME} --enable-pod-identity --network-plugin azure

A kubeconfig can be obtained by the following command:

az aks get-credentials --resource-group ${AKS_CLUSTER_RESOURCE_GROUP} --name ${AKS_CLUSTER_NAME}

Step 4. Create an identity

export IDENTITY_RESOURCE_GROUP="myIdentityResourceGroup"
az group create --name ${IDENTITY_RESOURCE_GROUP} --location eastus

export IDENTITY_NAME="application-identity"
az identity create --resource-group ${IDENTITY_RESOURCE_GROUP} --name ${IDENTITY_NAME}
export IDENTITY_CLIENT_ID="$(az identity show -g ${IDENTITY_RESOURCE_GROUP} -n ${IDENTITY_NAME} --query clientId -otsv)"
export IDENTITY_RESOURCE_ID="$(az identity show -g ${IDENTITY_RESOURCE_GROUP} -n ${IDENTITY_NAME} --query id -otsv)"

Step 5. Assign a read-only role to the identity.

export NODE_GROUP=$(az aks show -g $AKS_CLUSTER_RESOURCE_GROUP -n $AKS_CLUSTER_NAME --query nodeResourceGroup -o tsv)
export NODES_RESOURCE_ID=$(az group show -n $NODE_GROUP -o tsv --query "id")
az role assignment create --role "Monitoring Reader" --assignee "$IDENTITY_CLIENT_ID" --scope $NODES_RESOURCE_ID

Step 6. Create a pod identity in the “cloudnatix” namespace.

export POD_IDENTITY_NAME="cloudnatix-pod-identity"
export POD_IDENTITY_NAMESPACE="cloudnatix"
az aks pod-identity add --resource-group ${AKS_CLUSTER_RESOURCE_GROUP} --cluster-name ${AKS_CLUSTER_NAME} --namespace ${POD_IDENTITY_NAMESPACE}  --name ${POD_IDENTITY_NAME} --identity-resource-id ${IDENTITY_RESOURCE_ID}

Appendix C: Enabling Workload Identity

Please follow to enable workload identity and create a managed identity has the "Monitoring Reader" role for the cluster-watcher service account in the cloudnatix namespace.

Step 1. Update an existing AKS cluster to enable workload identity

export AKS_CLUSTER_RESOURCE_GROUP=<resource group name of your AKS cluster>
export AKS_CLUSTER_NAME=<your AKS cluster name>

az aks update -g "${AKS_CLUSTER_RESOURCE_GROUP}" -n "${AKS_CLUSTER_NAME}" --enable-oidc-issuer --enable-workload-identity

Step 2. Create a managed identity

export IDENTITY_RESOURCE_GROUP="myIdentityResourceGroup"
az group create --name ${IDENTITY_RESOURCE_GROUP} --location eastus

export IDENTITY_NAME="cloudnatix-cluster-watcher-identity"
az identity create --resource-group ${IDENTITY_RESOURCE_GROUP} --name ${IDENTITY_NAME}
export IDENTITY_CLIENT_ID="$(az identity show -g ${IDENTITY_RESOURCE_GROUP} -n ${IDENTITY_NAME} --query clientId -otsv)"

Step 3. Assign a read-only role to the identity.

export NODE_GROUP=$(az aks show -g $AKS_CLUSTER_RESOURCE_GROUP -n $AKS_CLUSTER_NAME --query nodeResourceGroup -o tsv)
export NODES_RESOURCE_ID=$(az group show -n $NODE_GROUP -o tsv --query "id")
az role assignment create --role "Monitoring Reader" --assignee "$IDENTITY_CLIENT_ID" --scope $NODES_RESOURCE_ID

Step 4. Establish federated identity credential trust

export FEDERATED_IDENTITY_CREDENTIAL_NAME=cloudnatix-cluster-watcher-federated-identity
export AKS_OIDC_ISSUER="$(az aks show -g "${AKS_CLUSTER_RESOURCE_GROUP}" -n "${AKS_CLUSTER_NAME}" --query "oidcIssuerProfile.issuerUrl" -o tsv)"

az identity federated-credential create \
  --identity-name "${IDENTITY_NAME}" \
  --resource-group "${IDENTITY_RESOURCE_GROUP}" \
  --issuer "${AKS_OIDC_ISSUER}" \
  --subject system:serviceaccount:"cloudnatix":"cluster-watcher" \
  --audience api://AzureADTokenExchange

Appendix C: Setting up an IAM Role for the CloudNatix Service Account in EKS clusters

If a cluster uses IAM roles for service accounts, the ec2:Describe* permission or the AmazonEC2ReadOnlyAccess policy needs to given to the cluster-watcher service account in the cloudnatix namespace.

Here is an example command for creating an IAM role and associating with the service account.

CLUSTER=<your cluster name>

eksctl create iamserviceaccount \
  --name cluster-watcher \
  --namespace cloudnatix \
  --cluster "${CLUSTER}" \
  --role-name CloudNatixClusterController2 \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess \

The service account created by the above command needs to be deleted so that CloudNatix will be able to create it later from its own Helm chart.

kubectl delete serviceaccount cluster-watcher -n cloudnatix

Appendix D: Enabling GKE Workload Identity and Configuring a service account

This section explains how to enable GKE Workload Identity and give sufficient permissions to CloudNatix.

First, follow the GKE guide on Workload Identity and enable it. If you enable Workload Identity on an existing cluster, type the following command:

CLUSTER_NAME=<your cluster name>
PROJECT_ID=<your project ID>
gcloud container clusters update "${CLUSTER_NAME}" \
  --region="${REGION}" \

NODE_POOL=<node pool>
gcloud container node-pools update "${NODE_POOL}" \
  --cluster="${CLUSTER_NAME}" \
  --region="${REGION}" \

Then create a service account that grants the compute.viewer role and the container.clusterViewer role to cluster-watcher in the cloudnatix namespace.

gcloud iam service-accounts create "${SERVICE_ACCOUNT}" \

gcloud iam service-accounts add-iam-policy-binding "${SERVICE_ACCOUNT}@${PROJECT_ID}" \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:${PROJECT_ID}[cloudnatix/cluster-watcher]"

gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
  --member "serviceAccount:${SERVICE_ACCOUNT}@${PROJECT_ID}" \
  --role roles/compute.viewer

gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
  --member "serviceAccount:${SERVICE_ACCOUNT}@${PROJECT_ID}" \
  --role roles/container.clusterViewer

Appendix E: Deploying a Cluster Controller Helm Chart via a CI/CD Pipeline

Suppose that we run cnatix clusters configure and generate a Helm chart for cluster my-cluster. The Helm chart is generated under ~/.cnatix/cluster-configs/my-cluster/ccoperator.

├── ccoperator
│   ├── Chart.yaml
│   ├── templates
│   │   ├── base.yaml
│   │   └── secrets.yaml
│   └── values.yaml
├── ccoperator.yaml
└── config.yaml

Please take the following steps when checking in a Helm chat to a repository:

  • Remove ccoperator/templates/secret.yaml
  • Remove apiKey and dockerconfigjson from ccoperator/values.yaml

These contain secrets that shouldn’t be checked into a repository. The following is the content of secret.yaml.

apiVersion: v1
  .dockerconfigjson: {{ .Values.dockerconfigjson }}
kind: Secret
  labels: cloudnatix
  name: docker-pull-secret
apiVersion: v1
kind: Secret
  labels: cloudnatix
  name: {{ .Release.Name }}-secret
  api-key: {{ .Values.apiKey }}
type: Opaque

The former secret is to pull Docker images. The latter secret is to create a JWT token embedded in API calls to Global Controller. The two secrets still need to be created in a target cluster. The actual way to create the secrets depends on a customer (e.g., helm-secrets, Hashicorp Vault).

When configuring a CI/CD pipeline for multiple clusters, the generated template files can be reused for the clusters. Only values.yaml needs to be updated.

The chart is also available at Here is an example command to pull the chart.

helm repo add cloudnatix
helm repo update
helm pull cloudnatix/ccoperator

Appendix F: Helm Values Definition

This section describes the definitions of the Helm values.


These values specify the secrets used to make API calls to CloudNatix as well as pull docker image.

apiKey: ""
dockerconfigjson: ""

The value for apiKey is obtained with cnatix iam users get-apikey.

The value for dockerconfigjson is obtained with the following command:

API_KEY=<Your API key>
cat << EOF | base64 --wrap=0

  "auths": {
    "": {
      "auth": "${API_KEY}"

Please see Appendix E if you don't want to check-in these values in source code repositories.

Per-Cluster Configuration

These values specify per-cluster configuration.

clusterName: ""

csp: ""
  region: ""
  vpcId: ""
  accountId: ""
  iamRole: ""
  iamExternalId: ""
  subscriptionId: ""
  location: ""
  resourceGroup: ""
  clusterResourceGroup: ""
  vnetName: ""
  podIdentityName: ""
    resourceId: ""
    clientId: ""
  projectId: ""
  region: ""
  zone: ""
  serviceAccount: ""
  datacenter: ""
  network: ""
  username: ""

clusterName is the name of the Kubernetes cluster.

csp is set to one of aws, azure, gcp, or baremetal. For each CSP, additional parameters need to be set.

aws.iamRole and aws.iamExternalId must be set if the cluster enables IAM roles for service accounts. Please see Appendix C for the details.

CCOperator and Component Versions

These values specify the CCOperator version and the installation parameters for the components installed by CCOperator.

ccOperatorVersion: (e.g., "0.293.0")

distributionVersion: "21.2"
channelName: stable

autoUpgrade: true
componentSpecs: {}
isPrometheusInstallSkipped: false
isVpaPreinstalled: false

ccOperatorVersion specifies the CCOperator version. Please see Release Notes for the latest available version.

distributionVersion and channelName control the components versions. 21.2 and stable are the default values to be set.

autoUpgrade controls whether all components are automatically updated or not. componentSpecs specifies individual component versions. Please visit this page for details.

isPrometheusInstallSkipped is set to true if CloudNatix should not install Prometheus. This is set to true when the cluster already has Prometheus or K8s Metrics API is used for metrics collection.

isVpaPreinstalled is set to true if CloudNatix should not install VPA.

Metrics Collection Configuration

These values specify how CloudNatix Cluster Controller collects performance metrics from K8s clusters.

useK8sMetrics: false
prometheusUrl: ""
isKubeSystemManaged: false

We currently support two ways to collect performance metrics: Prometheus and K8s Metrics API.

  • To use K8s Metrics API, set useK8sMetrics to true.
  • To use Prometheus installed by CloudNatix, set useK8sMetrics to false and set prometheusUrl to an empty string.
  • To use Prometheus that already exists in the cluster, set useK8sMetrics to false and set prometheusUrl to the URL of the existing Prometheus instance.

isKubeSystemManaged is set to true when the kube-system namespace of the cluster is managed by CSP and CloudNatix is not able to access (e.g., GKE autopilot clusters).

Other Miscellaneous Configuration

These values specify other miscellaneous configuration.

scrubSensitiveData: false
bindClusterAdminRole: true
clusterOwnerRole: ""
prometheusRemoteWriteDomain: ""
reportToDatadog: ""

scrubSensitiveData is set to true if container arguments and environment variables are dropped when CloudNatix Cluster Controller sends workload metadata information to Global Controller.

prometheusRemoteWriteDomain is set when Prometheus metrics are sent to a Grafana instance managed by CloudNatix. An example value is api.monitoring.<tenant-name>

reportToDatadog is set to none to disable Datadog integration. Please see the Datadog integration page for details.

Please Appendix G for bindClusterAdminRole and clusterOwnerRole.

Additional Specs for Deployments and Pods

These values specify additional specs added to deployments and pods.

nodeSelector: {}
tolerations: []
deployment: {}
podTemplate: {}

nodeSelector and tolerations specify node selectors and tolerations that are added to the CCOperator pods and other pods created by CCOperator.

deployment and podTemplate are used to specify additional labels and annotations that are set to the deployments and pods. Here is an example values:

  - key: value
  - key: value
  - key: value
  - key: value

The finalizers controls the behavior of uninstallation. If set, the ccoperator pod deletes all resources created by CloudNatix when CloudNatix is uninstalled from the cluster.

Parameters for Internal Use

These parameters control the internal mechanism of ccoperator and experimental features. These do not need to be changed.

healthUpdateInterval: 1m0s
installLoopInterval: 1m0s
terminationGracePeriodSeconds: 300
cnatixInstallMode: FullAccess
componentsVersions: {}

Appendix G: Installing CloudNatix without Binding the cluster-admin Role

By default the ccoperator deployment and clusteragent deployment have the cluster-admin role. This is necessary to give the cluster-admin role to cluster-owner org admins who access Kubernetes clusters with CloudNatix Secure Sessions.

If you do not want to bind the cluster-admin role to the deployments, please set the following in the values.yaml:

bindClusterAdminRole: false
clusterOwnerRole: (admin|edit|view)

clusterOwnerRole controls the role that is given to cluster-owner org admins.

Appendix H: Installing the CloudNatix CLI without login

We typically suggest using the CloudNatix web UI to download the CLI, but you can also do it without direct access to the UI. To perform this installation you will need JQ. If you do not already have it you can find the download link from the project github.

After JQ is installed, run the following commands to download the CLI and install it:

For Linux:

curl "" -G \
  --data-urlencode "name=cli" \
  --data-urlencode "distribution_version=21.2" \
  --data-urlencode "channel=stable" \
  --data-urlencode "os_arch=linux-amd64" \
  | jq '.downloadUrl' --raw-output \
  | xargs curl --output cnatix
sudo mv cnatix /usr/local/bin
sudo chmod +x /usr/local/bin/cnatix

For macOS:

curl "" -G \
  --data-urlencode "name=cli" \
  --data-urlencode "distribution_version=21.2" \
  --data-urlencode "channel=stable" \
  --data-urlencode "os_arch=darwin-amd64" \
  | jq '.downloadUrl' --raw-output \
  | xargs curl --output cnatix
sudo mv cnatix /usr/local/bin
sudo chmod +x /usr/local/bin/cnatix
Billing Data BI Dashboard
Installing CloudNatix to VM Environments