Optimizing Kubernetes with Cluster Rightsizing

Overview

CloudNatix provides cluster rightsizing. In principle, cluster rightsizing is similar to workload right-sizing; it provides an optimal configuration of a cluster based on past historical usage of all workloads running in the cluster.

The recommended cluster configuration generated by CloudNatix includes optimal instance types as well as optimal number of nodes. Here is an example case where CloudNatix recommends a smaller instance type as well as a smaller number of nodes for the cluster.

CloudNatix generates recommendations by running scheduling simulations and finding an optimal configuration among various possible configuration combinations (today "optimal configuration" is the lowest cost). The simulation takes various scheduling factors into account, including resource fragmentation, node selector, taints, and resource headroom.

Since the simulation can take some time, CloudNatix periodically makes the simulations and generates recommendations with a set of parameter combinations. That is stored in the database in CloudNatix and can be viewed from GUI or CLI. Users can also make some "one-shot" simulation and see how it goes.

Right now the following parameters are available for cluster rightsizing:

• constraint: how to choose the instance types in a recommendation. Possible variations are:
  • NO_CONSTRAINT: any instance type can be used in a recommendation
  • SAME_INSTANCE_TYPES: choose the instance types currently used in the cluster. That means instance types won't change, the recommendation simply changes the number of instances.
  • SAME_INSTANCE_FAMILY: choose the instance types whose families are same in the ones in the cluster. It does not change the instance family but can change its size.
• resources source: how to obtain the resource (CPU or memory) requests of individual workloads. Possible variations are:
  • REQUEST: the resource request specified in the workload's spec.
  • LIMIT: the resource limit set in the workload's spec.
  • VPA_RECOMMENDED: the resource amount recommended by the vertical pod autoscaler. That means the recommendation would provide the simulation result when autopilot is applied to all the workloads.
  • VPA_LOWER_BOUND: the lowest value of resource found in the vertical pod autoscaler. This is the cheapest possible result for the simulation.
• headroom ratio: the resource to be kept unused in the individual instances. A single value can be stored as a configuration of a cluster, and all pre-computed recommendations will use that value. The default value is 10%.

View Recommendations

You can view cluster rightsizing recommendations from CloudNatix GUI console (https://app.cloudnatix.com) or with CloudNatix CLI.

Viewing from CloudNatix GUI Console

Select "clusters" from the left tab. It shows the list of clusters. Expand the target cluster, and select "Recommendation" tab in the cluster detail view.

The right-side of the recommendation shows the tree-map view of the cluster resource, current usage vs the recommendation. Each rectangle is a node (an instance) in the cluster, and the size of the rectangle indicates the capacity (CPU or memory), and the color indicates its utilization. Typically it should show smaller rectangles in the recommendation (which means more compact instances) which achieves the cost reduction.

The left-side of the screen shows the summary of the recommendation for various aspects, e.g. number of instance types or nodes.

The lower-side of the screen lists the node groups (or node pools) in the cluster and comparison between the current usage (shown in grey text) and the recommendation (shown in white test).

The "settings" can be opened from the tab and some parameters (e.g. constraint or resources source) can be customized.

Viewing with CloudNatix CLI

To view recommendations with CLI, type the following command:

cnatix recommendations clusters get --cluster-name ${CLUSTER_NAME}

This provides the current recommendation for the cluster. --constraint and --resources-source flag can be specified to change the parameter. Note that NO_CONSTRAINT and REQUEST are the default values for those parameters.

If you want to see the past simulation results, you can use the following command:

cnatix recommendations clusters list --cluster-name ${CLUSTER_NAME}

Similarly --constraint and --resources-source can be specified.

This command shows the recommendations for 7 days, aggregated each day. The duration can be changed through --days flag.

When the duration is simply 1 day (e.g. --days=1), it shows individual recommendation in that day.

cnatix recommendations clusters list --cluster-name ${CLUSTER_NAME} --days 1

Each recommendation has its own UUID. This UUID can be used for the get command to see the detailed information.

cnatix recommendations clusters get --uuid ${RECOMMENDATION_UUID}

Apply the Recommendation

We haven't yet provided a tool to apply the cluster rightsizing recommendations. For now, please see the result and apply them manually by yourself.