Lack of Graviton Usage in Databricks Clusters

Matt Weingarten

Service Category

Compute

Cloud Provider

Databricks

Service Name

Databricks Clusters

Inefficiency Type

Suboptimal Instance Selection

Explanation

Databricks supports AWS Graviton-based instances for most workloads, including Spark jobs, data engineering pipelines, and interactive notebooks. These instances offer significant cost advantages over traditional x86-based VMs, with comparable or better performance in many cases. When teams default to legacy instance types, they miss an easy opportunity to reduce compute spend. Unless workloads have known compatibility issues or specialized requirements, Graviton should be the default instance family used in Databricks Clusters.

Relevant Billing Model

Databricks compute is billed based on:

Databricks Units (DBUs): Determined by cluster configuration, including instance type

AWS Infrastructure Charges: Graviton-based instances (e.g., m6g, r6g, c6g) are generally priced lower than their x86 equivalents, billed per second or minute

Choosing x86-based instances when Graviton would suffice leads to higher infrastructure costs for equivalent performance.

Detection

Query cluster configurations to identify use of non-Graviton (x86-based) instance types
Check for absence of Graviton enforcement in workspace-level compute policies
Review workload compatibility for clusters running frequently on x86
Engage with teams to confirm whether Graviton instances have been tested or benchmarked

Remediation

Monitor utilized instance types and recommend Graviton-based families
Reconfigure default cluster templates to use Graviton by default
Allow exceptions only for workloads with documented compatibility or performance issues

Relevant Documentation