Unnecessary Reset of Long-Term Storage Pricing in BigQuery

Ben de Mora

Service Category

Database

Cloud Provider

GCP

Service Name

GCP Big Query

Inefficiency Type

Behavioral Inefficiency

Explanation

BigQuery incentivizes efficient data retention by cutting storage costs in half for tables or partitions that go 90 days without modification. However, many teams unintentionally forfeit this discount by performing broad or unnecessary updates to long-lived datasets — for example, touching an entire table when only a few rows need to change. Even small-scale or programmatic updates can trigger a full reset of the 90-day timer on affected data. This behavior is subtle but expensive: it silently doubles the storage cost of large datasets for another 90-day cycle, even when the data itself is mostly static. Without intentional safeguards, organizations may repeatedly reset their discounted storage window without realizing it.

Relevant Billing Model

BigQuery storage is billed based on:

Active (Standard) Storage: Full price for tables or partitions modified in the last 90 days
Long-Term Storage: 50% discount applies after 90 days without modification. Any change — including update, insert, or delete — resets the 90-day clock for modified data.

Detection

Identify tables or partitions with high storage volume that consistently fail to enter long-term storage pricing
Correlate storage cost fluctuations with write operations or ETL jobs
Review update patterns on reference or static datasets (e.g., product catalogs, configuration data)
Check whether queries or pipelines are modifying more data than necessary

Remediation

Limit write operations to the exact data that requires change — avoid broad table rewrites
Partition large datasets so updates are scoped to specific partitions, minimizing disruption to cold data
For static reference tables, use append-only patterns or restructure workflows to avoid unnecessary modification
Educate teams on the impact of modifying long-lived data and incorporate cost-aware practices into pipeline development

Relevant Documentation

BigQuery Storage Pricing