petabytedata.dev
← ALL ARTICLES

Lakehouse

Treat data layout as a workload decision

A practical way to reason about layout without turning every table into the same optimization experiment.

Describe the readers first

Record common filters, join keys, update patterns and expected freshness. A table scanned once a month has different needs from a frequently queried operational dataset.

Measure a baseline

Use representative queries and record runtime, scanned bytes and compute configuration. Repeat runs and distinguish warm-cache results from cold reads. Avoid presenting a single faster run as a general performance claim.

Change one variable

Compare candidate layouts on the same data snapshot. Include ingestion overhead, maintenance cost and skew in the decision. An improvement in one query can create a regression elsewhere.

Keep the decision reversible

Document why a layout was chosen and the workload that justified it. Revisit the decision as table size and access patterns change.

More from the blog

AUTHOR PROFILE

Divya Madhavan

Certifications

Honors