Skip to content

Delta Table Maintenance Simplified

A Delta table is a folder of files plus a transaction log. Nine ideas explain why it slows down and how OPTIMIZE, VACUUM, clustering, and deletion vectors keep it fast. Build, wreck, diagnose, and repair a simulated table on this page.

Prasanth Sistla7 min read

TL;DR

A Delta table is a folder of data files plus a transaction log that records every change. It slows down because every write adds files, and small frequent writes add many tiny ones; nothing breaks, reads just get slower. Four maintenance moves fix it: compact tiny files into large ones (OPTIMIZE), clean up old copies after a retention window (VACUUM, default 7 days), lay out rows so queries skip whole files (clustering), and purge rows that were deleted by bookmark (deletion vectors). Cleanup and time travel pull against each other: the files you delete are the versions you can no longer rewind to. Measure before and after, and run the work on a schedule so a slow dashboard is a rare event.

This page carries a simulated table. You will decay it, compact it, clean it up, break a rewind, and diagnose it. The strip below is its live health monitor.

Scope

Platform-agnostic. Command names follow the open-source Delta Lake project; every vendor adds write-time optimizations and schedulers on top, and those are a separate subject. The table on this page is a simulation built for intuition: sizes are illustrative and a rewind here is a feasibility check.

the idea in one paragraph
files 3avg size 109 MBdisk 3versions 3bookmarks 0speed

1. A table is files plus a diary

Your table is two things: a folder of data files, and a diary that records every change ever made to them. Nothing edits a file in place. A change writes new files and adds a diary entry naming which files came in and which went out. Every maintenance question on this page is a question about the files, the diary, or the gap between them.

Your table · tap a diary entry to see which files it referenced
healthy file tiny file old copy on disk
The diary

2. Tables get messy by default

Every write adds files. Frequent small writes add lots of tiny files, and reading many tiny files is slow. Nothing is broken and every result stays correct. The table just quietly decays. Try it.

Decay lab · watch the health strip
tiny files 0a query reads 3 files

3. Compaction: merge small into large

The fix is simple: combine many tiny files into a few big ones. Same rows, fewer files, faster reads. One command does it.

Repair lab · needs tiny files from lab 2
note: disk usage does not drop. Old copies stay until cleanup (lab 4).

4. Cleanup reclaims space

Old copies pile up on disk so you can rewind (next idea). Cleanup deletes copies older than a retention window you choose. Space comes back; the oldest history goes.

Cleanup lab
7 days

5. Rewind and cleanup pull against each other

Because old copies stay on disk, you can rewind the table to any past version, right up until cleanup deletes the files that version needs. How far back you can go is exactly what you have not yet thrown away.

Time travel lab · try a rewind, then clean up in lab 4 and try again
v2Loaded February data · 1 filetoday
v1Loaded January data · 1 file1d ago
v0Created table · 1 file2d ago

6. Layout: keep similar rows together

If rows with similar values sit in the same files, a query can skip whole files that cannot contain its answer. A skipped file is free. Compare a messy layout with a sorted one.

File skipping lab · each bar is one file's value range
42
file_112–44
file_255–90
file_33–30
file_438–72
file_561–100
file_620–52
file_770–98
file_88–41
file_945–83
file_1027–60
this query reads 4 of 10 files

7. Deletes are bookmarks first

Files cannot be edited, so deleting a row does not rewrite the file. The row gets a bookmark that says "ignore me." That is fast, and bookmarks accumulate. Eventually you tidy up for real.

Delete lab · tap rows in this file to delete them

✕ = deleted by bookmark. Readers filter these out on every query. 24 rows in the file.

8. Measure, then act

Two questions diagnose almost everything: what does the table look like right now, and what has been happening to it? Ask them before and after every intervention. Ask them about the table you built on this page.

Diagnosis lab

9. Hygiene, on a schedule

Maintenance is scheduled, like backups. A slow dashboard should be a rare event, because the cadence ran first. Pick one, write it down, automate it. Assemble a starter policy:

Runbook builder · toggle what applies
Your runbook appears here.

Where this leaves you

Everything above is open-source Delta Lake behavior, checked against the project documentation in September 2026. Managed platforms change how fast a table decays, mostly through write-time defaults, and none of them change the nine ideas. If you run Delta on Microsoft Fabric, the five levers article maps these ideas onto that platform's defaults.

Reset the table in the strip above and run the sequence once more without reading: write, compact, clean up, rewind. If you can predict what each number does before you press the button, you have the model.