pgast
The Postgres dialect AST in Lean 4 + the pretty-printer gated against Postgres's own parser. Small and closed by design; the module the provable-migration work builds on.
| Latest | 17.6.2 |
|---|---|
| Versions | 3 |
| Category | Modules & tooling |
| Maintainers | Matt Marshall |
| Registry | https://registry.tbzl.dev/modules/pgast/ |
| Source | github.com/leangres/pgast |
bazel_dep(name = "pgast", version = "17.6.2")
View source & releases on GitHub ↗
The Postgres dialect AST, in Lean 4 — and the pretty-printer that makes it SQL.
The writing side of leangres: a small, closed, typed AST whose printed output is gated against Postgres’s own parser. This is the module the provable-migration work builds on.
Part of leangres.
The extension seam
sqlast holds the dialect-neutral ANSI
shapes, parameterised over a dialect extension. This module fills that slot:
abbrev Expr := Polyglot.Sql.Ast.Expr ExprExt
contributing 22 Postgres-specific expression constructors (ilike, eqAny,
ltreeDescendantOf, the jsonb* family, regexMatch, tgOp, …), 7
SelectSource constructors, and the 29-case PL/pgSQL BodyStmt.
Small and closed, on purpose
pgquery is complete over Postgres’s
grammar and proves nothing — it exists for reading SQL. This module is the
opposite trade: deliberately incomplete, because completeness would mean carrying
grammar corners nobody can state a theorem about.
Every constructor here has a printer, a place in an exhaustiveness lock, and a pinned rendering.
Two disciplines worth understanding before extending it
The exhaustiveness lock. Pg.ProceduralSurface classifies every BodyStmt
via a non-catchall match, plus a theorem that every case is one of the known
shapes. Adding a constructor breaks the build until the printer, the semantics
and the theorem are all updated. Discipline that depends on reviewers remembering
is not discipline.
The printer pins. 16 test targets assert exact printed output, one per constructor family. They are surface locks, not theorems: a printer change that alters bytes fails here loudly. Combined with the lock above, a new constructor cannot ship un-printed or un-exercised.
What is here today, and what is missing
Pg.Stmt.Stmt has 14 constructors: 13 CREATE plus one alterTable whose
action set is four row-level-security toggles.
Missing, and load-bearing for migrations: DROP of anything, RENAME, real
ALTER TABLE (add/drop/alter column, add/drop constraint), ALTER TYPE … ADD VALUE, GRANT/REVOKE, COMMENT ON, views, sequences, extensions — and all
top-level DML. INSERT/UPDATE/DELETE exist only inside PL/pgSQL function
bodies as BodyStmt arms.
rules_postgres’s changelog named the gap precisely:
“
ADD COLUMN/DROP COLUMNare the obvious next arms and are deliberately omitted: a migration DSL that can drop columns raises a different safety question, and nothing needs it yet.”
Closing that is the point of this module’s next phase, and of
pgmigrate above it.
⛔ Never add a raw : String → Stmt escape hatch. One raw arm makes every
theorem stated over this AST vacuous for any schema that uses it. The same goes for
DO $$ … $$ anonymous blocks.
A trap for anyone writing an evaluator
Expr Ext is a nested inductive — it carries List (Expr Ext). Structural
recursion is rejected on the list arms; well-founded recursion is accepted but
does not reduce under rfl, and simp with equation lemmas also fails.
Fuel-index the evaluator and prove a fuel-monotonicity lemma before stating
anything universally quantified. Relatedly, String.splitOn never reduces — write
a structural splitter over .data.
Contents
| module | what |
|---|---|
Pg.Ty | PgType — Postgres’s type names |
Pg.Ast | ExprExt, SelectSourceExt, BodyStmt, Identifier, Volatility |
Pg.Stmt | the Stmt inductive + the Create*Stmt structures |
Pg.AstSmart | inlined smart constructors for the Ext arms |
Pg.Pretty | printStmt : Stmt → String, modelled on libpg_query’s deparse |
Pg.ProceduralSurface | the exhaustiveness lock |
Pg.RegexAst | a Postgres-internal regex AST |
The printer is pure String concatenation with no hashmaps, so output is
bit-identical across runs — which is what makes byte-level pins meaningful.
Dependencies
sqlast (the generic AST) and pgcatalog (Pg.Catalog.QualifiedName serves as
Identifier, so the same value works as an emit input and as a key into a
catalog snapshot). Lean core otherwise — no mathlib, no batteries.
Notably not rules_lang: its atlas also ships Polyglot.Sql.*, so depending
on both would put two providers of one module namespace in a single dep closure.
Taking sqlast avoids that by construction.
Consuming it
bazel_dep(name = "pgast", version = "17.6.0")
lean_library(
name = "my_emit",
srcs = ["MyEmit.lean"],
deps = ["@pgast//lean:pgast"],
)
Published as compiled oleans — pin the same toolchain
(leanprover/lean4:v4.30.0-rc2) and select the artifact for your platform.
Versioning
<pg_major>.<pg_minor>.<patch>. PgType enumerates Postgres’s type names and
ExprExt its operators, both of which move between majors.
⚠ A convention, not enforced: compatibility_level would have been the
mechanism and Bazel 9 made it a no-op. See
pgcatalog’s MODULE.bazel for the two
candidate enforcement mechanisms, neither built yet.
Provenance
Carved from
tomato-bazel/rules_postgres
with git filter-repo, history preserved.
License
MIT.
Conformance#
No gate findings. 8 contested atoms. See how gating works or the full report.
Contested atoms
Third-party modules where this module resolves a different version than others do. Not a violation of anything this module did — it is the actionable form of a registry-level convergence finding, and the sentence a maintainer can act on.
| Atom | Resolved here | Elsewhere |
|---|---|---|
apple_support | 1.24.2 | 2.2.0 ×1 |
bazel_skylib | 1.8.2 | 1.9.0 ×2 |
nlohmann_json | 3.6.1 | 3.12.0.bcr.1 ×1 |
protobuf | 33.4 | 34.0.bcr.1 ×2 |
rules_jvm_external | 6.7 | 6.8 ×4 |
rules_python | 1.7.0 | 2.0.1 ×1 |
rules_swift | 3.1.2 | 3.6.1 ×1 |
upb | 0.0.0-20220923-a547704 | 0.0.0-20230516-61a97ef ×1 |
Dependencies#
Depends on
Used by (1 in the registry)
Versions#
3 published versions, newest first. Each resolves to an immutable, integrity-checked archive.
| Version | Integrity (sha256) | Source archive |
|---|---|---|
17.6.2 latest | KmRerrcML8YxhwQR… | tag archive ↗ |
17.6.1 | rIBq/afLP9l7r6Vw… | tag archive ↗ |
17.6.0 | hBAb2ck/dhhs76sQ… | tag archive ↗ |
Changelog#
17.6.2 — top-level INSERT / UPDATE / DELETE
INSERT/UPDATE/DELETE existed only as PL/pgSQL BodyStmt arms. The
Aion-owned half of the migration corpus is exactly the top-level
INSERT INTO graph.* files, so none of it could be expressed.
Adds InsertSource (values / query / defaultValues), InsertStmt,
SetClause, UpdateStmt, DeleteStmt, and the three Stmt arms, with printer
arms and 8 pins. ON CONFLICT reuses the existing ConflictAction rather than
duplicating it.
Scoping — these live outside Pg.Ast’s mutual block. They are plain
structures over the existing Expr / SelectQuery abbrevs. Sub-links
(EXISTS (SELECT …), x IN (SELECT …)) and data-modifying CTEs genuinely do
need mutual recursion with Expr; growing that block forces decisions about
deriving and optParam that this change does not need to make. So withCtes
is absent rather than present-and-ignored — a field you cannot use reads as
support that is not there.
BodyStmt is untouched. Collapsing its duplicated qualified/unqualified pairs
into a single dml arm is deferred, which is why all 137 existing printer pins
are unmoved by this release.
Two pins render with parentheses that look redundant — WHERE (kind IS NULL),
WHERE (id = 1) — because isNull and eq parenthesise their own operands.
They are pinned as-emitted, from a probe of the real printer rather than from a
prediction. Tightening the printer is a separate change that these pins would
then catch.
17.6.1 — DDL that mutates: the DROP family and a real ALTER TABLE
Stmt was 13 CREATEs plus an alterTable whose action set was four
row-level-security toggles. A migration is a state transition, and none of
that could express one.
AlterTableAction 4 → 17. Columns (addColumn, dropColumn, setNotNull,
dropNotNull, setDefault, dropDefault, setColumnType with USING),
constraints (addConstraint, validateConstraint, dropConstraint), and
renames/placement (renameColumn, renameTable, setSchema) — alongside the
original four, unchanged.
Stmt.dropObject. One DropStmt carrying a DropTarget, rather than a
constructor per object kind — what Postgres’s own grammar does. The printer, and
later the hazard classifier and catalog transition, each get one arm with an
inner match, and the exhaustiveness lock still bites because adding a target
kind breaks that match.
Function-like targets carry their argument types and table-scoped ones carry
their table, because Postgres needs both to disambiguate: DROP FUNCTION f is
ambiguous wherever two overloads exist.
The field that matters most
addConstraint’s notValid. ADD CONSTRAINT ... NOT VALID takes a brief
lock and does not scan; a plain ADD CONSTRAINT takes ACCESS EXCLUSIVE and
scans the whole table. On anything large that is the difference between an online
migration and an outage.
It is also what makes the correct backfill ordering expressible at all. Postgres
enforces a NOT VALID constraint on new and updated rows immediately, so
adding it before a backfill closes the race against concurrent writers, and
validateConstraint scans afterwards under a weaker lock. The usual
add-column/backfill/SET NOT NULL recipe has neither property.
Rendering moved, byte-for-byte
AlterTableAction.toSql could not render the new arms: they carry Expr,
ColumnDef and TableConstraint, whose printers live in Pg.Pretty, which
imports Pg.Stmt. Rendering moved to Pg.Pretty.printAlterTableAction, with
rlsToSql kept in Pg.Stmt so the four RLS renderings are still defined once.
The 16 existing printer pins passing unchanged is the evidence that move was byte-neutral — they assert exact output, so any drift would have failed them.
deriving DecidableEq, Repr is gone from AlterTableAction, matching
ColumnDef and TableConstraint, which already omit it because Expr is a
nested inductive.
New pins
17 in Pg/AstDropAlterTest.lean, one per new shape. Two are pinned to output
that reads oddly and is nonetheless correct — isNotNull parenthesises its own
operand so a CHECK renders doubled parens, and typeCast wraps the whole cast.
Pinned as the printer emits, not as it would ideally read: the pin’s job is to
catch drift, not to editorialise.
Still missing
Top-level DML — INSERT/UPDATE/DELETE remain PL/pgSQL-body-only. Also
ALTER TYPE ... ADD VALUE, views, sequences, extensions, GRANT/REVOKE,
COMMENT ON. And Stmt has no raw : String arm; it must never get one, as
that would make every theorem stated over this AST vacuous.
17.6.0 — carved out of rules_postgres
Pg.{Ty,Ast,Stmt,AstSmart,Pretty,ProceduralSurface,RegexAst} plus the PL/pgSQL
layer and the 16 per-constructor printer pins, extracted from
tomato-bazel/rules_postgres with git filter-repo, history preserved
(14 commits). 34 files, 5,983 lines, each verified byte-identical to source by
sha256.
The seam works. Pg/Ast.lean does
abbrev Expr := Polyglot.Sql.Ast.Expr ExprExt against @sqlast, and
Pg.Catalog.QualifiedName from @pgcatalog serves as Identifier — both arriving
as compiled oleans, not sources. All 16 printer pins pass against the compiled
core, which is the load-bearing check: they assert exact printed output, so if
consuming the core as oleans changed anything observable they would fail.
Not rules_lang. Its atlas also ships Polyglot.Sql.*, so depending on both
would put two providers of one module namespace in a single dep closure. Taking
@sqlast sidesteps that by construction rather than by coordinating a rules_lang
release — which was the one step of the leangres split that looked like it could
not be done additively, and turned out not to need coordinating at all.
Per-target compile cost. rules_postgres re-listed all ten core sources in
every one of the 16 pin targets. Here each lists only its own file and takes the
compiled core via deps: 16 × 11 = 176 file-compiles becomes 7 (once) + 16 = 23.
Requires rules_lean 0.6.1 — earlier releases’ lean_olean_archive fails on linux.