The managed data engineering platform
Data engineering that proves its work.
PillarMesh is being built for small data teams that need to operate sources, a dedicated warehouse, transformations, catalog, dashboards, and evidence as one governed system.
The operational problem
The stack is connected. Accountability is not.
Small data teams often carry source acquisition, warehouse operations, transformations, catalog, dashboards, retries, recovery, and evidence across separate tools. Each boundary creates another place for meaning, state, and responsibility to drift.
Python, SQL, schemas, retries, and backfills are familiar. Spark-style processing, pipelines, and tests are familiar. Reconstructing whether a business outcome was approved, operated, and reconciled should not consume the team.
Planned system
The operating surface, held together.
The planned initial managed warehouse engines are PostgreSQL and ClickHouse.
- Maturityplanned
Source acquisition
Operate supported source connections.
- Maturityplanned
Managed warehouse
Run a dedicated PostgreSQL or ClickHouse data plane.
- Maturityplanned
Transformations
Compile inspectable, versioned transformation artifacts.
- Maturityplanned
Catalog and BI
Operate governed discovery, dashboards, and reports.
- Maturityplanned
Recovery
Coordinate replay, backup, restore, and bounded remediation.
- Maturityplanned
Evidence
Reconcile source-to-consumer outcomes with attributable facts.
Operating sequence
Make the work legible from the start.
- 01Connect
- 02Declare
- 03Approve
- 04Operate
- 05Prove
Design principles
Contract-first, deterministic, and attributable.
Contract-first
A durable Integration Contract will state the outcome, boundaries, and constraints before a physical plan is selected.
Deterministic
Canonical inputs and compiler versions will produce the same semantic artifacts or an attributable No Valid Plan result.
Attributable
Approval, execution, reconciliation, and evidence will remain inspectable instead of being flattened into a green status.
Conceptual
Conceptual execution evidence
A conceptual evidence path: a contract admits a plan, the admitted plan records a run, and the run produces attributable evidence.
Scope and non-goals
A deliberately bounded first scope.
The planned first operating surface covers supported sources, a dedicated PostgreSQL or ClickHouse warehouse, transformations, catalog, dashboards, reconciliation, recovery, and evidence.
PillarMesh is not a general-purpose workflow scheduler, a universal connector layer, or an autonomous authority for meaning and policy.
Design partners
Bring a real operating constraint.
PillarMesh is seeking small, hands-on data teams willing to examine the planned model against concrete source, warehouse, authority, recovery, and evidence constraints.
Explore design partnership