Dynamic Data Masking

1. Data steward classifies columns in the catalog, tagging fields like credit_card, ssn, and email as sensitive and mapping each tag to a masking function (full redaction, partial XXXX-1234, or hash).
2. Governance engine stores these masking policies alongside role-to-clearance bindings, so a single physical table serves every audience without duplication.
3. Analyst submits an ordinary SQL query against the curated table; there is no separate "redacted view" to select.
4. Query engine intercepts the logical plan, resolves the caller's identity and group membership, and rewrites projections on tagged columns to inject the appropriate masking expression before execution.
5. Compute layer evaluates the masked expression row-by-row at scan time; unprivileged callers never receive the raw bytes, while privileged roles see cleartext through the exact same query.
6. Audit sink records who queried which sensitive columns and which mask was applied, feeding compliance dashboards and anomaly detection.

• Zero data duplication vs. added query-plan rewriting and per-row masking overhead at scale.
• Single source of truth vs. reliance on correct identity propagation; a misconfigured service role leaks cleartext.
• Application-transparent vs. masking is only as strong as the compute boundary—direct file access to the underlying storage sidesteps it.
• Centralized policy vs. engine lock-in, since masking syntax rarely ports cleanly between platforms.

A payments fintech keeps a single transactions table in its lakehouse. Fraud analysts querying it see full card numbers because their role carries PCI clearance; a marketing team running the identical SELECT for cohort analysis receives XXXX-XXXX-XXXX-4419, and offshore support sees a fully redacted ********. When an auditor later asks how customer PANs are protected in analytics, the team points to one policy set in the catalog rather than a sprawl of nightly-scrubbed copies—eliminating dozens of ETL jobs that previously existed only to produce redacted clones.

• Storage-layer bypass: Masking lives only in the query engine, so any path that reads underlying files directly (external table engines, storage-API access, raw exports) sees cleartext. Lock down object-store ACLs and restrict which engines can mount the physical path.
• Identity propagation: Masks resolve against the caller's identity, so pooled connections, shared service accounts, or BI tools that connect as one technical user collapse everyone to a single clearance. Enforce per-user identity pass-through end to end.
• Policy-tag coverage drift: New columns and CTAS/SELECT *-derived tables inherit no tags by default, silently exposing sensitive data. Add pipeline checks that block untagged columns matching PII patterns and re-run classification on schema changes.
• Query-plan overhead: Per-row masking functions (especially hashing) execute at scan time and defeat predicate pushdown or column pruning on tagged fields, inflating scan cost on wide, high-cardinality tables. Prefer cheap partial-redaction expressions and benchmark masked vs. unmasked scans.
• Audit completeness: Log the caller, columns touched, and mask applied per query, not just table access, so you can prove which roles ever saw a given PAN. Alert on privileged roles reading sensitive columns at anomalous volume.
• Portability and versioning: Masking DDL and function syntax rarely port between engines; keep policy definitions in a declarative catalog spec under source control and version tag-to-mask mappings so historical audits reflect the rule in force at query time.