Data Inventory and Cataloging Analysis
Description
This workflow analyze data inventory and identify potential PII across various business connections. It ingests data from multiple tables, transforms it to create summary tables for table counts and PII column counts, validates the presence of business glossary associations, and generates visualizations including bar charts and scatter plots to represent metadata completeness and lineage presence, ensuring schema enforcement and governance reporting.
Details
Organizations often lack a single connected view that shows where sensitive data may exist, how well assets are documented with business glossary terms, how complete metadata curation is across systems, and whether BI reporting assets have traceable lineage. Without this recipe, these signals live in separate tables and require manual cross-referencing, which slows down governance reviews and increases the risk of undetected PII or ungoverned reporting assets.
Business value
- Prioritize PII remediation and masking controls on connections with the highest sensitive-column concentration
- Direct stewardship effort toward tables and charts still missing business glossary associations
- Identify connections with weak metadata curation scores and target them for cleanup or re-profiling
- Flag BI charts with no lineage for impact-analysis risk and prioritize lineage remediation
- Use the connection-level footprint view to scope future governance rollouts by platform size
Who benefits
- Data Governance teams running periodic governance health checks
- Data Stewards conducting PII risk assessments and remediation
- Privacy/Compliance officers auditing glossary coverage and sensitive-data exposure
- BI/Analytics owners assessing lineage traceability across reporting assets
What you receive
How the recipe works (guided flow)
Step 1 — Build Enterprise Table Inventory
Joins oetable, oeschema, and connectioninfo to build a master inventory of every active table across all connections.
Step 2 — Measure Table Footprint by Connection
Counts distinct tables per connection to summarize each platform's data footprint.
Step 3 — Visualize Table Distribution Across Connections
Renders a horizontal bar chart ranking connections by table count for quick footprint comparison.
Step 4 — Prepare Column-Level Discovery Inventory
Joins column metadata to the table inventory to prepare the data for sensitive-column detection.
Step 5 — Detect Candidate PII Columns
Applies rule-based pattern matching on column names (e.g., name, email, phone, ssn, address, dob, account) to flag likely PII fields.
Step 6 — PII Columns by Connection
Aggregates distinct PII columns per connection to quantify sensitive-data exposure.
Step 7 — Visualize PII Exposure by Connection
Renders a horizontal bar chart showing PII column concentration by connection.
Step 8 — Assess Business Glossary Coverage
Checks whether each table and chart has an associated business glossary term and flags the association status.
Step 9 — Visualize Glossary Association Gaps
Renders a bar chart comparing associated versus not-associated object counts by connection.
Step 10 — Calculate Metadata Completeness by Connection
Averages the metadata curation score across tables and charts for each connection.
Step 11 — Compare Metadata Completeness Across Object Types
Renders a bar chart comparing average curation scores between tables and charts by connection.
Step 12 — Build BI Asset and Lineage Inventory
Links BI charts to their source connections and flags whether traceable lineage exists for each asset.
Step 13 — Visualize BI Assets by Connection and Lineage
Renders a scatter plot of BI charts by connection, colored by lineage presence, to spot reporting areas still lacking traceability.
Sample insights (illustrative)
| Insight Category | What the recipe discovered | Business Impact |
|---|---|---|
| PII concentration | A small subset of connections account for a disproportionate share of candidate PII columns | Focus masking and access-control effort on those connections first |
| Glossary coverage gaps | A meaningful share of tables and charts have no business glossary term attached | Directs stewardship effort to the specific assets most in need of documentation |
| BI lineage blind spots | Several BI charts show no traceable lineage back to source tables | Flags high-risk reporting assets for lineage remediation before they drive downstream decisions |
Make sure the following ingredients are available in your workspace:
- Connected datasets are crawled and profiled
- Columns have distinct counts and top values generated