Setting up an Azure SQL Metadata Store
Introduction
The metadata store is a critical component of the data masking solution. It consists of a set of tables that store metadata about data sources, masking rules, and data mappings. This section provides an overview of the metadata store, its components, and setup process.
Setting up the metadata store
To set up the metadata store, create the required tables and run the provided scripts. The scripts in the metadata_store_scripts directory facilitate the process.
Required tables
The metadata store consists of the following tables and must be in the same schema.
Table Name | Purpose |
| Defines a ruleset to mask the data. A ruleset uniquely identifies columns and the algorithms that should be applied to those columns. This table is populated automatically by the discovery pipeline and can be edited manually. |
| Defines source to destination mappings for the masking pipeline. This table maps a source dataset, database, schema, or table to a corresponding destination dataset, database, schema, or table. The rules in the ruleset (defined in
|
| Tracks how data should be treated as it flows through the masking pipeline in Azure Data Factory. The values in this table are inserted in the migration scripts.
|
Refer to the DCS for Azure templates for more comprehensive documentation on how the tables are populated.
Database versioning
The metadata store uses date-based versioning. For new setups, use the bootstrap.sql script to create the initial tables and populate them with the required data. For existing metadata stores, apply migration scripts to update to the latest version.
Migration Scripts
Migration scripts do not specify a schema, so the schema must be specified before running the scripts. The default dbo schema or a custom schema can be used.