Data discovery and classification: building a data map you can trust

Almost every failure in a privacy programme traces back to the same root cause: the organisation did not know where its personal data was. You cannot honour a deletion request in a system you forgot about. You cannot report on a breach in a table you did not know held personal data. Discovery and classification are not glamorous, but they are the foundation everything else stands on.

You cannot protect what you cannot see

Personal data spreads. It starts in the primary database, then flows into a warehouse for analytics, a SaaS tool for support, an export someone made for a campaign, a backup, a log file. Each copy is a place a right must reach and a breach could occur. A data map is simply an honest, current answer to one question: where is personal data, and what kind?

Structured is easy; unstructured is where it hides

• Structured data — rows and columns in databases and warehouses — is the tractable part. Column names, types and sample values give you strong signals.
• Unstructured data — documents, spreadsheets, email, file shares, support tickets — is where personal data quietly accumulates and where most programmes go blind. A CV attached to an email, a scanned ID in a shared drive, a spreadsheet of leads. These need content inspection, not just schema inspection.

How classification actually works

Good classification layers several techniques, because no single one is reliable alone:

• Pattern matching. Regular expressions for structured identifiers — but a string that looks like a PAN or Aadhaar number is not always one.
• Checksum validation. Many Indian identifiers have a check digit. Validating it turns a guess into a much stronger signal and cuts false positives dramatically.
• Contextual signals. A column named dob full of dates is personal data; the same dates in a column named invoice_date are not. Context disambiguates.
• Statistical and ML methods. For unstructured text, models that recognise names, addresses and other entities in free-form content.

A classifier that cries wolf is worse than none. If every scan returns thousands of false positives, people stop trusting it — and an ignored data map is no map at all.

The false-positive problem

The hardest part of classification is not finding personal data; it is finding it accurately. A tool that flags everything is easy to build and useless in practice. Precision comes from combining signals — a checksum-valid identifier in a column whose name and neighbours are consistent with personal data — and from showing why each verdict was reached, so a human can trust or challenge it. Every verdict should carry its evidence: the test that produced it.

Keeping the map current — by re-scanning, not remembering

A data map built once and filed away is wrong within weeks. New systems appear, schemas change, someone spins up a new SaaS tool. The only reliable map is one maintained by continuous re-scanning, not by institutional memory. Treat the map as a living artefact that the systems refresh, not a document a person updates when they remember to.

Lineage: from where to where

Knowing a system holds personal data is the start. Knowing how it got there and where it flows next — lineage — is what lets you reason about risk and answer questions like "if we delete this, what breaks?" or "which downstream systems received this field?" Lineage turns a static inventory into something you can act on.

From map to action

A trustworthy data map is not the goal; it is the enabler. Once you have it:

• Rights requests fan out to exactly the systems that hold the person's data.
• Breach assessment answers "what was affected" in minutes.
• Consent and retention can be enforced where the data actually is.
• Risk can be scored by where sensitive data concentrates.

Start here. Every other privacy capability is easier — or only possible — once you can see your data clearly and keep seeing it.