Normalised records
Normalised records (silver)
Each raw row is parsed into consistently formatted fields. Every change from the raw value is recorded as a data-quality flag.
Normalised rows
99,495
usable in matching
Flagged rows
99,881
99.9% carry at least one flag
Junk quarantined
505
no usable last name
Average quality score
0.75
1.00 = no corrections needed
Quality flags
What normalisation had to do, and how often. Select a flag to filter the table.
- Placeholder values cleared71,257Placeholder tokens such as NULL, N/A, UNKNOWN or "-" treated as empty
- Whitespace trimmed57,061Leading or trailing whitespace removed from one or more fields
- Address re-cased56,490Street line re-cased and punctuation removed
- Street suffix standardised55,824Street suffix standardised to USPS abbreviation (Street to St, Avenue to Ave, and so on)
- Phone reformatted50,244Phone reduced to ten digits, punctuation and leading country code removed
- Name re-cased39,996Name letter-case rewritten to Title Case and stray characters removed
- Date of birth reformatted32,400Date of birth rewritten to ISO 8601 from a source format (MM/DD/YYYY, YYYYMMDD, Mon D, YYYY)
- City re-cased17,850City re-cased to Title Case
- Gender normalised16,828Gender mapped to F or M from a longer or variant value
- Email lower-cased11,635Email lower-cased
- Name split from “Last, First”9,980First-name field held "LAST, FIRST"; the two parts were split into their own fields
- State name converted to code9,084Full state name replaced by its two-letter USPS code
- ZIP+4 trimmed8,263ZIP+4 trimmed to the five-digit ZIP
- Member ID reformatted7,723Member ID re-hyphenated and upper-cased to the UHC-#####-## pattern
- ZIP missing7,146Street address present but no ZIP supplied
- Unit split from street6,994Apartment, suite or unit designator split from street line and re-appended in standard form
- Two-digit birth year6,661Date of birth carried a two-digit year; century inferred by pivot
- State code upper-cased4,010Two-letter state code upper-cased
- Plan code upper-cased3,278Plan code upper-cased
- SSN last four extracted2,817Last four digits of SSN extracted from a longer or punctuated value
- Date of birth unreadable2,274Date of birth did not match any known format or was out of range; cleared
- Phone missing area code2,021Phone had only seven digits (no area code); cleared
- Middle initial moved out of first name1,125A single middle initial was embedded in the first-name field and moved to middle name
- Date of birth missing990Date of birth absent or a null token; left empty
- No usable name (quarantined)505No usable last name; record quarantined as junk and excluded from matching
Normalised records
Provider network, flag “Plan code upper-cased”, junk only, lowest dq score first