What a data cleaning report should cover
A useful data cleaning report answers:
- Which fields failed validation?
- Where are the duplicates?
- Which values are missing and how should they be annotated?
- Which records look anomalous and why?
Field validation checklist
| Field Type | Checks | Example |
|---|---|---|
| Format, domain | user@example.com | |
| Phone | Format, country code | +1-555-123-4567 |
| Date | Parseable, range | 2026-03-16 |
| Numeric | Type, range | 0 ≤ value ≤ 100 |
| ID | Uniqueness, format | UUID or internal ID pattern |
Define these in your AGENTS.md or skill config so OpenClaw applies them consistently.
Duplicate detection
- Use deterministic keys: email, phone, or composite keys.
- Flag fuzzy duplicates separately (e.g., minor spelling differences) and treat them as "review" items, not auto-merge.
Missing value annotation
- Distinguish: empty, null, placeholder (e.g., "N/A"), and unknown.
- Report counts per category so you can decide whether to impute or exclude.
Anomaly markers
- Outlier thresholds (e.g., values beyond 3σ)
- Impossible combinations (e.g., end_date before start_date)
- Sudden spikes or drops compared to historical baselines
Report format example
# Data Cleaning Report — 2026-03-16
## Summary
- Total rows: 1,240
- Validation failures: 12
- Duplicates: 5
- Missing (actionable): 8
- Anomalies: 3
## Validation Failures
| Row | Field | Value | Issue |
|-----|-------|-------|-------|
| 42 | email | invalid | Malformed |
| 87 | phone | 123 | Too short |
## Duplicates
| Key | Count | Rows |
|-----|-------|------|
| user@a.com | 2 | 12, 305 |
## Anomalies
| Row | Field | Value | Note |
|-----|-------|-------|------|
| 501 | amount | 99999 | > 99th percentile |
Install relevant skills
openclaw skills install data-validation
openclaw skills install csv-report
Check available skills for your OpenClaw version; names may vary.
Review checklist before publishing reports
- Validation rules match current schema
- Duplicate logic uses the correct key
- Missing value categories are defined
- Anomaly thresholds are documented
- Report schedule (daily/weekly/monthly) is set
Daily, weekly, monthly templates
- Daily: Quick counts and critical failures only.
- Weekly: Full validation + duplicate summary + trend notes.
- Monthly: Full report + aggregated metrics + recommendations for schema or process changes.