CASE STUDY | CANADIAN PORK COUNCIL
Canadian Pork Council
Restoring National Traceability at Scale
INDUSTRY
Agriculture
Data Migration
Reporting & BI
ETL Modernization
SOLUTION AREA
AWS S3 & QuickSight
Glue & Iceberg
Eventbridge
TECHNOLOGY
The Challenge
The Canadian Pork Council (CPC) is a federation of nine provincial pork industry associations representing 22,000 farms. The organization plays a leadership role in achieving and maintaining a dynamic and prosperous Canadian pork sector. The CPC needed reliable, monthly national reconciliation of pig-movement data received through its mandatory PigTRACE program. A legacy PHP/Oracle job that processed 80M records locked the production database for hours, required manual spreadsheet reporting, and left the CPC unable to deliver timely compliance insight. Data Elephant replaced the brittle process with a serverless AWS Lakehouse to restore national monitoring, automate reporting, and dramatically cut costs and run-time.
CPC was facing production outages including an 8-hour “brute force” reconciliation job, legacy and inefficient reprocessing, manual work and hours spent compiling spreadsheets for financial compliance reports, and unsustainable costs across licensing and infrastructure.
The Data Elephant Difference
Data Elephant was selected as CPC’s trusted AWS partner to design and implement a modern Medallion Lakehouse on AWS over a 4-week project that moved compute off the legacy production database, implemented managed chunking for large windows, and automated the full compliance lifecycle:
Serverless Medallion Architecture using S3 Tables and Iceberg for ACID, partitioning and compaction
Full orchestration with AWS Step Functions and EventBridge; ingestion and ETL via AWS Glue
Infrastructure-as-Code (8-stack IaC) including TypeScript-defined QuickSight dashboards and YAML schema-as-code
Governance-as-code, data quality checks that sent alerts with SNS emails, and end-to-end tests to ensure production stability
Automated QuickSight reporting (sub-second dashboards) to eliminate manual spreadsheets
“Data Elephant delivered an incredibly valuable improvement to one of our most important business tools. Their work also provided a vision for further modernization of data processing in our new AWS Cloud environment following the lift-and-shift of our platform from on-premises. We are excited for the future and will continue to lean on Data Elephant for guidance, advice, and execution.”
JEFF CLARK - DIRECTOR, PIGTRACE CANADA
The Outcome
Data Elephant delivered a resilient, production-grade compliance engine that removed database contention, automated reporting, and restored national traceability - all within a four-week engagement. CPC now runs reliable, audit-ready reconciliation monthly with instant dashboards and far lower operational costs.
Results at a Glance
SEAMLESS MIGRATION
8M+ records processed and restored, predictable monitoring and compliance
REDUCED RUN TIME
Reduced Run Time by 96% from 8 hours to 20 minutes
TIME SAVINGS
Hours of manual work eliminated through automation and dashboards
COST REDUCTION
Significantly reduced cloud data infrastructure costs
