01
Source triggers
DV360 metadata, DV360 report tables, and Ads Data Hub jobs trigger separate Cloud Run and workflow paths.
DV360 & Ads Data Hub Data Pipeline
FeaturedCase Study
Production event-driven data platform for DV360 metadata, DV360 report normalization, Ads Data Hub match-rate workflows, BigQuery processing, and reliable AWS SQS delivery.
Advertising Operations Agency5 months - January 2026Solo
System architecture
Architecture / Flow
The practical path from source data to reliable reporting output.
02
BigQuery processing
Landing/report tables are detected, normalized, and prepared for downstream delivery from BigQuery.
03
Outbox delivery
Expected messages are staged through a BigQuery outbox before batched delivery to AWS SQS.
04
Reconciliation
SQS receipts are compared with expected delivery counts so missing or failed sends are visible.
Project Overview
Built a production multi-pipeline data platform that synchronizes Google DV360 advertising data from Google Cloud to AWS. The system includes metadata sync with parent-before-child ordering, report table auto-detection and normalization, and Ads Data Hub match-rate orchestration. Delivery uses a BigQuery outbox pattern, SQS batch sends, receipt tracking, and reconciliation so cross-cloud movement can be monitored instead of treated as a black box.
Key Challenges
- DV360 campaign entities needed dependency-aware delivery so parent objects reached downstream systems before child entities
- DV360 report tables arrived asynchronously in BigQuery and needed automatic detection and normalization
- Ads Data Hub workflows required long-running async orchestration with privacy-aware failure handling
- Cross-cloud delivery to AWS SQS needed receipt tracking and reconciliation to prevent silent message loss
Results & Impact
- Built three independent event-driven pipelines with source-specific trigger patterns
- Implemented tiered DV360 entity delivery for parent-before-child dependencies
- Normalized multiple DV360 report formats from BigQuery landing tables
- Added receipt tracking and expected-vs-actual reconciliation for cross-cloud delivery
Technology Stack
Project Details
Industry:Marketing Data Engineering
Duration:5 months
Team Size:Solo
Completed:January 2026
Tags
data-engineeringbigquerygoogle-cloudcloud-runcloud-taskscloud-workflowseventarcads-data-hubdv360aws-sqspython
Have a similar data workflow?
If your reporting process depends on APIs, spreadsheets, ad platforms, or asynchronous exports, I can help turn it into a reliable pipeline with validation, monitoring, and clean outputs.