The Client is a global company that helps global companies to run effective cross-channel, dynamic content marketing campaigns. The campaigns use different popular media channels and platforms to deliver the content. The client runs campaigns worldwide and has offices around the world.
Multi-channel, distributed marketing campaigns are always a challenge. It is necessary to coordinate and verify content published by multiple offices and have insights regarding the performance they deliver. Moreover, it is clear, that data by itself has a huge value and must be accumulated for further processing.
Challenges by names:
- Deliver solution to aggregate published ads from different media publishing platforms (Facebook, TikTok, Sizmek, Instagram, etc.)
- Build the Data Lake as a foundation for further processing
- Standardize the content across the channels so that cross-channel and cross-platform analysis is possible
- Implement in a cost-effective and highly scalable way
- successfully delivered Proof-of-Concept (PoC) phase to demonstrate the potential of the solution
- Minimum Viable Product (MVP) phase – development is in progress
The overall project timeline:
- PoC phase – 3 weeks
- MVP phase – in progress
- The overall architecture for the eco-system of aggregators, Data Lake, and Data as a Product has been designed
- Full processing flow has been implemented to integrate specific media platform to process one of the ad types as a demo of the approach
- The prototype has been successfully deployed into AWS for pilot operations
SUCCESS STORY IN DETAILS
Geo-distributed marketing campaigns are a real pain for big companies because there are too many moving parts. So, the only real “source of truth” of end result is ads by themselves. The location of ads could be very different, and the format of ads could be different, but their number is limited. The objective is to be able to aggregate those ads and standardize them so that cross-channel analysis is possible. Derived insights generation could be the next goal.
The client has clearly articulated the priorities of media platforms to be integrated and formats of ads to be collected and analyzed.
The challenge was to design the system in such a way that further extension could be done without the necessity to rebuild and redeploy the whole system. Plus, to find a cost-effective way to process huge volumes of info with proper performance requirements.
The team has proposed a microservice-based solution to be hosted in the cloud. The idea was to build the solution around the Data Lake to be built around the AWS eco-system with an ability to develop and deploy the aggregators and analysis tool in Cloud-agnostic and programming language agnostic ways.
What has been done:
- Careful review and analysis of existing alternatives
- High-Level Architecture (HLA) design for the whole eco-system of data aggregation, data storage, and data processing pipelines
- The whole pipeline of data aggregation, data storage, data processing, and exposure as Data as a Product through standardized API supporting SQL-like queries for data analysis
- A pure serverless solution to address high scalability and cost-effectiveness requirements
- Complete Continuous Integration and Continuous Delivery (CICD) processes to ensure the highest possible Time-To-Market delivery speed
- Successful deployment to Pre-Prod environment for pilot operations
- Out-of-the-box solutions from popular cloud solutions (Azure, AWS) can provide cost-effective and scalable solutions
- Proper design can avoid cloud lock and provide the way for cloud-agnostic implementation
- Selecting mature mainstream AWS technologies (such as AWS Service Catalog) brings benefits immediately in reduction of maintainability costs.
- “Cloud-agnostic” should be carefully discussed up-front