Aggregator affiliate marketing platform connecting retail business to end customers through interesting cashback

Our client has helped it s members — 10 million+ — earn over $1 billion in Cash Back at their favorite stores. By connecting these savvy shoppers with America’s best brands, the platform gives them a hassle-free way to save money on the things they buy every day, while the retail partners find loyal new customers and drive record sales.

Our client thrived on the understanding of their customer behavior through data gathered in shopping trails through the app, website, toolbar and referral emails. In the evolving world of data engineering and science, the need to adapt to an optimal data life cycle management was the absolute need of the hour.

This is where MX Techies came in. Our team worked as a solution provider alongside with the client’s Engineering team translating the old architecture progressively to a highly scalable model.

Creating the Data Lake & Migration To Cloud

Our client has their presence in USA, Canada, Japan, Korea as the key geographies with data like shopping trips(customer data) , most clicks, geographic interest, etc. scattered in Postgres, HDFS and Amazon S3. Scheduled jobs were written in shell scripts that were wrapped in PERL to ingest this data through Kafka Streams. This dat a went through intelligent ETL through Apache Spark and found it s residence in the Snowflake data lake.

The current architecture that is on on-premise HDFS using Cloudera, is being migrated to AWS with the above Snowflake setup being transformed into Microservice architecture causing the service to pull dat a from different sources.

Real Time Actionable Intelligence

Once the data was in the data lake, the multiple pipelines moved data from ‘raw’ state to ‘refined’ and finally to ‘production’ ready in parquet HDFS Datawarehouse based on few metrics. Amazon EMR transient clusters were implemented. The visualization and BI tool adopted was Tableau.

The key insights gained were Customer Behavior, Key Events of engagement and Event Diary & achieved in real- time. This enabled our client to participate in deals like amazon prime day and monitor the real-time trends and behavior live.

Benefits