Problem

Business needed an automated way to ingest data from multiple different sources and store them all in a database. They then needed a way to scrub and store in a SQL database.

Solution

We created an Azure cloud solution that had multiple Azure data factory jobs, combined with Python Functions that aggregated data and then stored it in a SQL database. Once it was in the database, the data was scrubbed using SQL stored procedures and moved into the tables where it could be reported from.