Jesse Pepple Portfolio

This Data Engineering project demonstrates the full potential of Azure Databricks , Azure Data Factory (ADF) , and Azure Data Lake , showcasing how modern data solutions can be built with scalability and automation in mind. The project incorporates key engineering practices such as Slowly Changing Dimensions (SCD) Type 1  — implemented manually — and Type 2 , automated through Delta Live Tables (DLT) , alongside Star Schema modeling  and incremental data loading . Using Spark

This project focused on building an end-to-end Azure & Databricks data pipeline  using the Olympics 2024 dataset . I began by provisioning the required Azure resources and designing the solution around a Medallion Architecture (Bronze → Silver → Gold)  to ensure scalability, data quality, and reliability. Data was ingested using Azure Data Factory (ADF)  and stored in Azure Data Lake Storage , establishing the raw Bronze layer. From there, I applied data governance through Un

In this project, I implemented Unity Catalog in Databricks  to establish secure, centralized data governance , ensuring controlled access and compliance across all datasets. The solution leveraged Delta Lake and Delta Tables  to provide scalable, reliable, and efficient storage , while the pipeline followed the Medallion Architecture (Bronze → Silver → Gold)  to maintain data quality, reliability, and usability  at every stage. The end-to-end workflow utilized Azure Data Fact