Azure Databricks Project Information

This project focuses on creating a scalable and secure data pipeline to ingest, store, process, and analyze traffic data using Azure Data Lake Storage (ADLS), Azure Databricks, and Unity Catalog. Data is ingested through Azure Data Factory, stored in ADLS, and processed in Databricks to transform raw traffic data into valuable insights. Unity Catalog ensures centralized governance and access control, while a CI/CD pipeline facilitates continuous integration and deployment of Databricks notebooks, enabling efficient management and monitoring of traffic data.