Snowflake Data Warehouse Integration

Developed a robust data integration pipeline to automate the nightly transfer of critical accounting and project data from a Snowflake data warehouse to a customer relationship management (CRM) system.

Key Achievements

• Seamless data extraction: Implemented a scheduled job to efficiently extract account and project data from Snowflake using the platform's REST API, ensuring accurate and up-to-date information transfer.
• Scalable job chaining: Designed a system of queueable jobs to handle large datasets, allowing for efficient processing and seamless integration with the target CRM system.
• Configurable logging: Built a customizable logging mechanism to track the success and errors of each record transfer, providing detailed insights into the pipeline's performance.
• Automated reporting: Implemented an email-based reporting feature that delivers a comprehensive CSV report to stakeholders, summarizing the results of each nightly data transfer.