I’m a Senior Software Engineer with 3.5 years of experience in Big Data stack, passionate about building products that have the potential of benefitting millions.
⦿ I am working as a Senior Data Engineer in the Data Science and Engineering team.
⦿ My key responsibilities include designing and implementing optimised data flow pipelines at scale.
⦿ Refactored and optimised the low code wrapper project making it easier to use and reducing run times by 100x.
⦿ I worked as a Data Engineer in the Data Platform team.
⦿ My key responsibilities included designing and implementing the data flow pipelines at scale in multiple areas while mentoring the interns.
⦿ I designed and implemented change data capture-based pipelines improving the data quality by 10% and delay by 30%.
⦿ I helped reduce the S3 storage and read/write costs by up to 40% by optimizing Apache Spark and Delta Lake.
⦿ I led the schema registry project which gave one view of 90% of the tables' schema present in the data warehouse and different source databases.
⦿ I created a streaming pipeline to make data warehouse GDPR and CPRA compliant.
⦿ My key responsibilities included designing, implementing and managing the data flow pipelines, raw data ingestion and facts creation.
⦿ I worked on optimizing the raw data ingestion pipelines improving the ingestion time by 50%.
⦿ I implemented encryption-decryption of 100% of the PII data present in the data warehouse improving the security and lowering the risk of a data breach.
⦿ I created a validation script to point out the mismatches in data at different places affecting more than 33% of the total users.
⦿ I created a library for sending alerts of all our jobs which resulted in 20% less code repetition and the ability to capture failures easily.
⦿ I worked as a Data Engineer in the Data Sharing and Ingestion team.
⦿ My key responsibilities included designing, implementing and managing the data flow pipelines, ingestion modules and metadata store.
⦿ I designed and implemented a unified data pipeline to ingest 60+ datasets accounting for more than 30% of the total load.
⦿ I was able to design data flow and implement validations or transformations for datasets having as many as 1.5 million records.
⦿ I worked (along with a couple of team members) on the automation of the creation of rules which reduced the manual intervention and effort by 80%.
⦿ I improved the efficiency of the system by up to 15% by creating a wrapper/generic script that reduces the repetition of code and effort to perform similar tasks.
⦿ I created academic content for 15 units of fundamental subjects including Mathematics, Operating System and more.
⦿ I participated in weekly meetings and collaborated with the mentor and delivered appreciable content.
8.59 / 10
88.4%
8.6 / 10
Social Profiles