Home

Soujanya G - Data Engineer

[email protected]

Location: Cumming, Georgia, USA

Relocation: In Georgia State (Remote)

Visa: GC EAD (+1 470 253 1144)

8 years of data engineering experience creating, constructing, and improving scalable ETL/ELT pipelines and cloud-based data solutions for financial and enterprise clients. Worked with AtScale to build scalable semantic data models for enterprise BI and analytics reporting. Designed and maintained Snowflake and Delta Lake data warehouses, including schema design, analytical data modeling, query optimization, and performance tuning for large datasets. Designed and built scalable real-time and batch data pipelines integrating Spark (PySpark/Scala), Kafka, and Databricks, enabling near real-time processing and advanced analytics. Developed and deployed MongoDB data models for semi-structured and unstructured datasets, ensuring optimal schema design for performance and scalability. Designed and implemented distributed data processing solutions using HDFS, YARN, Hive, and HBase for large-scale data workloads. Applied data quality, governance, and metadata management practices using tools like Ab Initio Metadata Hub, Precisely, and Openflow to ensure lineage, compliance, and enterprise data integrity. Experienced in multi-cloud environments with AWS (Glue, Lambda, Step Functions, S3, EMR) and GCP fundamentals, enabling seamless integration with Azure and Snowflake-based architectures. Developed and optimized data transformations using SQL, Python, PySpark, and ELT tools such as dbt for efficient analytical processing. Implemented robust data security and compliance frameworks, including PII/HIPAA standards, encryption, and IAM policies across cloud platforms. Actively participated in application design discussions, translating business requirements into scalable data solutions and well-structured technical implementations. Conducted code reviews, enforced best practices, and contributed to maintaining high-quality, maintainable, and reliable data pipelines. Implemented testing and QA strategies, including data validation, unit testing, and pipeline monitoring, to ensure accuracy, reliability, and production stability. Developed and optimized OLAP-style data models in AtScale to support self-service analytics across large datasets in Snowflake, BigQuery, and Azure Synapse. Created technical documentation and delivered user training and support to ensure smooth adoption of data solutions and effective knowledge transfer across teams. Developed scalable data processing pipelines using Spark (PySpark & Scala). Designed and developed scalable data pipelines using Google Cloud Platform (GCP) services including Dataflow and Composer for batch and real-time processing. Delivered BI and analytics solutions utilizing Power BI, Excel, and Tableau, including dashboards and reports to meet regulatory, risk, and business reporting needs. Designed and managed workflow orchestration using Apache Airflow DAGs. Integrated Kafka with Spark Streaming for near real-time data processing. Strong collaboration with Data Architects, Business Analysts, and DevOps teams, using Agile/Scrum approaches to create scalable, automated, and maintainable data pipelines. Optimized NoSQL schemas for low-latency and high-throughput applications. Integrated Python scripts with cloud services, APIs, and orchestration tools. Implemented REST APIs and data services supporting ETL pipelines. Built SQL and Python-based analytical datasets supporting transaction monitoring and financial crime detection models, enabling risk teams to identify suspicious transaction patterns across banking transactions. Supported quantitative risk and monitoring models by preparing large-scale transactional datasets, improving analytics performance for AML compliance and regulatory reporting. Keywords: quality analyst business intelligence sthree