SQL Interviews in the Age of LLMs: Patterns Over Queries
Stop memorizing queries. Learn to recognize the 12 problem shapes that keep showing up.
Topic
dataengineering
A collection of 15 issues
Three Formats Walk into a Lakehouse: Iceberg, Delta and Hudi in a Local Setup You Can Run on Your Laptop
I still remember building an entire bookkeeping system in FoxPro for my university coursework back in 2007. Planning every table structure upfront, carefully designing indexes because adding them later meant locking everything up for a reindex, managing backups manually, worrying about file corruption. Every schema change was a small crisis.
When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)
It all began with a fairly normal data pipeline, the kind I've built dozens of times over the years working with event-driven systems on AWS. Events were coming in through Kafka, landing in AWS S3 as Parquet files after going through some lightweight microbatch processing. It looked
Scaling Data Analytics: Building a Starter Kit with Neon, Airflow, and Streamlit
Setting up a data analytics project from scratch can be a headache. You need a database, a way to pull in data, and a dashboard to make sense of it all. I wanted to make that process easier—something lightweight, flexible, and beginner-friendly.
That’s how I ended up
Handling Dates in Argo Workflows
How to format, shift and convert dates in Argo Workflows with the Sprig template library
Logs Don’t Lie: Debugging My Data Engineering Crisis in 2025
Every data pipeline has its breaking point. Mine came in late 2024, throwing errors I couldn’t ignore. Logs showed signs of stagnation, over-processing, and the need for a refreshed perspective.
But here’s the thing: I don’t really have anything to complain about. I hold a classical
12 Steps to Organize and Maintain Your Python Codebase for Beginners
The reason why I’m writing this post is to share some insights on keeping a project clean, even with lots of contributors. This is especially important for data engineers, given the ever-changing nature of data and the processing demands in Python libraries and applications.
The title might sound
The Data Engineering Docker-Compose Starter Kit
We’ve all been there. Starting a new data engineering project and, instead of diving right into the fun stuff, we’re stuck setting up configurations. Again.
So, I’ve put together a Docker Compose config with a bunch of services you’ll often see in data engineering projects. Just
The Golang Saga: A Coder’s Journey There and Back Again. Part 3: The Graphing Conundrum
Welcome back to the third part of “The Golang Saga: A Coder’s Journey There and Back Again.” In the first part of this series, I started work on a personal project, the Climate Change Visualizer, using Go as my chosen programming language. In the second part, we delved into
The Golang Saga: A Coder’s Journey There and Back Again. Part 2: The Data Expedition
Welcome back to the second part of “The Golang Saga: A Coder’s Journey There and Back Again.” In the first part of this series, I started working on a personal project, the Climate Change Visualizer, using Golang as my chosen programming language.
Just a quick recap, I had already