Junior knows the syntax, middle knows what breaks in production, senior knows what not to build. A candidate who writes a five-line happy-path parser and says «done» hasn’t touched real CSV files. Bonus points if you mention that production systems should schema-validate before flattening, not after. The defaultdict pattern here is the same logic under the hood of GROUP BY. Sliding window isn’t just an algorithm pattern; it’s how you monitor pipeline throughput, detect burst traffic, and build alerting logic.
The plan on this page is 10 weeks and assumes you can already write a loop and a JOIN. A team running modern dbt on Snowflake mostly focuses on SQL and modeling and won’t grill you on Spark internals. Open the file with csv.DictReader, iterate, yield transformed rows one at a time, and let the caller decide what to do with them. If it’s a fixed-size attribute like a list of 3 coordinates, you might serialize or split into positional columns.
- A leaf is any non-list value, so a string stays whole rather than being split into its characters, and empty sub-lists contribute nothing.
- Interviewers plant this in code-review prompts because it looks correct and fails statefully.
- Small-integer and string interning make is appear to work on values in quick experiments and then fail in production, which is exactly why the question gets asked.
- Idempotency means that executing the same ETL job multiple times produces the same final result without creating duplicate or inconsistent records.
- This means you get Python’s readability and ecosystem while still running jobs on Spark’s distributed engine.
- Create a script to parse and transform a JSON file into a structured CSV.
Or dict plus doubly linked list if they want the internals. This is the merge step of external sort and exactly how log shards and sorted partition files get combined. A generator computes values on demand and https://uvik.io/ holds one at a time; a list holds everything.
How have you used Python’s built-in data types in a real-world project?
The GIL is a mutex (a lock) in CPython that allows only one thread to execute Python bytecode at a time, even on multi-core machines. This question is one of the clearest ways interviewers gauge whether you can write code that handles real-world data volumes. You create one with the yield keyword or a generator expression. A generator produces items one at a time, on demand, rather than loading everything into memory at once. How you approach error handling tells interviewers a lot about whether you’ve worked with production code before. Getting this wrong causes unpredictable bugs that are painful to trace.
Question 4: Window and Session Logic Without SQL
The follow-up they askA bug corrupted silver for the last 30 days. OBT (pre-joined wide tables) wins for dashboard latency, ML feature consumption, and column-store engines that make width nearly free to scan. The follow-up they askThe org chart is SCD2 on top of being a hierarchy. Trade write-time explosion for read-time simplicity; in a read-heavy warehouse that trade is usually right. Parent-child (adjacency) is natural but needs recursion per query. The follow-up they askWhen does a junk dimension’s combination count blow up, and what do you do then?
Counter.most_common(k) uses a heap internally, giving you O(n + k log n) without building one yourself. Quarantining bad rows instead of crashing is the single most senior-signaling habit you can show. Somewhere around 20+ loops in a single job search.