📊 Day 09 : Loops

🎯 Enterprise Objective

Iteration is the engine of data processing. Today we graduate from basic loops to professional iteration patterns. You will learn to write memory-efficient pipelines using generators and the itertools module, allowing you to process gigabytes of data seamlessly without crashing your system.

📋 Strategic Overview

A for loop in Python iterates over an iterable (like a list, string, or dictionary). Unlike C-style loops that use index counters, Python's for loop extracts the actual items directly. Under the hood, it calls iter() to get an iterator, and next() until it hits a StopIteration error.

• Data Processing: Iterating over rows in a dataset or JSON objects in an API response

• File Parsing: Reading gigabytes of log files line-by-line without loading them into memory

Never modify a list (adding/removing items) while iterating over it with a for loop. It will skip elements or crash. Iterate over a copy instead: for item in my_list.copy():.

A while loop runs as long as a condition evaluates to True. It is used when the number of iterations is unknown beforehand (e.g., waiting for an API response, polling a database, or reading user input until they type 'quit').

# Polling pattern
while not db.is_ready():
    time.sleep(1)
    print("Waiting...")

• API Pagination: Fetching pages of data until next_page_token is null

• Retry Logic: Attempting to connect to a server up to 5 times before failing

• Simulation: Running a simulation until a specific convergence threshold is met

Always ensure the condition inside a while loop will eventually become False, or provide a break statement. Otherwise, your program will freeze forever.

Python provides keywords to alter loop flow. break exits the loop entirely. continue skips the rest of the current iteration and jumps to the next. Python also has a unique for...else construct where the else block runs only if the loop completes without hitting a break.

• Data Filtering: Use continue to skip invalid rows without deeply nesting if statements

• Search Algorithms: Use break to stop searching a massive dataset once the target is found

• Validation: Use for...else to verify that ALL items pass a check

The for...else naming is confusing. Think of it as for...no_break. It's the most Pythonic way to perform a 'search and report if not found' pattern.

Python emphasizes readability. Instead of manually tracking indices with range(len(data)), Python provides elegant built-in functions: enumerate() (value + index), zip() (parallel iteration), and reversed() (backwards).

• Feature Engineering: Combining multiple columns using zip(col1, col2)

• Logging: Printing row numbers alongside data errors using enumerate(data, start=1)

A generator is a special function that returns an iterator. Instead of computing an entire list and storing it in memory, it uses the yield keyword to produce values one at a time. It pauses its state between yields.

# Normal Function (Eats RAM)
def get_millions():
    return [x for x in range(1_000_000)] # list in memory

# Generator (O(1) Memory)
def yield_millions():
    for x in range(1_000_000):
        yield x  # yields one by one

• Big Data Processing: Reading a 50GB CSV file line-by-line without crashing your laptop

• Infinite Streams: Processing real-time sensor data or Kafka event streams

• API Pagination: Yielding records page-by-page as they are requested

Once a generator is exhausted (loop finishes), it is empty. You cannot iterate over it a second time. You must create a new generator object.

A generator expression is exactly like a list comprehension, but it uses parentheses () instead of square brackets []. It creates a generator object instead of a list, saving massive amounts of memory.

• Math Reductions: sum(x**2 for x in data) — computes the sum without creating a temporary list

• Pipelining: Chain multiple generator expressions to create a clean, memory-efficient data pipeline

When passing a generator expression as the ONLY argument to a function, you can omit the extra parentheses: sum(x2 for x in range(10)) instead of sum((x2 for x in range(10))).

The itertools module is a standard library toolkit for creating and combining iterators. It provides fast, memory-efficient tools for combinatorial math, grouping, and advanced looping patterns.

Key Itertools:

• Feature Interactions: Generating all pairs of columns using combinations for machine learning

• Flattening Data: Using chain.from_iterable(list_of_lists) to flatten nested structures efficiently

Tools like cycle, count, and repeat generate infinite streams. Always use a break condition or zip them with a finite sequence, otherwise your program will hang.

Theory is useless without muscle memory. Complete these tasks to solidify your understanding.

✅ Instructor's End-of-Day Checklist

• [ ] I understand how for loops work under the hood (iter() and next()).

• [ ] I know how to use while loops for unknown iteration counts (polling).

• [ ] I can use continue to filter and for...else for search operations.

• [ ] I always use enumerate() instead of range(len()).

• [ ] I understand how yield saves memory compared to returning a full list.

• [ ] I can write a generator expression for memory-efficient math reductions.

• [ ] I have completed all 5 practice tasks.

• [ ] I have reviewed all 25 interview questions.

*Next Up: Day 10 - Functions: Scope, Arguments, \args/\\kwargs, and Lambdas**