The Ultimate DSA Master Course (Python)

1.1 What is an Algorithm

Introduction

Before you write a single line of code for any problem, you need a clear, step-by-step plan. That plan—when it is precise, unambiguous, and finite—is what we call an algorithm. This is the first and most important idea in the entire DSA course: every correct program is the implementation of some algorithm. If you understand what an algorithm is and what makes one good or bad, you have the foundation for everything that follows.

Real-World Analogy

Think of an algorithm like a recipe. A recipe has:

• Inputs: ingredients (flour, eggs, sugar)
• Steps: mix, bake at 350°F for 30 minutes
• Output: a cake

If the steps are vague (“add some flour” or “bake until it looks done”), two people can get different results. If the steps are exact (“add 2 cups flour,” “bake 30 minutes”), anyone who follows them gets the same cake. An algorithm is the same: it takes defined inputs, follows a fixed set of clear steps, and produces a defined output. The more precise the steps, the more reliable the result.

Formal Definition

In computer science, we define an algorithm as:

Breaking that down:

• Finite: The list of steps has an end. It doesn’t run forever.
• Well-defined: Each step is clear. There’s no “do something useful” or “maybe do this.”
• Unambiguous: Only one interpretation. No “sometimes do A, sometimes B” unless we explicitly say under what condition.
• Inputs: We know what we’re given (e.g., a list of numbers, a string).
• Outputs: We know what we must produce (e.g., the maximum, a sorted list, yes/no).
• Terminates: The process stops. We don’t require infinite time or infinite memory.

Why This Topic Matters

Interviews and real-world design both revolve around algorithms. Interviewers ask you to “design an algorithm” for a problem—they want to see that you can break a task into clear, correct steps before coding. In production, the algorithm you choose decides whether your system scales (e.g., O(n log n) vs O(n²)) or fails under load. Understanding what an algorithm is—and what makes one better than another—is the basis for the rest of this course.

Mental Model

Picture an algorithm as a black box with a contract:

  ┌─────────────────────────────────────┐
  │  INPUT(s)                            │
  │  e.g. array [3, 1, 4, 1, 5]          │
  └──────────────┬──────────────────────┘
                 │
                 ▼
  ┌─────────────────────────────────────┐
  │  ALGORITHM (sequence of steps)       │
  │  • Step 1: ...                       │
  │  • Step 2: ...                       │
  │  • ...                               │
  └──────────────┬──────────────────────┘
                 │
                 ▼
  ┌─────────────────────────────────────┐
  │  OUTPUT(s)                           │
  │  e.g. maximum value 5                │
  └─────────────────────────────────────┘
        

You don’t have to care how the box is implemented (which language, which data structures) when you’re defining the algorithm. You only care: given these inputs, these steps produce this output. Implementation comes later.

Properties of a Good Algorithm

Not every set of steps is a “good” algorithm. We usually require:

Step-by-Step: Describing an Algorithm

When you explain an algorithm, you typically:

1. State the problem: What are we given, and what must we return?
2. Assume valid input: (Or state what you assume, e.g., “array is non-empty.”)
3. List steps in order: Numbered, clear, one idea per step.
4. Handle edge cases: Empty input, single element, duplicates—whatever the problem allows.

You can describe the same algorithm in plain English, pseudocode, or code. The algorithm is the idea; the code is one way to run it.

Example: Find the Maximum in a List

Problem: Given a list of numbers, return the largest number.

Input: A list arr of numbers (we’ll assume at least one element).

Output: The maximum value in arr.

Algorithm (steps):

1. Assume the first element is the maximum; call it max_so_far.
2. For each remaining element in the list: if that element is greater than max_so_far, update max_so_far to this element.
3. When all elements have been checked, return max_so_far.

This is finite (we do one pass), well-defined (each step is clear), and effective. It’s an algorithm.

Python Implementation

def find_max(arr):
    if not arr:
        return None  # or raise, depending on contract
    max_so_far = arr[0]
    for i in range(1, len(arr)):
        if arr[i] > max_so_far:
            max_so_far = arr[i]
    return max_so_far

The code above is one implementation of that algorithm. The algorithm itself is the three-step idea; Python is just the language we used to run it.

Algorithm vs Program

• Algorithm: Language-independent, step-by-step procedure. It’s the “what to do” and “in what order.”
• Program: An implementation of one or more algorithms in a specific language (Python, Java, etc.). It’s the “how it runs on a machine.”

One algorithm can be implemented in many languages; the core logic stays the same. When we analyze “time complexity” or “space complexity” later, we’re analyzing the algorithm, not the quirks of a single language.

Summary

• An algorithm is a finite, well-defined, unambiguous sequence of steps that maps inputs to outputs and terminates.
• Good algorithms are finite, definite, have clear input/output, and are effective.
• You can describe an algorithm in words, pseudocode, or code; the algorithm is the idea, the code is one implementation.
• Thinking in algorithms first (then implementing) is the foundation of DSA and of strong technical interviews.

1.2 What is a Data Structure

Introduction

An algorithm tells you what steps to follow. But those steps operate on something—numbers, names, relationships. How you organize and store that “something” so that your algorithm can work efficiently is the job of a data structure. Algorithms and data structures are inseparable: the right structure makes the algorithm simple and fast; the wrong one makes it clumsy or slow.

Real-World Analogy

Imagine you need to find a book in a library.

• Stack of unsorted books: You look one by one until you find it. Slow to search, but easy to add a new book (just put it on top).
• Books sorted by title on a shelf: You can jump to the right section (like binary search). Fast to search, but inserting a new book may require shifting others.
• Index cards (catalog): You look up “Author name” and get the shelf location. Very fast lookup if you know the key.

Same “data” (books), different ways of organizing it—each with different tradeoffs for “find a book,” “add a book,” “remove a book.” A data structure is exactly that: a chosen way to organize and store data so that the operations we care about (search, insert, delete, etc.) are as efficient as we need.

Formal Definition

“Efficiently” depends on the problem: sometimes we need fast lookup by key (dictionary); sometimes we need order (sorted list); sometimes we need fast insert at one end (list, queue). The data structure is the organization; the algorithm is the procedure that uses it.

Why This Topic Matters

In interviews and in production, the first question after “what’s the algorithm?” is often “what data structure will you use?” Picking an array vs a hash set can change the time complexity from O(n²) to O(n). Trees, graphs, and heaps exist because certain problems are naturally expressed and solved with those shapes. You don’t just “use a list for everything”—you choose a structure that matches the operations and constraints of the problem.

Mental Model

Every data structure gives you two things:

1. Storage layout: How the data is arranged in memory (contiguous array, linked nodes, key–value buckets, tree, graph).
2. Operation set: What you can do with it—and how fast. For example: “add at end O(1),” “search by value O(n),” “look up by key O(1) average.”

  ALGORITHM (steps)     +     DATA STRUCTURE (storage + operations)     =     PROGRAM
  "How to solve"               "Where to put data & how to access"           Working code
        

Change the data structure and the same high-level algorithm might become faster or simpler; change the algorithm and you might need a different structure. They are designed together.

Algorithm and Data Structure: A Partnership

An algorithm assumes it can do certain things with the data: “get the next element,” “check if this key exists,” “remove the smallest.” The data structure provides those operations. If your algorithm needs “find minimum quickly,” you might choose a heap; if it needs “check membership quickly,” you might choose a set. So:

• Algorithm = what steps to perform.
• Data structure = how data is organized so those steps are efficient.

You’ll often hear “hash map” or “two pointers” or “binary search”—the first is a data structure, the others are algorithmic ideas. In practice we combine them: e.g., “use a hash map and one pass” (structure + algorithm).

Types of Data Structures (High-Level)

We don’t need to memorize a catalog yet. It’s enough to see the landscape:

• Linear: Data in a line—arrays, linked lists, stacks, queues. Order matters; access by position or by scanning.
• Tree: Hierarchical—binary trees, BSTs, heaps. Good for “smallest/largest,” “split by range,” “parent–child” relationships.
• Graph: Nodes and edges—networks, dependencies, maps. Good for “paths,” “connected components,” “shortest route.”
• Hash-based: Store by key; access by key in (average) constant time. Dictionaries, sets.

Later sections of the course go deep on each. Here the takeaway is: different shapes support different operations and tradeoffs.

Same Data, Different Structure

Suppose you must support: “add an element at the front” and “traverse all elements.”

• Array (list): Insert at front is expensive—you shift every element one position. O(n). Traverse is O(n).
• Singly linked list: Insert at front is just “create a node and point the head to it.” O(1). Traverse is still O(n).

So for “insert at front” often, a linked list can be a better choice than an array. The data is the same (a sequence of items); the structure decides the cost of each operation.

Choosing a Data Structure

Ask:

1. What operations do I need? (insert, delete, search, get min, range query, etc.)
2. How often is each used? (e.g., search often → consider hash or sorted structure)
3. What are the constraints? (memory, order, duplicates allowed?)

Then match the structure to the operations. This “operation-first” thinking is how experienced engineers pick structures in interviews.

Summary

• A data structure is a way to store and organize data so that the operations you need (access, insert, delete, search) can be done efficiently.
• Algorithms and data structures work together: the algorithm defines the steps; the structure defines how data is stored and what operations cost.
• Different structures (linear, tree, graph, hash) offer different tradeoffs; choose based on the operations and constraints of the problem.

1.3 How to Think Like a Problem Solver

Introduction

Solving a problem in an interview or in real code isn’t about memorizing solutions—it’s about a repeatable way of thinking. This section gives you a framework: what to do from the moment you read the problem until you have a clear, implementable plan. Master this and you’ll handle new problems you’ve never seen before.

Why a Framework Matters

Without a process, it’s easy to jump into code, miss edge cases, or get stuck. With a process, you: clarify the problem, test your understanding with examples, get a simple solution first, then improve. That’s how strong problem solvers work—and how you should too.

The Problem-Solving Mindset

• Clarify before solving: Make sure you understand inputs, outputs, and rules. Ask “what if” (empty input? duplicates? negative numbers?).
• Start simple: Get a correct solution first, even if it’s slow. Correctness is the baseline; optimization comes next.
• Use examples: Work 2–3 small examples by hand. If your approach works on paper, it’s easier to translate to code.
• Break it down: If the problem is big, solve a smaller version or one part first.

Step-by-Step Framework

Follow these steps every time. They work for interviews and for practice.

Step 1: Read and Restate

Read the problem fully. Then restate it in your own words: “So I’m given X, and I need to return Y, with these rules: …” If you can’t restate it, you don’t understand it yet. Ask clarifying questions (in an interview) or re-read (in practice).

Step 2: Identify Inputs and Output

Write them down explicitly. Example: “Input: array of integers, length n. Output: indices of two numbers that add up to target, or [-1, -1] if none.” Knowing I/O prevents you from solving the wrong problem.

Step 3: Work Small Examples and Edge Cases

Pick 2–3 small examples and solve them by hand. Then think of edge cases:

• Empty input, single element, two elements
• No valid answer (e.g., no pair sums to target)
• Duplicates, negatives, zeros
• Maximum size (what if n is huge?)

If your approach breaks on an edge case, fix the approach before coding.

Step 4: Brute Force First

Describe the simplest solution that definitely works—even if it’s “try every pair” or “check every possibility.” Say it out loud or write 3–5 steps. This gives you a correctness baseline and often reveals a path to a better solution.

Step 5: Optimize (If Needed)

Look for repeated work, unnecessary loops, or a better data structure. Can you use a hash map to avoid a second loop? Can you sort and use two pointers? We’ll build a toolkit of patterns; for now, the habit is: “I have a working solution; where is the waste?”

Step 6: Write the Code

Only after steps 1–5, translate your algorithm to code. You’re implementing a plan, not exploring in the dark. Test with the examples and edge cases you already thought through.

  READ → RESTATE → I/O → EXAMPLES & EDGE CASES → BRUTE FORCE → OPTIMIZE → CODE → TEST
        

Restate in Your Own Words

This is the single most underused habit. Before writing a single line, say: “I need to … given … and return … when …” If you do this, you’ll catch misunderstandings early and your code will match the problem.

Work Examples by Hand

Take a tiny input (e.g., array [2, 7, 11], target 9). Walk through your intended steps. Do you get [0, 1]? If yes, your logic is clear. If no, fix the logic. This “hand trace” is how you avoid bugs and how you explain your approach in an interview.

Summary

• Use a framework: read → restate → I/O → examples & edge cases → brute force → optimize → code.
• Clarify the problem and restate it; identify inputs and output explicitly.
• Work small examples and edge cases by hand before coding.
• Get a correct (brute force) solution first, then look for optimization.

1.4 Brute Force Approach

Introduction

The brute force approach is the straightforward solution that “just tries everything” or does the most obvious thing—no clever tricks yet. It’s often slow for large inputs, but it’s simple, easy to get right, and gives you a correct baseline. In problem-solving and in interviews, you should start with brute force, then optimize only when needed.

What Is Brute Force?

Examples:

• Find two numbers that sum to target: Check every pair. Two nested loops. Slow (O(n²)) but obviously correct.
• Find maximum in an array: Scan every element and keep the largest. One loop. For this problem, that “naive” scan is already optimal—so “brute force” here is also the best solution.
• Find a value in an unsorted list: Linear search—check each element until you find it. Brute force and, for unsorted data, the only option.

So “brute force” doesn’t always mean “bad.” It means “no clever optimization yet.” Sometimes the brute force solution is already good enough.

Why Start With Brute Force?

1. Correctness: You have a solution that works. You can test it, debug it, and verify against examples.
2. Baseline: You know the worst-case behavior (e.g., O(n²)). Optimization then means “do better than this.”
3. Clarity: Simple logic is easier to explain and to implement without bugs.
4. Interview signal: Showing brute force first, then improving, demonstrates structured thinking—better than jumping to a half-remembered “optimal” idea and getting stuck.

Mental Model

Think: “What’s the dumbest correct solution?” That’s brute force. Then ask: “Where am I doing redundant work? Can a different data structure or a different order of operations remove it?” That’s the path from brute force to a better solution.

Example: Two Sum (Evolution)

Problem: Given an array of integers and a target, return indices of two numbers that add up to the target. Assume exactly one valid pair.

Brute Force: Try Every Pair

For each index i, for each index j > i, check if arr[i] + arr[j] == target. If yes, return [i, j].

• Time: Two nested loops over n elements → O(n²).
• Space: O(1) besides the input.

def two_sum_brute(arr, target):
    n = len(arr)
    for i in range(n):
        for j in range(i + 1, n):
            if arr[i] + arr[j] == target:
                return [i, j]
    return [-1, -1]

This is correct. For small n it’s fine. For large n we want to do better.

Better: One Pass With a Hash Map

As we scan the array, for each value arr[i] we need a partner target - arr[i]. If we’ve already seen that partner at some index j, we can return [j, i]. Store “value → index” in a dictionary.

• Time: One pass, O(n). Each lookup/insert in the dict is O(1) average.
• Space: O(n) for the dictionary.

def two_sum_better(arr, target):
    seen = {}  # value -> index
    for i, x in enumerate(arr):
        need = target - x
        if need in seen:
            return [seen[need], i]
        seen[x] = i
    return [-1, -1]

Same problem, same correctness—faster when n is large. This is the “evolution”: brute force first, then optimize by removing the inner loop with a hash map.

When Is Brute Force Acceptable?

• Small input: If n is tiny (e.g., ≤ 20), O(n²) or even O(2^n) might run in milliseconds. Brute force is fine.
• Quick to code: In a contest or interview, a slow-but-correct solution can be better than no solution or a buggy “clever” one.
• No better known: Some problems (e.g., certain NP-hard cases) don’t have a much better algorithm; brute force (or clever brute force) is what we use.

Evolution: Brute Force → Better → Optimal

Get in the habit of this progression:

1. Brute force: Correct, simple, maybe slow. State it and implement it.
2. Better: Identify the bottleneck (extra loop? repeated work?). Use a better structure or invariant (e.g., hash, two pointers, sort).
3. Optimal: Often “better” is already optimal (e.g., one pass + hash for Two Sum). Sometimes you can prove “we must look at each element at least once” → O(n) is a lower bound.

Don’t skip step 1. It keeps your thinking grounded and your code correct.

Summary

• Brute force = straightforward, try-all or naive solution. Correctness first; no clever optimization yet.
• Always consider brute force first: it’s a correct baseline and often reveals how to optimize.
• Evolution: brute force → find bottleneck → better data structure or algorithm → optimal (if needed).
• For small inputs or when a better solution isn’t obvious, brute force is acceptable and sometimes preferred.

1.5 Optimization Strategy

Introduction

Once you have a correct solution—often brute force—the next step is to ask: where is the time or space being wasted? Optimization isn’t random cleverness; it’s a systematic way to find bottlenecks and remove them. This section gives you a repeatable optimization strategy so you can move from “it works” to “it scales.”

Why Optimize Systematically?

Without a strategy, you might optimize the wrong part (e.g., micro-tune a loop that’s already O(n) while the real cost is an O(n²) nested loop elsewhere). With a strategy, you measure or reason about where the cost is, then attack that first. One bottleneck fixed often yields a bigger win than many small tweaks.

Mental Model

Think of your program as a pipeline: some steps run once, some run inside loops. The total time is dominated by what runs most often or on the largest data. Optimization means: find that hot spot, then either do less work there or do the same work fewer times.

Step-by-Step Optimization Strategy

Step 1: Establish Correctness First

Never optimize broken code. Get a solution that passes your tests and handles edge cases. Optimization changes code; if the baseline is wrong, you’ll either hide bugs or optimize the wrong behavior.

Step 2: Identify the Bottleneck

Ask: “What is the slow part?”

• By counting: Look at loops. A single loop over n → O(n). Two nested loops over n → O(n²). Three nested → O(n³). The deepest or most repeated structure usually dominates.
• By profiling (when you can run code): Use a profiler to see where CPU time is spent. Focus on the top few functions or lines.

In interviews you usually reason by counting. In production, profiling confirms your guess.

Step 3: Ask “What Am I Doing Redundantly?”

Often the bottleneck is repeated work. Examples:

• Repeated search: For each element, scanning the rest of the array to find a match → O(n²). Replace the inner scan with a hash lookup → O(n).
• Repeated computation: Computing the same sum or max over and over in a loop. Compute once, reuse (e.g., prefix sum, sliding window).
• Repeated traversal: Walking the whole list to find one item, many times. One pass with a hash or one sort + linear pass can often replace many passes.

Step 4: Consider the Right Data Structure

The wrong structure forces extra work. Examples:

• “Check if this value exists” in a list → O(n) per check. In a set → O(1) average. Replace list membership with a set when you need many lookups.
• “Get the smallest element” in a list → O(n). In a min-heap → O(log n) for extract-min. Use a heap when you need repeated min/max.
• “Insert at front” in an array → O(n) shift. In a linked list → O(1). Choose the structure that makes your dominant operation cheap.

Step 5: Trade Space for Time (When Appropriate)

Often you can use extra memory to avoid recomputation: hash maps, prefix arrays, caching. If the problem allows O(n) extra space and you can turn O(n²) into O(n), that’s usually a good trade. Don’t over-optimize space when time is the constraint.

Step 6: Re-evaluate After a Change

After each optimization, confirm correctness and re-check the bottleneck. Sometimes the bottleneck moves (e.g., I/O or a different loop). Stop when you meet the required performance or when further optimization isn’t worth the complexity.

  CORRECT solution → FIND bottleneck (loops, repeated work) → REMOVE redundancy
  (data structure / cache / one pass) → VERIFY still correct → REPEAT if needed
        

Common Optimization Patterns

• Hash for lookups: Replace “search for X in the rest of the array” with “is X in this set?” → often O(n²) to O(n).
• Two pointers / sliding window: Replace “for every start, for every end” with one pass where start and end move in one direction → O(n²) to O(n).
• Sort first: Sorted data enables binary search or two-pointer scans. Sort is O(n log n); if that’s cheaper than repeated linear scans, sort once and reuse.
• Prefix sum: Replace “sum from i to j” computed in a loop with precomputed prefix array → O(1) per query after O(n) setup.

When to Stop Optimizing

Optimize until:

• You meet the time/space constraints (e.g., problem says n ≤ 10^5 and your solution is O(n log n)), or
• Further optimization would make the code much harder to read or maintain without a clear need.

Don’t optimize for the sake of it. Correct and clear first; then fast enough.

Summary

• Optimize only after you have a correct solution.
• Find the bottleneck (nested loops, repeated work); attack that first.
• Look for redundancy: repeated search → hash; repeated range computation → prefix sum or sliding window.
• Choose the right data structure so the dominant operation is cheap.
• Trade space for time when it removes a bottleneck.

1.6 Writing Pseudocode

Introduction

Pseudocode is a compact, language-agnostic way to describe an algorithm using a mix of plain English and simple control structures (loops, conditionals). It’s the bridge between “I know what to do” and “here’s the code.” Writing pseudocode first helps you get the logic right before you worry about syntax, and it’s how you’re expected to communicate your approach in interviews.

Why Write Pseudocode?

• Clarify logic: You focus on steps and order, not semicolons or types. Mistakes in logic show up before you write a single line of code.
• Communicate: In interviews, pseudocode lets you explain your algorithm quickly. Interviewers can follow it even if they don’t use your language.
• Plan: It’s an outline. You can refine it (e.g., “here I need a loop”) and only then translate to real code.

What Pseudocode Looks Like

There’s no single standard. The goal is: readable and unambiguous. Use:

• Indentation for blocks (like Python).
• Keywords such as if / else / for / while / return.
• Short names for variables and data (e.g., arr, n, result).
• Plain English for high-level steps when that’s clearer than fake code.

Avoid: language-specific syntax (e.g., list comprehensions or pointer arithmetic) unless they make the idea obvious. Prefer clarity over looking like real code.

Conventions We’ll Use

• Assignment: x = value or result ← value.
• Indexing: arr[i] for the i-th element (0-based unless stated).
• Length: length(arr) or n when we’ve set n = length(arr).
• Loops: for i from 0 to n-1 or for each element x in arr.
• Return: return result.

Example: Two Sum in Pseudocode

Input: Array arr of integers, integer target.

Output: Indices i, j such that arr[i] + arr[j] == target, or “none” if no such pair.

function twoSum(arr, target):
    n = length(arr)
    for i from 0 to n-1:
        for j from i+1 to n-1:
            if arr[i] + arr[j] == target:
                return [i, j]
    return "none"

This is brute force. Now optimized version using a hash map:

function twoSum(arr, target):
    seen = empty map   // value -> index
    for i from 0 to length(arr)-1:
        need = target - arr[i]
        if need is in seen:
            return [seen[need], i]
        put (arr[i], i) in seen
    return "none"

Same problem; the second version makes the “lookup complement” step explicit and shows we only need one pass.

How Much Detail?

Enough that someone could implement it without guessing. Include:

• Loop bounds and what the loop variable means.
• Conditions for if/else and what you return in each case.
• Where you update state (e.g., “add current element to set”).

You can omit: exact type names, error handling, or trivial details (“increment i”).

Summary

• Pseudocode = language-agnostic description of an algorithm using simple control flow and names.
• Use it to clarify logic, communicate in interviews, and plan before coding.
• Keep it readable: indentation, clear loops and conditions, enough detail to implement without guessing.

1.7 Debugging Techniques

Introduction

Debugging is the process of finding and fixing the cause of incorrect behavior—wrong output, crash, or infinite loop. It’s a core skill: even correct algorithms get implemented with off-by-one errors or missed edge cases. This section gives you a systematic way to narrow down where the bug is and fix it quickly.

Mindset

Assume the bug is in your code or assumptions, not the compiler or the problem. Form a hypothesis (“maybe the loop starts at the wrong index”), test it, and adjust. Random changes rarely help; targeted checks do.

Step-by-Step Debugging Process

Step 1: Reproduce the Failure

Get a concrete input that produces the wrong result (or crash). If you can’t reproduce it, you can’t fix it reliably. Use the problem’s examples first; then try small inputs you design (edge cases: empty, one element, two elements).

Step 2: State the Expected vs Actual

Write down: “For input X, I expected Y, but I got Z.” That makes the bug unambiguous. Sometimes in writing this you already spot the mistake (e.g., “I expected the first index to be 0 but I’m returning 1”).

Step 3: Narrow the Location

Find where the program first goes wrong. Methods:

• Print / log: Print key variables at the start of the loop, after conditionals, and before return. Check: “At step 2, is i what I think it is? Is seen correct?”
• Binary search: Comment out the second half of the logic (or test with half the input). If the bug disappears, it’s in the part you removed. Narrow until you’re down to a few lines.
• Rubber duck: Explain the code line by line to yourself or someone else. Often you hear yourself say something wrong (“and here we add 1 … wait, we shouldn’t”).

Step 4: Find the Root Cause

Don’t stop at “it’s in this function.” Identify the exact wrong assumption or wrong operation. Examples: “I used < but the problem says strictly less,” “I’m updating the index after the loop so it’s off by one,” “I’m not handling empty input.” Fix that assumption or line.

Step 5: Fix and Re-test

Make the minimal change that fixes the bug. Re-run the failing case and the examples you had passing. Add a test for the edge case you missed so it doesn’t come back.

Techniques in Practice

Print Debugging

Insert print(...) (or logs) at critical points: loop start (loop variable, key state), after branches, before return. Inspect the output. Remove or comment out prints when done.

# Example: debugging a loop
for i in range(len(arr)):
    print("i:", i, "arr[i]:", arr[i], "need:", target - arr[i])  # temporary
    if (target - arr[i]) in seen:
        return [seen[target - arr[i]], i]
    seen[arr[i]] = i

Use a Debugger

When you can run the code locally, use a debugger: set breakpoints, step line by line, inspect variables. You see state at each step without adding prints. Essential for larger programs.

Check Edge Cases Explicitly

Many bugs are at boundaries: empty input, one element, first/last index. Add a small test for these. If the code fails on “empty list,” the fix is usually at the start of the function (early return or different initialization).

Compare With a Working Example

Trace through your code by hand on the same input that fails. Write down the value of each variable at each step. Where does your trace first differ from what the code actually does? That’s near the bug.

Common Bug Categories

• Off-by-one: Loop runs one too many or too few times; index is 0-based vs 1-based. Check loop bounds and indexing.
• Wrong condition: Used < instead of <=, or forgot to handle equality. Re-read the problem and your condition.
• State not updated: Forgot to add an element to a set or update a variable inside a loop. Ensure every path that should update state does.
• Edge case: Empty input, single element, or “no answer” case not handled. Add explicit checks.

Summary

• Reproduce the failure with a concrete input; state expected vs actual.
• Narrow the bug location (prints, binary search, rubber duck).
• Find the root cause (wrong condition, off-by-one, missing edge case), fix it, then re-test.
• Watch for off-by-one, wrong conditions, missing state updates, and unhandled edge cases.

1.8 Handling Edge Cases

Introduction

An edge case is an input or situation at the “boundary” of what your solution is designed for: empty input, a single element, maximum size, or a “no valid answer” scenario. Code that works on “normal” examples often fails on edge cases. Handling them explicitly is what separates a quick hack from a robust solution—and what interviewers look for.

Why Edge Cases Matter

In production, edge cases are where systems break: empty list causing a crash, or one user when the logic assumed many. In interviews, test cases often include edge cases on purpose. If you don’t handle them, your solution is incomplete. Thinking about edge cases up front also improves your algorithm design: you clarify the problem and avoid bugs before they happen.

What Counts as an Edge Case?

Depends on the problem. Common categories:

Size Boundaries

• Empty: Empty array, empty string, zero elements. Many algorithms assume “at least one”; they crash or return nonsense on empty.
• Single element: One item in the list, one character in the string. Loops that assume “first and rest” or “pair” can break.
• Two elements: Minimal “non-trivial” case. Good for testing “first and last” or “pair” logic.
• Large input: Maximum n. Tests performance and overflow (e.g., integer or recursion depth).

Value Boundaries

• Zero, negative, or maximum values: Division by zero, negative indices, or values at the limit of the type.
• Duplicates: All elements equal, or many duplicates. “Find two that sum to target” with many repeated numbers can affect indexing or “same index” bugs.
• No valid answer: Problem says “return -1 if not found” or “return empty list.” Your code must explicitly handle and return that.

Structural / Problem-Specific

• Already sorted / reverse sorted: For sorting or search, these can stress your algorithm.
• All same: All zeros, all same character. Can break “find the different one” or “max” logic if you’re not careful.

How to Handle Edge Cases

1. List Them Before Coding

After reading the problem, write down 3–5 edge cases: “What if empty? What if one element? What if no pair exists?” Then ensure your algorithm and code account for each. In an interview, say them out loud: “I’ll need to handle empty input and the case when no two numbers sum to target.”

2. Early Returns / Guards

At the start of the function, check for edge cases and return a defined result. That keeps the main logic simple and avoids special cases inside loops.

def find_max(arr):
    if not arr:           # edge: empty
        return None
    if len(arr) == 1:     # edge: single element (optional, main loop handles it)
        return arr[0]
    # main logic
    result = arr[0]
    for i in range(1, len(arr)):
        if arr[i] > result:
            result = arr[i]
    return result

3. Design the Main Logic to Naturally Handle Boundaries

Sometimes the “normal” logic already works for one element or two (e.g., a loop that runs from 0 to n-1 and updates a result). Other times you need a special case. Prefer one clear path when possible; use early returns when the edge is truly different.

4. Test Edge Cases Explicitly

Before submitting or shipping, run: empty input, one element, two elements, no answer, all same. If any fails, fix the code and add that case to your mental (or automated) test list.

Example: Two Sum Edge Cases

• Empty array: Return “no pair” (e.g., [-1, -1] or empty list) without entering the loop.
• Single element: Can’t form a pair; return “no pair.”
• No pair sums to target: After the loop, return the agreed “not found” value.
• Duplicate values: Ensure you don’t use the same index twice (e.g., i and j must be different). Your algorithm should already enforce that (e.g., j > i or storing index when we see a value).

Summary

• Edge cases = boundary inputs or situations: empty, single element, no answer, duplicates, max size, etc.
• List them before coding; use early returns or design main logic so boundaries are handled.
• Test edge cases explicitly; they define the contract of your solution.

1.9 Problem Decomposition

Introduction

Problem decomposition is the skill of breaking a large, complex problem into smaller subproblems that are easier to solve. You solve the small pieces (often recursively or in order), then combine their results to get the full solution. It’s the same idea behind “divide and conquer,” dynamic programming, and clean system design—and it’s how you avoid feeling overwhelmed by a hard problem.

Why Decompose?

• Manageable pieces: A subproblem like “find the max in the left half” is easier to think about than “find the max in the whole array” when you’re building recursion or iteration.
• Reuse: The same subproblem often appears many times (e.g., “sort this range”). Solve it once and reuse—that’s the heart of dynamic programming and recursion.
• Clear structure: Dependencies between subproblems (e.g., “solve A and B before C”) define the order of computation and help you avoid circular or missing steps.

Mental Model

Picture the problem as a tree: the root is the original problem; children are subproblems. You solve leaves first (base cases), then combine upward until you reach the root. The art is choosing a split that makes the subproblems simpler and the combination step cheap.

  Original: "Sort array A[0..n-1]"
       |
       v
  Subproblem 1: Sort A[0..mid-1]    Subproblem 2: Sort A[mid..n-1]
       |                                    |
       v                                    v
  (base: 1 element, done)            (base: 1 element, done)
       |                                    |
       +---------> COMBINE: merge two sorted halves --------> sorted A
        

How to Decompose

Step 1: Identify Natural Subproblems

Ask: “What smaller version of this problem would help?” Examples: “max of array” → “max of left half” and “max of right half,” then take the larger. “Count ways to reach step n” → “ways to reach n-1” and “ways to reach n-2,” then add. The subproblems should have the same structure as the original (same kind of input/output, smaller size or simpler case).

Step 2: Define the Dependency

Does subproblem A need the result of B? Then B must be solved first. In recursion, that’s “call B, then use its result in A.” In dynamic programming, that’s “fill table in an order so that when we compute A, B is already computed.”

Step 3: Identify Base Cases

What’s the smallest or simplest case you can solve directly? Empty array, single element, zero steps—these stop the recursion or form the first row/column of your DP table. Without clear base cases, decomposition doesn’t terminate.

Step 4: Define How to Combine

Given solutions to the subproblems, how do you get the solution to the bigger problem? Merge two sorted halves, take max of two values, add counts, etc. The combine step should be simpler than solving the whole problem from scratch.

Example: Counting Inversions

Problem: Count pairs (i, j) with i < j and arr[i] > arr[j].

Decomposition: Split array into left and right halves. Count inversions in left, count in right, and count inversions that cross the middle (one in left, one in right). The cross count can be computed during a merge-like pass: when we take an element from the right and there are remaining elements in the left, each of those left elements forms an inversion with this right element. So we get:

• Subproblem 1: count inversions in left half (same problem, smaller n).
• Subproblem 2: count inversions in right half.
• Combine: add left count + right count + cross count (computed in O(n) during merge).

Base case: 0 or 1 element → 0 inversions. This is merge-sort with an extra counter—decomposition turned a hard count into “solve two halves + merge.”

Top-Down vs Bottom-Up

• Top-down (recursion / memoization): Start from the full problem, recurse to subproblems, combine on the way back. Natural to think about; you must handle overlapping subproblems (memoize) to avoid exponential time.
• Bottom-up (tabulation): Solve smallest subproblems first, then larger ones, in order. No recursion; good when dependency order is clear (e.g., “solve for size 1, then 2, then … n”).

Same decomposition; different order of evaluation. Choose based on what’s easier to implement and what fits the problem.

Summary

• Problem decomposition = break a big problem into smaller subproblems, solve them, then combine.
• Identify subproblems, their dependencies, base cases, and the combine step.
• Same idea underlies recursion, divide-and-conquer, and dynamic programming.

1.10 Pattern Recognition in Problems

Introduction

Many problems that look different on the surface share the same underlying pattern: same kind of input, same kind of operation, same data structure or technique that fits. Once you recognize “this is a two-pointer problem” or “this is a BFS shortest-path,” you can apply a known strategy instead of inventing from scratch. Pattern recognition is what turns experience into speed and confidence.

Why Patterns Matter

Interview and contest problems are often designed to test a specific technique. The problem statement might not say “use a hash map”—you’re expected to see that “find two things that sum to target” or “check if we’ve seen this before” suggests a hash map. The more patterns you know, the faster you match the problem to the right tool and the less you waste time on the wrong approach.

Mental Model

Read the problem → notice structure (sorted? pairs? subsequences? paths?) and operations (find, count, maximize, exists?) → map to a pattern → apply the standard strategy for that pattern. Pattern recognition is the step between “what’s being asked” and “which algorithm/structure to use.”

Common Patterns (High-Level)

You’ll see these again in later sections. Here we name them so you can start building the mapping.

Two Pointers

Two indices moving over a sequence (often from both ends or both from the start). Good for: “two numbers that sum to target” in a sorted array, “remove duplicates in place,” “palindrome check,” “merge two sorted arrays.” Clue: sorted array or string, “pair,” “two indices.”

Sliding Window

A contiguous segment of fixed or variable size that moves one step at a time. Good for: “longest subarray with sum ≤ K,” “minimum window containing all characters,” “max in every window of size k.” Clue: “subarray,” “substring,” “contiguous,” “window.”

Hash Map / Set for Lookup

Store what you’ve seen; check membership or fetch in O(1). Good for: “two sum,” “first duplicate,” “group anagrams,” “subarray with sum 0.” Clue: “find a pair,” “have we seen this,” “count distinct,” “group by.”

Prefix Sum

Precompute cumulative sums (or other aggregates) so range queries are O(1). Good for: “subarray sum equals K,” “range sum queries.” Clue: “sum of range,” “subarray sum,” multiple range queries.

Binary Search (on Answer or on Index)

When the answer or the “split point” is in a sorted space, binary search can reduce tries. Good for: “find minimum capacity,” “search in sorted array,” “kth smallest.” Clue: sorted data, “minimum maximum,” “feasibility check.”

BFS / DFS (Graph or Implicit Graph)

Explore neighbors level by level (BFS) or depth-first (DFS). Good for: shortest path in unweighted graph, connected components, “reachable,” “level order.” Clue: “shortest path,” “neighbors,” “grid,” “level.”

Dynamic Programming

Optimal substructure + overlapping subproblems. Good for: “maximum sum,” “count ways,” “longest increasing subsequence,” “edit distance.” Clue: “maximum/minimum,” “count,” “choose or skip,” “subsequence.”

How to Get Better at Recognition

• Solve by pattern: After solving a problem, label it (“two pointers,” “sliding window”). Next time you see similar wording or structure, try that pattern first.
• Note keywords: “Subarray” often suggests sliding window or prefix sum; “two numbers” in sorted array suggests two pointers; “shortest path” in unweighted graph suggests BFS.
• Use constraints: Large n with “find pair” often rules out O(n²) brute force and points to hash or two pointers. Small n might allow brute force or bitmask DP.

Summary

• Pattern recognition = matching problem structure and operations to a known technique (two pointers, hash, sliding window, BFS, DP, etc.).
• Use keywords, constraints, and experience to narrow the strategy.
• Naming the pattern and applying the template speeds you up and impresses interviewers.

1.11 Fermi Estimation for System Design

Introduction

Fermi estimation (named after physicist Enrico Fermi) is the skill of getting an order-of-magnitude answer to a question using rough assumptions and simple arithmetic—no calculator, no exact data. “How many piano tuners are in Chicago?” “How many queries per second does Google handle?” You break the question into a few factors, estimate each to the nearest power of 10 or a round number, multiply or divide, and get a number that’s right within a factor of 10 or so. In system design interviews and in real-world capacity planning, this is how you sanity-check ideas and communicate scale.

Why It Matters

Interviewers ask “how many servers do you need?” or “what’s the storage for 1 billion users?” They don’t expect a precise number—they want to see that you can reason about scale: break the problem down, estimate each piece, and combine. Fermi estimation is that process. In practice, it’s how you quickly check if a design is in the right ballpark before diving into details.

How to Do It

Step 1: Break the Question Into Factors

Turn the big question into a product or quotient of quantities you can guess. Example: “Queries per second for Google search?” → (queries per user per day) × (number of users) ÷ (seconds per day). Each factor is easier to estimate than the whole.

Step 2: Estimate Each Factor

Use round numbers and powers of 10. “Number of people on Earth” ≈ 10^9 (actually ~8 billion, but 10^9 is fine). “Searches per person per day” might be 1–10; say 5. “Seconds per day” = 24 × 3600 ≈ 10^5. It’s okay to be wrong by 2–3×; we’re aiming for order of magnitude.

Step 3: Multiply or Divide

Do the arithmetic. Prefer mental math: 5 × 10^9 / 10^5 = 5 × 10^4 = 50,000. So “on the order of 10^4 to 10^5 queries per second” is a valid answer. Stating “roughly 50k QPS” or “tens of thousands” is better than “I don’t know” or an overly precise wrong number.

Step 4: Sanity Check

Does the result make sense? If you got 10^9 QPS for Google, that might be too high (global internet traffic is finite). If you got 10 QPS, that’s too low for a global product. Adjust assumptions if the result is obviously off.

Example: How Many Servers for a Video Platform?

Question: Roughly how many servers does a YouTube-scale video platform need?

Breakdown:

• Assume 1 billion daily active users, each watching ~30 min of video per day → 0.5 billion hours of video streamed per day.
• Assume average bitrate ~1 Mbps (mixed quality) → 0.5 × 10^9 × 3600 × 1 Mbit ≈ 1.8 × 10^15 bits per day. In bytes, ~2 × 10^14 bytes/day ≈ 2 × 10^11 bytes per second (roughly 200 Gbps global).
• If one server can serve ~10 Gbps (simplified), we need on the order of 200/10 ≈ 20 servers just for egress? That’s too low because we ignored replication, peaks, storage, etc. So we might say “order of 10^5 to 10^6 machines” when we include redundancy, storage servers, and peak load. The exact number isn’t the point—the reasoning is.

Interviewers care that you: (1) break it down, (2) state assumptions, (3) do the math, (4) sanity check.

Rules of Thumb

• Round boldly: 7 billion → 10^9; 24 × 3600 → 10^5. Fewer significant figures = faster and often “good enough.”
• State assumptions: “I’m assuming 10 million users …” so the interviewer can correct you or follow your logic.
• One or two significant figures: “About 50k” or “on the order of 10^5” is fine. Avoid fake precision like “47,382.”

Summary

• Fermi estimation = order-of-magnitude answers using rough assumptions and simple arithmetic.
• Break the question into factors, estimate each (powers of 10, round numbers), multiply/divide, then sanity check.
• State assumptions; aim for one or two significant figures. The reasoning is what interviewers evaluate.

1.12 Invariants & Monovariants in Logical Problems

Introduction

In logical and algorithmic problems, an invariant is something that stays true throughout a process (e.g., before and after each step of a loop or each move in a game). A monovariant (or monotonic variant) is a quantity that only moves in one direction—usually it only decreases (or only increases). Invariants help you prove correctness or narrow down possible states; monovariants help you prove that a process eventually terminates. Together they’re powerful tools for reasoning about algorithms and puzzles.

Why They Matter

In interviews you might get a puzzle or a “prove this loop terminates” question. In competitive programming, some problems are solved by finding an invariant that must hold and then constructing the answer from it. In code, loop invariants are what you use to reason that your algorithm is correct. So even if you don’t hear the words “invariant” and “monovariant” every day, the ideas are central to rigorous thinking.

Invariants

Definition

An invariant is a condition or quantity that remains true (or unchanged) every time a certain operation is applied. You typically state it “at the start of the loop” and “after each iteration”—and show that if it was true before the step, it’s still true after.

Example: Sum of Array Mod 2

Suppose you have a game where in each move you pick two elements and replace them with their sum. The sum of the whole array mod 2 (i.e., even or odd) never changes: (a + b) mod 2 = (a mod 2 + b mod 2) mod 2, so replacing two numbers by their sum doesn’t change the parity of the total. So if the initial sum is odd, the final single number (if you merge everything) must be odd. That invariant restricts what the answer can be.

Example: Loop Invariant in Code

In “find max in array,” a useful invariant is: “At the start of each iteration, max_so_far is the maximum of all elements we’ve seen so far.” Before the loop it’s true (we’ve seen only the first element). Each step we compare the next element and update; so after the step we’ve still seen a prefix and max_so_far is still the max of that prefix. When the loop ends we’ve seen the whole array, so max_so_far is the global max. That’s how you prove the loop correct.

Monovariants

Definition

A monovariant is a quantity that (under the rules of the process) only increases or only decreases. For example: “the number of inversions in the array” might only decrease when we swap two elements in a certain way. If it’s bounded below (e.g., by 0) and it’s integer, it can only decrease a finite number of times—so the process must stop. That’s a termination argument.

Example: Distance to Goal

In a BFS over a grid, “distance from start” increases as we go to deeper layers. So “distance from start” is a monovariant (monotonically increasing along any path). We don’t use it for termination (BFS stops when we find the goal or exhaust the graph), but we use a similar idea: “steps taken” only increases, and the graph is finite, so we can’t run forever.

Example: Inversion Count

In bubble sort, each swap reduces the total number of inversions by exactly 1. So “inversion count” is a monovariant that decreases with each swap. It’s bounded below by 0. So after a finite number of swaps, we must have 0 inversions—i.e., the array is sorted. That proves bubble sort eventually terminates (and with a sorted array).

Using Invariants and Monovariants Together

In a puzzle: first find an invariant that limits what’s possible (e.g., “parity of sum”). Then, if the process has steps that change the state, find a monovariant that only decreases (or increases) and is bounded, so the process must stop. The invariant might tell you the only possible final state; the monovariant tells you you’ll get there.

Summary

• An invariant is a condition or quantity that stays true (or unchanged) through each step; use it to prove correctness or narrow possibilities.
• A monovariant is a quantity that only increases or only decreases and is bounded; use it to prove termination.
• Together they support rigorous reasoning about loops, games, and puzzles.

2.1 Python Data Types

Introduction

Every value in Python has a type: it’s an integer, a string, a list, a dictionary, and so on. The type determines what you can do with the value (index it, add to it, hash it) and how it behaves in memory (mutable vs immutable, shared by reference). For DSA, you need a clear picture of the built-in types so you can pick the right one and avoid subtle bugs—especially around mutability and copying.

Why Data Types Matter for DSA

Different types support different operations at different costs. You use a list when you need order and index access; you use a set or dict when you need fast membership or key lookup. Immutable types (e.g., str, tuple) can be used as dictionary keys and in sets; mutable ones cannot. Knowing types helps you reason about time complexity (e.g., “in” on a list is O(n), on a set is O(1) average) and about correctness (e.g., “did I just mutate a shared list?”).

Numeric Types

int

Integers in Python have arbitrary precision: they can be as large as memory allows. There’s no fixed 32- or 64-bit overflow like in C or Java (though operations get slower for huge numbers). For DSA, this means you usually don’t worry about integer overflow in pure Python; in contests or when interfacing with other systems, modulo arithmetic might still be required by the problem.

x = 42
y = 10**100   # valid in Python
print(type(x))  # <class 'int'>

float

Floating-point numbers are approximate. Avoid using == for equality; use tolerance checks (e.g., abs(a - b) < 1e-9) or integer arithmetic when possible. For DSA, many problems use only integers; when floats appear, be aware of precision issues.

bool

True and False. They’re a subtype of int (True is 1, False is 0), but for clarity use them as booleans. Used in conditionals and as results of comparisons.

Sequence Types

str (string)

Immutable sequence of Unicode characters. Indexing s[i] and slicing s[l:r] are O(1) for access, but slicing creates a new string. Concatenating with + in a loop is O(n²) for n characters; prefer ''.join(...) for building strings. For DSA, strings often appear in pattern matching, palindromes, and parsing.

s = "hello"
# s[0] = 'H'   # TypeError: str does not support item assignment
t = s.upper()  # returns new string; s unchanged

list

Mutable, ordered sequence. Append at end is amortized O(1); insert at front is O(n). Indexing and assignment arr[i] = x are O(1). The main workhorse for arrays in DSA. Supports negative indices: arr[-1] is the last element.

arr = [3, 1, 4, 1, 5]
arr.append(9)    # O(1) amortized
arr[0] = 10     # O(1)
# arr[10]       # IndexError

tuple

Immutable sequence. Same indexing and slicing as list, but you can’t change elements. Use when you need a fixed sequence (e.g., (x, y) coordinates, return multiple values) or when you need a hashable value (e.g., as dict key or set element).

t = (1, 2, 3)
# t[0] = 10    # TypeError
point = (x, y)  # common for coordinates

Mapping and Set Types

dict

Key–value mapping. Keys must be hashable (immutable types: int, str, tuple, etc.). Lookup, insert, and delete by key are O(1) average. Essential for “count frequency,” “have we seen this,” “two sum” style problems. Iteration order is insertion order (Python 3.7+).

d = {}
d["a"] = 1
d["b"] = 2
if "a" in d:      # O(1) average
    print(d["a"])

set

Unordered collection of unique, hashable elements. Add, remove, and “in” are O(1) average. Use when you need fast membership or to remove duplicates. No indexing; iteration order is arbitrary.

s = {1, 2, 3}
s.add(2)         # no change; 2 already there
print(len(s))     # 3

None

None is the single value of type NoneType. Used to mean “no value” or “missing.” Functions that don’t explicitly return something return None. In DSA you’ll use it for “not found” or optional results when you don’t want to use -1 or an exception.

Mutability and Why It Matters

Mutable objects (list, dict, set) can be changed in place. When you pass them to a function, the function gets a reference to the same object—so if the function modifies it, the caller sees the change. That’s useful when you want to build or update a structure, but dangerous when you didn’t intend to share: two names pointing to the same list can cause bugs if you assume they’re independent.

Immutable objects (int, float, str, tuple) can’t be changed. “Operations” that look like changes (e.g., s += 'x') create a new object. So you can’t accidentally mutate shared state; also, immutable values are hashable and can go in sets and as dict keys.

Type Checking

type(x) returns the type of x. isinstance(x, int) is True if x is an int (or a subclass). For branching on type, isinstance is preferred because it respects inheritance. In DSA code you often assume types from the problem statement; type hints (e.g., def f(arr: list[int]) -> int:) document and can be checked with tools.

Quick Reference: DSA-Oriented

• Need ordered sequence, index access, append: list.
• Need fast “have we seen this” or unique elements: set.
• Need fast lookup by key or count by key: dict.
• Need immutable sequence (e.g., as dict key): tuple.
• Need sequence of characters, often read-only: str.

Summary

• Python’s main types for DSA: int, float, bool, str, list, tuple, dict, set, None.
• Mutability matters: mutable (list, dict, set) can be changed in place and are passed by reference; immutable (int, str, tuple) are hashable and safe to share.
• Choose the type that supports the operations you need at the right cost (e.g., set/dict for O(1) lookup).

2.2 Conditionals & Loops

Introduction

Conditionals let your program choose different paths (if this, do that; else do something else). Loops let you repeat a block of code—over a sequence, a range of indices, or until a condition is false. Together they form the control flow of almost every algorithm. For DSA you must be fluent: clean conditionals for edge cases and branches, and correct loops with the right bounds and iteration style.

Conditionals

if / elif / else

Execute a block only when its condition is true. elif is “else if”; only one branch runs. Indentation defines the block (typically 4 spaces).

if n < 0:
    print("negative")
elif n == 0:
    print("zero")
else:
    print("positive")

Truthiness

Conditions don’t have to be strictly True or False. Values are “truthy” or “falsy”: False, None, 0, 0.0, '', [], {} are falsy; most other values are truthy. So if arr: means “if arr is non-empty”; if not arr: means “if arr is empty.” Use this for clean guards.

if not arr:
    return 0
if key in d:   # key exists in dict
    ...

Comparison and Logical Operators

Comparisons: <, <=, >, >=, ==, !=. Chained comparisons: a < b < c is equivalent to a < b and b < c. Logical: and, or, not. Short-circuit: and stops at first falsy; or stops at first truthy. Use parentheses when it helps readability.

Loops

for loop

Iterate over an iterable: a sequence (str, list, tuple) or something that yields values (e.g., range).

for x in [1, 2, 3]:
    print(x)           # 1, 2, 3

for i in range(5):     # i = 0, 1, 2, 3, 4
    print(i)

for i in range(2, 10, 2):  # start 2, stop 10 (exclusive), step 2 → 2,4,6,8
    print(i)

range(n) gives 0 to n-1 (n values). range(a, b) gives a to b-1. range(a, b, step) goes from a by step, stopping before b. So “iterate indices 0 to n-1” is for i in range(len(arr)); “iterate indices 0 to n-1 inclusive in reverse” is for i in range(len(arr)-1, -1, -1).

enumerate

When you need both index and value in a loop, use enumerate(iterable, start=0). It yields (index, value) pairs. Avoids manual index management and off-by-one errors.

arr = [10, 20, 30]
for i, x in enumerate(arr):
    print(i, x)   # (0,10), (1,20), (2,30)

for i, x in enumerate(arr, start=1):
    print(i, x)   # (1,10), (2,20), (3,30)

while loop

Repeat while a condition is true. Use when the number of iterations isn’t known in advance (e.g., “while the stack is not empty,” “while n > 0”). Ensure the condition eventually becomes false to avoid infinite loops.

n = 10
while n > 0:
    print(n)
    n -= 1

break and continue

break: exit the innermost loop immediately. Use when you’ve found what you need (e.g., found a pair that sums to target).

continue: skip the rest of the current iteration and go to the next. Use to skip unwanted elements without nesting.

for i in range(n):
    if arr[i] < 0:
        continue   # skip negative
    process(arr[i])

else on Loops

A for or while can have an else block. It runs only if the loop completes without hitting a break. Useful for “search and report if not found.”

for x in arr:
    if x == target:
        print("Found")
        break
else:
    print("Not found")  # runs only if we never broke

Nested Loops and Complexity

A loop inside a loop typically does work proportional to the product of the iteration counts. Two loops over n → O(n²). That’s why we optimize by removing inner loops (e.g., with a hash map) when possible. Be aware of what’s inside the inner loop: if each iteration does O(1), total is O(n²); if the inner loop does O(n), total can be O(n³).

Common Patterns in DSA

• Iterate by index: for i in range(len(arr)) when you need to read or write arr[i] or use i in logic.
• Iterate by value: for x in arr when you only need the elements.
• Index and value: for i, x in enumerate(arr).
• Reverse index: for i in range(len(arr)-1, -1, -1) or for x in reversed(arr) (latter doesn’t give index).

Summary

• Conditionals: if / elif / else; use truthiness (if arr:, if not arr:) for clean guards.
• for: over iterables and range; use enumerate for index + value; range(n) is 0..n-1.
• while: when iterations aren’t known in advance; ensure condition eventually becomes false.
• break exits the loop; continue skips to the next iteration; else on a loop runs when no break occurred.

2.3 Functions

Introduction

A function is a named block of code that takes inputs (arguments), does work, and optionally returns a value. Functions let you reuse logic, structure programs into clear steps, and express algorithms in small, testable pieces. For DSA, almost every solution is written as one or more functions—so you need a solid grasp of how to define them, pass data in and out, and avoid common pitfalls (especially with mutable arguments and scope).

Why Functions Matter for DSA

In problems you’ll write a function that takes the input (e.g., an array and a target) and returns the answer. Recursion is “a function that calls itself.” Helper functions keep your main solution readable (e.g., “is_valid,” “merge”). Understanding parameters (by value vs by reference), return values, and default arguments helps you avoid bugs and write clean, interview-ready code.

Defining a Function

Use the def keyword, the function name, parentheses with parameters, and a colon. The body is indented. Execution starts at the first line of the body and ends at return or at the end of the block (then the function returns None).

def greet(name):
    return "Hello, " + name

result = greet("Alice")   # result is "Hello, Alice"

Parameters and Arguments

Parameters are the names listed in the def line; arguments are the values you pass when you call the function. In Python, arguments are passed by object reference: the parameter name is bound to the same object the caller passed. For immutable types (int, str, tuple), reassigning the parameter doesn’t affect the caller. For mutable types (list, dict), modifying the object does affect the caller—because it’s the same object.

Positional and Keyword Arguments

Arguments can be passed by position or by name. f(1, 2) passes 1 and 2 to the first and second parameters. f(a=1, b=2) or f(1, b=2) uses names; keyword arguments must come after positional ones in a call.

def add(a, b):
    return a + b

add(3, 5)       # 8, positional
add(a=3, b=5)   # 8, keyword
add(3, b=5)     # 8, mixed

Default Parameter Values

You can give a parameter a default so callers can omit it. Defaults are evaluated once at function definition time—so avoid using mutable defaults (like def f(arr=[])); that one list is shared across all calls that omit arr. Use def f(arr=None) and if arr is None: arr = [] instead.

def power(x, n=2):
    return x ** n

power(5)     # 25
power(5, 3)  # 125

Return Values

return exits the function and sends a value back to the caller. You can return one value, multiple values (as a tuple—often unpacked by the caller), or nothing (returns None). Returning multiple values is just return a, b; the caller gets (a, b) and can write x, y = f().

def min_max(arr):
    if not arr:
        return None, None
    return min(arr), max(arr)

lo, hi = min_max([3, 1, 4])  # lo=1, hi=4

Scope

Variables defined inside a function are local to that function; they aren’t visible outside. Variables defined at the top level (module level) are global. Reading a global name inside a function is allowed; assigning to it (e.g., count = 0) creates a new local name unless you declare global count. For DSA, prefer passing values in and returning them—avoid global state so your functions are easy to test and reason about.

Mutability and Side Effects

If you pass a list and the function appends to it or changes elements, the caller’s list is modified. That’s useful when you intentionally want to build or update a structure in place (e.g., “fill this list with results”). It’s a bug when the caller didn’t expect the input to change. When in doubt, don’t mutate the input unless the problem or API says so; if you need to return a modified copy, build a new list/dict and return it.

Recursion Preview

A function can call itself—that’s recursion. You’ll see it in depth in the next section. For now: every recursive function needs a base case (when to stop) and a recursive case (how to express the result in terms of a smaller subproblem). Example: factorial—fact(0)=1; fact(n)=n*fact(n-1).

def fact(n):
    if n <= 0:
        return 1
    return n * fact(n - 1)

Summary

• Functions are defined with def; they take parameters and return values (or None).
• Arguments are passed by object reference: mutating a mutable argument affects the caller; reassigning an immutable doesn’t.
• Avoid mutable default arguments; use None and create a new list/dict inside if needed.
• Prefer passing data in and returning results; be clear about whether your function mutates input.

2.4 Recursion Basics

Introduction

Recursion is when a function calls itself to solve a smaller instance of the same problem. Many algorithms (tree traversal, divide-and-conquer, backtracking) are naturally expressed recursively. To use it well you need a clear base case (when to stop), a correct recursive case (how to reduce the problem), and an understanding of the call stack so you can reason about correctness and space.

Why Recursion Matters for DSA

Recursion appears everywhere: merge sort (“sort left half, sort right half, merge”), binary search (“search left or right half”), tree DFS (“process root, then recurse on each child”), and backtracking (“try a choice, recurse, undo”). If you’re comfortable with recursion, these patterns are much easier to write and debug. The same logic can often be converted to an iterative loop with an explicit stack—but thinking recursively first is a core skill.

Two Parts of Every Recursive Function

Base Case

The base case is when the problem is so small that you can return an answer directly without calling yourself again. Without a base case, recursion never stops and you get infinite recursion (and eventually a stack overflow). Examples: “if n is 0, return 1” (factorial); “if the list is empty, return 0” (sum of list).

Recursive Case

The recursive case expresses the answer in terms of the same function on a smaller input. You must ensure the subproblem is strictly smaller (e.g., n−1 or half the array) so that repeated recursion eventually hits the base case. Example: “fact(n) = n * fact(n−1)”—each call reduces n until n is 0.

  Recursive function:
  1. Base case: return known answer for smallest input.
  2. Recursive case: call self on smaller input; combine result with current step.
        

Example: Factorial

Definition: fact(0) = 1; fact(n) = n × fact(n−1) for n ≥ 1. Base case: n ≤ 0 → return 1. Recursive case: return n * fact(n−1).

def fact(n):
    if n <= 0:
        return 1
    return n * fact(n - 1)

Trace for fact(3): fact(3)=3*fact(2), fact(2)=2*fact(1), fact(1)=1*fact(0), fact(0)=1. Then 1→1, 2*1=2, 3*2=6. So fact(3)=6.

Example: Sum of an Array

Sum of arr = first element + sum of the rest. Base case: empty list → 0. Recursive case: arr[0] + sum(arr[1:]).

def sum_arr(arr):
    if not arr:
        return 0
    return arr[0] + sum_arr(arr[1:])

Note: arr[1:] creates a new list each time, so this uses O(n) extra space per level and O(n²) time for the slicing. For real DSA you’d often pass indices (left, right) instead of slicing—same recursive idea, better complexity. The point here is the structure: base + recursive case.

Example: Fibonacci

F(0)=0, F(1)=1, F(n)=F(n−1)+F(n−2). Base cases: n=0 → 0, n=1 → 1. Recursive case: return fib(n−1) + fib(n−2).

def fib(n):
    if n <= 0:
        return 0
    if n == 1:
        return 1
    return fib(n - 1) + fib(n - 2)

This version is correct but exponential in n because it recomputes the same F(k) many times. Later you’ll fix that with memoization or iteration—but as a first recursive definition it’s the standard example.

Call Stack Intuition

Each recursive call pushes a new frame onto the call stack (with its own parameters and local variables). When a call returns, its frame is popped and execution resumes in the caller. So recursion uses space proportional to the depth of the recursion (e.g., fact(n) has depth n; sum_arr on a list of length n has depth n). If the depth is too large (e.g., recursion on a list of 10^5 elements without tail-call optimization), you get a stack overflow. For deep recursion, an iterative solution or passing indices to limit depth is often better.

Common Mistakes

• No base case or wrong base case: Recursion never stops, or returns wrong value for the smallest input. Always ask: “What’s the smallest input, and what should I return?”
• Recursive case not smaller: If you call f(n) in terms of f(n) or f(n+1), you never reach the base case. Ensure the argument (or problem size) strictly decreases.
• Wrong recurrence: The formula that combines “current step” with “result of smaller problem” must match the problem definition. Check with a small example by hand.

Summary

• Recursion = function calls itself on a smaller instance; needs a base case and a recursive case.
• Base case: return directly for smallest input. Recursive case: call self on smaller input and combine.
• Recursion depth = stack space; avoid very deep recursion (e.g., long lists) unless you use indices or iteration.
• Naive Fibonacci is exponential; memoization or iteration fixes it (covered later).

2.5 Lists

Introduction

A list in Python is a mutable, ordered sequence of values. It’s the primary “array” type for DSA: you use it for sequences that need index access, in-place updates, and dynamic growth. Understanding how to create, index, slice, and modify lists—and what each operation costs—is essential for implementing algorithms correctly and efficiently.

Why Lists Matter for DSA

Most array-based problems give you or expect a list. You’ll index by position (arr[i]), append during a single pass, or use a list as an explicit stack or queue. Knowing that append is amortized O(1) but insert(0, x) is O(n) helps you choose the right structure (e.g., collections.deque for frequent front insert/delete).

Creating Lists

arr = []                    # empty list
arr = [1, 2, 3]             # literal
arr = list()                # same as []
arr = list(range(5))        # [0, 1, 2, 3, 4]
arr = [0] * 10              # [0, 0, ..., 0] — same object repeated (careful with mutable elements!)
arr = [x * 2 for x in range(5)]  # list comprehension: [0, 2, 4, 6, 8]

Indexing and Slicing

Indices are 0-based. arr[i] is the element at position i; valid range is 0 to len(arr)-1. Negative indices count from the end: arr[-1] is the last element, arr[-2] the second-to-last. Indexing is O(1).

Slicing arr[start:stop] returns a new list from index start up to but not including stop. Omitted start defaults to 0; omitted stop defaults to the end. arr[start:stop:step] steps by step. Slicing creates a copy—O(k) where k is the slice length.

arr = [10, 20, 30, 40, 50]
arr[0]      # 10
arr[-1]     # 50
arr[1:4]    # [20, 30, 40]  — indices 1,2,3
arr[:3]     # [10, 20, 30]
arr[::2]    # [10, 30, 50]  — every 2nd element

Mutability and In-Place Operations

Lists are mutable: you can change elements, append, insert, and remove. Assigning to an index updates that slot: arr[i] = x is O(1). In-place operations modify the list and often return None (e.g., arr.sort() returns None; sorted(arr) returns a new list).

Key Operations and Time Complexity

append is amortized O(1) because the list occasionally grows its backing storage; over many appends the average cost per append is constant. insert(0, x) shifts all elements, so it’s O(n). Use collections.deque when you need O(1) front and back operations.

Common Methods

• arr.append(x) — add at end. Amortized O(1).
• arr.extend(iterable) — add all elements from iterable at end. O(k) for k elements.
• arr.insert(i, x) — insert x at index i; elements shift. O(n).
• arr.pop() — remove and return last element. O(1). arr.pop(i) removes at index i, O(n).
• arr.remove(x) — remove first occurrence of x. O(n).
• arr.reverse() — reverse in place. O(n).
• arr.sort() — sort in place. O(n log n). sorted(arr) returns new list, doesn’t change arr.
• len(arr), arr.count(x), arr.index(x) — length O(1); count and index O(n).

List as Stack or Queue

Stack: Use append for push and pop() for pop—both O(1) amortized. Perfect for DFS, expression evaluation, etc.

Queue: Using insert(0, x) to enqueue and pop() to dequeue makes enqueue O(n). For a real queue use collections.deque (append and popleft both O(1)).

# Stack
stack = []
stack.append(1)
stack.append(2)
top = stack.pop()   # 2

Copying vs Reference

Assignment does not copy: b = arr makes b refer to the same list. Modifying b changes arr. To copy: arr.copy() or arr[:] — shallow copy (same elements; if elements are mutable, inner objects are shared). For a deep copy of nested structures use copy.deepcopy(arr).

Summary

• Lists are mutable, ordered sequences; index and slice with 0-based indices; negative indices from the end.
• append / pop at end are O(1) amortized; insert(0) / pop(0) / in are O(n).
• Use a list as a stack (append + pop); for a queue use deque.
• Assignment shares the list; use arr.copy() or arr[:] for a shallow copy when needed.

2.6 Tuples

Introduction

A tuple is an immutable, ordered sequence of values. Like a list, it supports indexing and slicing—but you cannot add, remove, or change elements after creation. That immutability makes tuples hashable: they can be used as dictionary keys and as set elements, and they safely represent fixed structures (e.g., coordinates, multi-value returns) without accidental mutation.

Why Tuples Matter for DSA

You’ll use tuples when you need a fixed pair or record (e.g., (row, col) for grid positions, (distance, node) for Dijkstra’s priority queue). Returning multiple values from a function is done with a tuple (return i, j). Storing “visited” or “seen” composite keys (e.g., (i, j) for 2D states) requires a hashable type—list won’t work in a set or as a dict key; tuple will. When you don’t need to mutate the sequence, tuple is a clear and efficient choice.

Creating Tuples

t = ()                    # empty tuple
t = (1,)                  # single-element tuple — comma required
t = (1, 2, 3)             # literal
t = 1, 2, 3               # parentheses optional when unambiguous
t = tuple()               # empty, same as ()
t = tuple([1, 2, 3])      # from iterable: (1, 2, 3)

Single-element tuple needs a trailing comma: (1,). Without it, (1) is just the integer 1 in parentheses.

Indexing and Slicing

Same as lists: 0-based indices, negative indices from the end, slicing t[start:stop] returns a new tuple. Indexing and slicing are O(1) and O(k) respectively. You cannot assign: t[0] = 5 raises TypeError.

t = (10, 20, 30, 40)
t[0]      # 10
t[-1]     # 40
t[1:3]    # (20, 30)

Unpacking

You can assign elements of a tuple to multiple names in one line. Useful for function returns and loop variables.

x, y = (3, 5)           # x=3, y=5
a, b = b, a             # swap without temp
for i, (r, c) in enumerate([(1,2), (3,4)]):
    ...                 # i=0, r=1, c=2; then i=1, r=3, c=4

Star unpacking: first, *rest = (1, 2, 3, 4) gives first=1, rest=[2, 3, 4] (rest is a list).

Hashable: Dict Keys and Set Elements

Because tuples are immutable, they can be hashed (assuming their elements are hashable). So you can use a tuple as a dict key or put it in a set—unlike a list.

seen = set()
seen.add((0, 0))        # valid
# seen.add([0, 0])      # TypeError: unhashable type: 'list'

dist = {}
dist[(1, 2)] = 5       # key is tuple (1, 2)

In DSA, “visited cells” in a 2D grid are often stored as set() of (r, c) tuples. State in memoization (e.g., DP) is often a tuple of parameters so it can be used as a key.

Tuple vs List: When to Use Which

• Tuple: Fixed structure, multiple return values, keys or set members, or when you want to signal “this sequence is not meant to change.”
• List: Need to append, remove, or change elements; building a sequence over time; stack/queue.

Tuples are slightly more memory-efficient and can be slightly faster for creation and access in some cases, but the main reason to choose a tuple is semantics: immutability and hashability.

Summary

• Tuples are immutable, ordered sequences; same indexing and slicing as lists, but no assignment.
• Tuples are hashable (if elements are hashable)—use as dict keys and set elements; lists are not.
• Use tuples for fixed structure, multi-value returns, and composite keys (e.g., (r, c), (i, j)).

2.7 Sets

Introduction

A set is an unordered collection of unique, hashable elements. No duplicates are stored; adding the same element again has no effect. Membership test (x in s), add, and remove are O(1) average. Sets are the go-to structure when you need “have I seen this?” or “distinct values” without caring about order or index.

Why Sets Matter for DSA

Many problems reduce to “check if this value exists” or “collect unique items.” With a list, x in arr is O(n); with a set it’s O(1) average—so a single pass with a set can replace nested loops (e.g., Two Sum: for each element, check if complement is in a set). Sets also give you deduplication for free: set([1,2,2,3]) → {1, 2, 3}. In graph BFS/DFS, “visited” is often a set of nodes. Use a set whenever the key operation is membership or uniqueness.

Creating Sets

s = set()                  # empty set (not {} — that's empty dict)
s = {1, 2, 3}             # literal
s = set([1, 2, 2, 3])     # from iterable: {1, 2, 3}, duplicates removed
s = set("hello")          # {'h', 'e', 'l', 'o'} — unique chars

Elements must be hashable (immutable: int, str, tuple, etc.). You cannot have a list or another set inside a set.

Key Operations and Time Complexity

• x in s, x not in s — O(1) average.
• s.add(x) — add element; no effect if already present. O(1) average.
• s.remove(x) — remove x; raises KeyError if not in set.
• s.discard(x) — remove x if present; no error if absent. O(1) average.
• len(s) — O(1).

There is no indexing: sets are unordered. Iteration order is arbitrary (and can change). Use for x in s when you only need to process each element once.

Set Operations

Mathematical set operations are built in:

a = {1, 2, 3}
b = {2, 3, 4}
a | b   # union: {1, 2, 3, 4}
a & b   # intersection: {2, 3}
a - b   # difference (in a, not in b): {1}
a ^ b   # symmetric difference (in one but not both): {1, 4}

These return a new set. In-place versions: a.update(b) (union), a.intersection_update(b), a.difference_update(b). Useful for “remove all seen elements” or “keep only common elements.”

No Duplicates, No Order

Adding the same element multiple times doesn’t change the set. So building a set from a list is an easy way to get unique values—but you lose order. If you need “unique but preserve insertion order” (Python 3.7+), dict.fromkeys(arr) gives you an ordered mapping; list(dict.fromkeys(arr)) is unique elements in first-seen order.

When to Use a Set

• Membership: “Is x in the collection?” — use a set for O(1) instead of list O(n).
• Uniqueness: “Collect distinct values” — add to a set; duplicates are ignored.
• Visited / seen: In BFS/DFS, store visited nodes in a set for O(1) check and add.
• No indexing needed: If you don’t need order or position, set is simpler and faster for in/add/remove.

frozenset

frozenset is an immutable set. It’s hashable, so it can be used as a dict key or as an element of another set. Use it when you need a set that must not change and must be storable in a set or as a key (e.g., “set of frozensets” for distinct subsets).

fs = frozenset([1, 2, 3])
# fs.add(4)   # AttributeError
seen_subsets = {fs}       # valid — frozenset is hashable

Summary

• Sets store unique, hashable elements; unordered; no indexing.
• in, add, remove, discard are O(1) average—use sets for fast membership and deduplication.
• Use a set for “seen,” “visited,” “distinct values,” or any problem where the main operation is “is this in the collection?”

2.8 Dictionaries

Introduction

A dictionary (dict) is a mapping from keys to values. Keys must be hashable (immutable); values can be anything. Lookup, insert, and delete by key are O(1) average. In Python 3.7+, dictionaries preserve insertion order. For DSA, dicts are essential: frequency counts, “value → index” or “value → count,” memoization caches, and graph adjacency (when nodes are hashable) all use dicts.

Why Dictionaries Matter for DSA

Whenever you need “for this key, what’s the value?” or “have I seen this key and what did I store?” a dict is the right structure. Two Sum: map each value to its index so you can find a complement in O(1). Frequency: map each element to its count in one pass. Memoization: map (arguments as key) to return value. Subarray with sum K: prefix sum → count of prefixes seen. Most hash-based optimizations in this course are implemented with a dict.

Creating Dictionaries

d = {}                      # empty dict
d = {"a": 1, "b": 2, "c": 3}   # literal
d = dict()                  # empty
d = dict(a=1, b=2)          # keyword args (keys are strings)
d = dict([("a", 1), ("b", 2)])  # from list of (key, value) pairs
d = dict.fromkeys([1, 2, 3], 0)  # {1: 0, 2: 0, 3: 0} — same value for all keys

Accessing and Modifying

d[key] returns the value for key; raises KeyError if the key is missing. d[key] = value sets or overwrites. del d[key] removes the key; key in d is O(1) average. Use d.get(key, default) to avoid KeyError: returns d[key] if key exists, else default (or None if no default given).

d = {"x": 10, "y": 20}
d["x"]       # 10
d["z"] = 30  # add or update
d.get("x")   # 10
d.get("w")   # None
d.get("w", 0)   # 0 — default when key missing

Key Operations and Time Complexity

• d[key], d[key] = value, del d[key], key in d — O(1) average.
• len(d) — O(1).
• d.get(key, default) — O(1) average.

Iteration: for k in d iterates keys; d.keys(), d.values(), d.items() give key, value, or (key, value) pairs. In Python 3 these are view-like and reflect the current dict; iteration order is insertion order (3.7+).

Common Patterns in DSA

• Frequency count: for x in arr: d[x] = d.get(x, 0) + 1 — one pass, O(n).
• Value → index (e.g., Two Sum): for i, x in enumerate(arr): store d[x] = i (or list of indices if duplicates matter). Check target - x in d before storing.
• Memoization: Key = tuple of arguments (or something hashable); value = computed result. Check if key in d: return d[key] before computing.

# Frequency count
freq = {}
for x in arr:
    freq[x] = freq.get(x, 0) + 1

defaultdict

collections.defaultdict(factory) is a dict that never raises KeyError on access: if the key is missing, it creates a value with factory() and stores it. defaultdict(int) gives 0 for missing keys—so freq[x] += 1 works without get. defaultdict(list) gives [] for missing keys—useful for “group by key.” Same O(1) average behavior as dict.

from collections import defaultdict
freq = defaultdict(int)
for x in arr:
    freq[x] += 1   # no get() needed

groups = defaultdict(list)
for item in items:
    groups[item.key].append(item)   # each key gets a list

Keys Must Be Hashable

Dictionary keys must be immutable and hashable: int, str, tuple (of hashable elements), etc. You cannot use a list or another dict as a key. For composite keys (e.g., (i, j) for 2D state), use a tuple. If you need to key by something mutable, convert to something hashable (e.g., tuple(sorted(items)) for a “canonical” set-like key).

Summary

• Dictionaries map hashable keys to values; lookup, insert, delete by key are O(1) average.
• Use d.get(key, default) to avoid KeyError; use defaultdict when you want a default value for missing keys.
• Dicts are the standard tool for frequency counts, value→index (Two Sum), and memoization.

2.9 Strings

Introduction

A string in Python is an immutable sequence of Unicode characters. Like a tuple, you can index and slice it, but you cannot change a character in place. Strings are hashable (so they can be dict keys or set elements) and support many methods for searching, splitting, and transforming. For DSA, strings show up in palindromes, anagrams, pattern matching, and parsing—and building strings efficiently (e.g., with ''.join()) matters for performance.

Why Strings Matter for DSA

String problems often ask: “Is it a palindrome?” “Are two strings anagrams?” “Find pattern in text.” You’ll index by position (s[i]), slice substrings (s[l:r]), and compare or count characters. Because strings are immutable, “changing” a character means creating a new string (e.g., s[:i] + c + s[i+1:])—O(n). For heavy string building, use a list of characters and ''.join(...) at the end for O(n) total instead of O(n²) from repeated concatenation.

Indexing and Slicing

Same as lists: 0-based indices, negative indices from the end. s[i] is O(1). s[start:stop] returns a new string (substring); omitted start/stop mean “from start” / “to end.” s[::-1] is the reversed string—handy for palindrome checks. Slicing is O(k) where k is the slice length.

s = "hello"
s[0]      # 'h'
s[-1]     # 'o'
s[1:4]    # "ell"
s[::-1]   # "olleh" — reverse

Immutability

You cannot do s[0] = 'H'—that raises TypeError. To “change” a string you build a new one: s = s[:i] + new_char + s[i+1:], or use methods like replace, upper, etc., which all return new strings. The original string is never modified.

Concatenation and Building Strings

a + b creates a new string of length len(a)+len(b). Doing result = result + c in a loop is O(n²) over n characters, because each concatenation copies the whole result. For building a string from many parts, collect parts in a list and join once: ''.join(parts) is O(n).

# Slow: O(n^2)
result = ""
for c in s:
    result = result + c.upper()   # avoid in loops

# Fast: O(n)
result = ''.join(c.upper() for c in s)
# Or: parts = []; parts.append(...); ''.join(parts)

Common Methods

• s.split(sep=None) — split by whitespace (default) or by sep; returns list of strings. "a b c".split() → ['a','b','c'].
• s.strip(), s.lstrip(), s.rstrip() — remove leading/trailing whitespace (or specified chars).
• s.upper(), s.lower() — return new string in upper/lower case.
• s.replace(old, new) — return new string with all occurrences of old replaced by new.
• s.startswith(prefix), s.endswith(suffix) — boolean.
• s.find(sub) — index of first occurrence of sub, or -1. s.index(sub) same but raises ValueError if not found.
• s.count(sub) — number of non-overlapping occurrences of sub.

All of these return new strings or other values; they do not modify s.

Membership and Iteration

c in s is O(n) — Python scans the string. for c in s iterates over characters. for i, c in enumerate(s) gives index and character. To get a list of characters: list(s). To convert a list of characters back: ''.join(lst).

s = "abc"
list(s)   # ['a', 'b', 'c']
''.join(['a', 'b', 'c'])   # "abc"

Comparing Strings

Strings are compared lexicographically (dictionary order): character by character, using Unicode code points. ==, !=, <, >, <=, >= all work. So sorted(list_of_strings) sorts them alphabetically. For case-insensitive sort: sorted(lst, key=str.lower).

Summary

• Strings are immutable sequences of characters; index and slice like lists; slicing creates a new string.
• Build strings with ''.join(parts), not repeated s += c in a loop, to avoid O(n²).
• Use split, strip, upper/lower, replace, find, startswith/endswith as needed; all return new values.

2.10 List Comprehension

Introduction

A list comprehension is a compact syntax for building a list from an iterable (and optionally filtering). Instead of a multi-line loop that appends to a list, you write a single expression in brackets. It’s idiomatic in Python, readable once you’re used to it, and often slightly faster than an equivalent loop. For DSA you’ll use it to build lists of indices, transformed values, or filtered elements in one line.

Basic Syntax

[expression for item in iterable] — for each item in the iterable, evaluate expression and collect the results into a new list. The result has the same length as the iterable (unless you add a filter).

squares = [x * x for x in range(5)]     # [0, 1, 4, 9, 16]
uppers  = [c.upper() for c in "hello"]  # ['H', 'E', 'L', 'L', 'O']
lengths  = [len(w) for w in ["a", "bb", "ccc"]]  # [1, 2, 3]

With a Filter: if

[expression for item in iterable if condition] — only include expression when condition is true. The result can be shorter than the iterable.

evens = [x for x in range(10) if x % 2 == 0]   # [0, 2, 4, 6, 8]
pos   = [x for x in arr if x > 0]              # only positive elements

There is no else in the filter position. To map some items to one value and others to another, use a conditional expression in the expression part: [a if cond else b for x in ...].

# Map: even -> "e", odd -> "o"
labels = ["e" if x % 2 == 0 else "o" for x in range(5)]  # ['e','o','e','o','e']

Nested Loops

You can use multiple for clauses. They nest left to right: the rightmost iterable is the “inner” loop.

pairs = [(i, j) for i in range(2) for j in range(2)]
# [(0,0), (0,1), (1,0), (1,1)]

Equivalent to:

pairs = []
for i in range(2):
    for j in range(2):
        pairs.append((i, j))

You can add an if after the loops to filter. Keep comprehensions readable; if they get long or complex, an explicit loop is often clearer.

When to Use List Comprehensions

• Use: Simple “build a list from an iterable” or “build with one filter”—one line, easy to read.
• Avoid: Complex logic, multiple conditions, or when you need side effects (e.g., printing, updating other variables). Use an explicit for loop instead.

Dict and Set Comprehensions

Same idea with different brackets: {key_expr: value_expr for x in it} builds a dict; {expr for x in it} builds a set. Useful for “list of pairs → dict” or “unique values from a transformation.”

d = {x: x * x for x in range(5)}   # {0:0, 1:1, 2:4, 3:9, 4:16}
s = {c.upper() for c in "hello"}   # {'H', 'E', 'L', 'O'}

Summary

• List comprehension: [expr for item in iterable] or [expr for item in iterable if cond].
• Use for simple “build list / filter list” in one line; use explicit loops for complex logic or side effects.
• Dict comprehension: {k: v for ...}; set comprehension: {expr for ...}. Generator: (expr for ...) when you don’t need a full list.

2.11 Lambda Functions

Introduction

A lambda is an anonymous function defined with a single expression. You write lambda arguments: expression—no def, no block, no explicit return. Lambdas are useful when you need a small function as an argument (e.g., key for sorted, or a comparator). For anything longer or reusable, use a normal def function.

Syntax

lambda x: x * 2 is a function that takes one argument x and returns x * 2. You can have multiple arguments: lambda a, b: a + b. You cannot have multiple statements or assignments—only one expression. The expression is implicitly returned.

f = lambda x: x * 2
f(5)   # 10

g = lambda a, b: a - b
g(7, 3)   # 4

Common Use: sorted and key

Many built-ins accept a key function that maps each element to a value used for comparison. sorted(iterable, key=...) sorts by the result of key(item). A lambda is often the shortest way to supply that.

arr = [("b", 2), ("a", 3), ("c", 1)]
sorted(arr)                    # by first element: [('a',3), ('b',2), ('c',1)]
sorted(arr, key=lambda t: t[1])   # by second: [('c',1), ('b',2), ('a',3)]

# Sort list of numbers by absolute value
sorted([-3, 1, -2], key=lambda x: abs(x))   # [1, -2, -3]

Same idea for min, max: min(items, key=lambda x: x.weight). For custom comparison in sorting (e.g., multiple criteria), you can use key=lambda x: (x.a, -x.b) to sort by a ascending and b descending.

map and filter

map(f, iterable) returns an iterator of f(x) for each x. filter(pred, iterable) returns an iterator of elements for which pred(x) is true. Lambdas can be used as f or pred.

list(map(lambda x: x * 2, [1, 2, 3]))     # [2, 4, 6]
list(filter(lambda x: x > 0, [-1, 2, 0, 3]))  # [2, 3]

In Python, list comprehensions are often clearer: [x * 2 for x in lst] and [x for x in lst if x > 0]. Use map/filter with lambda when you prefer that style or when you need an iterator without building a list.

When to Use Lambda vs def

• Lambda: One-off function passed to sorted, min, max, key=, or similar; expression fits on one line and is simple.
• def: Reusable logic, multiple lines, default arguments, or when a name would make the code clearer. If you’re doing f = lambda ... and using f in several places, a named function is better.

Summary

• Lambda = anonymous function: lambda args: expression; one expression only, implicitly returned.
• Use with sorted(..., key=lambda x: ...), min/max key=, or map/filter when the logic is one short expression.
• Use def for anything multi-line, reusable, or complex.

2.12 Sorting in Python

Introduction

Sorting arranges elements in a defined order (ascending or descending). In Python you have two main ways: sorted(iterable)—which returns a new sorted list and leaves the original unchanged—and list.sort()—which sorts the list in place and returns None. Both use the same underlying algorithm (Timsort) and support a key function and reverse flag. For DSA, you’ll sort to enable binary search, two-pointer techniques, or to order items by a custom rule; knowing the API and the cost is essential.

Why Sorting Matters for DSA

Many algorithms assume or produce sorted data: binary search requires a sorted array; “merge two sorted lists” and “find pairs with sum K” often start by sorting; greedy problems sometimes need items ordered by value or deadline. Python’s sort is O(n log n) and stable—so you get predictable, efficient ordering. Custom key lets you sort by a field, by multiple criteria, or by a computed value without writing a comparison function by hand.

Two Ways to Sort

sorted(iterable, key=..., reverse=...)

Built-in function. Accepts any iterable (list, tuple, string, etc.) and returns a new list in sorted order. The original is not modified. Use when you need to keep the original or when the input isn’t a list (e.g., sorted("hello") → ['e','h','l','l','o']).

arr = [3, 1, 4, 1, 5]
new_list = sorted(arr)   # [1, 1, 3, 4, 5]; arr unchanged
s = "hello"
sorted(s)                # ['e', 'h', 'l', 'l', 'o'] — returns list of chars

list.sort(key=..., reverse=...)

Method on lists only. Sorts the list in place and returns None. Slightly more efficient when you don’t need the original: no extra list is allocated. Use when the variable is a list and you’re fine mutating it.

arr = [3, 1, 4, 1, 5]
arr.sort()   # arr is now [1, 1, 3, 4, 5]; returns None
# new_list = arr.sort()   # wrong — new_list would be None

The key Parameter

Both sorted() and list.sort() accept key=function. For each element x, the sort compares key(x) instead of x. So you can sort by a field, by a transformation, or by a tuple for multiple criteria—without implementing a comparator. The key function is called once per element; the result is cached, so cost is O(n) key calls plus O(n log n) comparisons.

Sort by a Single Field

pairs = [(2, "b"), (1, "a"), (2, "a")]
sorted(pairs)                      # by first elem: [(1,'a'), (2,'a'), (2,'b')]
sorted(pairs, key=lambda p: p[1])  # by second: [(2,'a'), (1,'a'), (2,'b')]

Sort by Multiple Criteria

Return a tuple from key. Python compares tuples lexicographically: first element, then second, and so on. To sort by field A ascending and field B descending, use (x.a, -x.b) for numbers (negate for descending), or a custom key that returns a tuple where the “descending” part is inverted (e.g., -x.b for numeric B).

# Sort by second element ascending, then first descending
pairs = [(2, 1), (3, 1), (1, 2)]
sorted(pairs, key=lambda p: (p[1], -p[0]))  # [(3,1), (2,1), (1,2)]

Sort by Computed Value

arr = [-4, 2, -1, 3]
sorted(arr, key=abs)   # [-1, 2, 3, -4] — by absolute value
sorted(arr, key=lambda x: -x)   # [3, 2, -1, -4] — descending (key negated)

The reverse Parameter

reverse=True sorts in descending order. Same comparisons as ascending; order of output is reversed. Equivalent to sorting by key(x) and then reversing, but reverse=True is built in and clear.

sorted([3, 1, 4], reverse=True)   # [4, 3, 1]

Stability

Python’s sort is stable: when two elements compare equal, their relative order in the output is the same as in the input. So you can sort by one field first, then sort again by another field—the second sort preserves the order of ties from the first. Example: sort by last name, then by first name; people with the same last name stay in the order of first name from the previous pass. Or: sort by value, then by index—equal values keep original index order.

Time and Space Complexity

Python uses Timsort (a hybrid of merge sort and insertion sort). Time: O(n log n) in the average and worst case. Each key is computed once—O(n) key calls—and the comparison-based sort does O(n log n) comparisons. Space: sorted() allocates a new list—O(n) extra space. list.sort() sorts in place but Timsort uses O(n) auxiliary space for the merge step. So both are O(n log n) time, O(n) space; sort() avoids a second list.

Edge Cases

• Empty iterable: sorted([]) → []; [].sort() does nothing. Both are safe.
• Single element: Returns or leaves the list with one element; no error.
• Duplicates: All kept; order among equal elements is stable (same as input).
• Mixed types: In Python 3, comparing incompatible types (e.g., int vs str) raises TypeError. Ensure elements are comparable or supply a key that returns a comparable type (e.g., all numbers or all strings).

Sorting Custom Objects

By default, objects are compared by identity (or by __lt__ if defined). To sort by an attribute, use key=lambda obj: obj.attr. For multiple attributes, key=lambda obj: (obj.a, obj.b). You can also define __lt__ (and optionally other rich comparison methods) on the class so the object is directly comparable; then sorted(list_of_objs) works without key. For one-off sorts, key is usually simpler.

class Person:
    def __init__(self, name, age):
        self.name, self.age = name, age
people = [Person("Bob", 30), Person("Alice", 25)]
sorted(people, key=lambda p: p.age)   # by age
sorted(people, key=lambda p: (p.age, p.name))  # by age, then name

Common Mistakes

• Expecting list.sort() to return the list: It returns None. Write arr.sort() and then use arr; don’t assign the return value.
• Sorting in place when you need the original: If you still need the unsorted list, use sorted(arr) and assign to a new variable, or copy first: arr_copy = arr.copy(); arr_copy.sort().
• Wrong multi-criteria order: Remember that key=lambda x: (x.a, x.b) sorts by a first, then b. For descending on one field, use negation (numbers) or a key that inverts order.

Summary

• sorted(iterable) returns a new sorted list; list.sort() sorts in place and returns None.
• Use key=function to sort by a field, computed value, or tuple for multiple criteria; use reverse=True for descending.
• Python’s sort is stable and O(n log n) time, O(n) space; choose sorted() vs sort() based on whether you need to keep the original.

2.13 Time Complexity of Built-ins

Introduction

To analyze your algorithms you need to know the cost of the operations you use. Python's built-in types have specific time complexities: list index is O(1), but x in list is O(n); dict and set lookup are O(1) average; string concatenation in a loop is O(n²). This section summarizes the main time complexities of built-in operations so you can reason correctly about your overall complexity and avoid hidden bottlenecks.

Why This Matters for DSA

If you use in on a list inside a loop, you may turn an O(n) idea into O(n²). If you use a set for membership instead, you stay O(n). Choosing the right structure and knowing the cost of each call (index, append, insert, in, sort, etc.) lets you state and optimize complexity accurately in interviews and in code.

List

Dict and Set

Average case assumes a good hash function and no excessive collisions. Worst case (many collisions) can degrade toward O(n) per operation, but in practice the average case is the one to use for analysis.

String

Repeated s += c in a loop does a new concatenation each time (copy of growing string), so total is O(n²). Use ''.join(parts) for O(n).

Deque (collections.deque)

append, appendleft, pop, popleft are O(1). Indexing d[i] in the middle is O(n). Use when you need O(1) at both ends (queue, sliding window).

Summary

• List: index/append/pop at end O(1) (append amortized); insert/pop at front or middle, in, index/count O(n); sort O(n log n).
• Dict / Set: get, set, delete, in O(1) average; iteration O(n).
• String: index O(1), slice O(k); in O(n); avoid repeated +=, use ''.join().

2.14 collections Module

Introduction

The collections module provides high-performance alternatives and extensions to built-in types. For DSA the most used are: deque (double-ended queue with O(1) append/pop at both ends), Counter (count hashable elements and get frequencies), and defaultdict (dict with a default value for missing keys). Knowing when to use each keeps your code simple and efficient.

deque (Double-Ended Queue)

collections.deque is a sequence that supports O(1) append, appendleft, pop, and popleft. Use it instead of a list when you need to add or remove at both ends (e.g., BFS queue, sliding window from both sides). Indexing in the middle is O(n), so use for queue/stack style access, not random access.

from collections import deque

q = deque()
q.append(1)
q.append(2)
q.appendleft(0)   # [0, 1, 2]
x = q.popleft()   # 0, q is [1, 2]
y = q.pop()       # 2, q is [1]

BFS pattern: q = deque([start]), then while q: node = q.popleft(); ...; q.append(neighbor). This is the standard queue for level-order traversal and shortest path in unweighted graphs.

Counter

Counter counts hashable elements. Give it an iterable (list, string) and it returns a dict-like object: keys are elements, values are counts. in, [], and iteration work like dict; missing keys return 0 (no KeyError).

from collections import Counter

c = Counter([1, 2, 2, 3, 3, 3])   # Counter({3: 3, 2: 2, 1: 1})
c = Counter("hello")              # Counter({'l': 2, 'h': 1, 'e': 1, 'o': 1})
c["x"]   # 0 (missing key)
c.most_common(2)   # [(l, 2), (h, 1)] — 2 most common
c.subtract(Counter("he"))   # subtract counts in place

Building a frequency map from an array: freq = Counter(arr) is one line and O(n). Use most_common(k) for “top k frequent” and elements() to iterate with repetition. For anagrams, Counter(s1) == Counter(s2) is a clean O(n) check.

defaultdict

defaultdict(factory) is a dict that calls factory() when a key is missing. defaultdict(int) gives 0 for missing keys—so d[x] += 1 works without get. defaultdict(list) gives []—so d[key].append(item) groups by key. Same O(1) average as dict.

from collections import defaultdict

freq = defaultdict(int)
for x in arr:
    freq[x] += 1   # no KeyError

groups = defaultdict(list)
for item in items:
    groups[item.key].append(item)

When to Use Which

• deque: Queue (BFS), stack with occasional left pop, sliding window that shrinks from the left.
• Counter: Frequency count from iterable, “top k frequent,” anagram check, subtract/add counts.
• defaultdict: Frequency (int) or “group by key” (list) when you want to avoid get or explicit key checks.

Summary

• deque: O(1) append/pop at both ends; use for BFS queue and when you need operations at both ends.
• Counter: Count elements from iterable; most_common(k), subtract; ideal for frequency and anagram problems.
• defaultdict: Dict with default value for missing keys; use for counts (int) or grouping (list).

2.15 heapq

Introduction

The heapq module provides a min-heap implementation on top of a list. The smallest element is always at index 0; after you pop it, the heap reorganizes so the next smallest is at the top. Operations heappush, heappop, and heapify are O(log n). There is no separate heap type—you use a list and call heapq functions so the list satisfies the heap invariant. For DSA, heaps are used for “top K,” “merge K sorted lists,” and priority queues (e.g., Dijkstra).

Why heapq Matters for DSA

When you need repeated “smallest (or largest) element” or “smallest among many candidates,” a heap gives O(log n) insert and O(log n) extract-min. That’s better than sorting the whole list each time (O(n log n)) or scanning for the min (O(n)). Top K elements, merge K sorted lists, and Dijkstra’s algorithm all rely on this “get min, add new candidates” pattern.

Basic API

• heapq.heapify(lst) — Turn list lst into a min-heap in place. O(n). After this, lst[0] is the smallest element.
• heapq.heappush(heap, x) — Add x to the heap, keeping the heap invariant. O(log n).
• heapq.heappop(heap) — Remove and return the smallest element. O(log n).
• heapq.heappushpop(heap, x) — Push x then pop the smallest. More efficient than push + pop when you’re replacing the min.
• heapq.nsmallest(k, iterable) / heapq.nlargest(k, iterable) — Return the k smallest or k largest. Useful for one-off “top k”; for streaming or repeated updates, maintain a heap yourself.

import heapq

arr = [3, 1, 4, 1, 5]
heapq.heapify(arr)   # arr is now a min-heap; arr[0] == 1
x = heapq.heappop(arr)   # 1, heap size reduced
heapq.heappush(arr, 0)   # add 0; 0 becomes new min
smallest = arr[0]        # 0 — peek without popping (don't pop yet)

Min-Heap vs Max-Heap

heapq implements a min-heap only. For a max-heap, negate values: push -x, and when you pop, negate again to get the original maximum. Or use key with a wrapper: store (-priority, item) so that “largest priority” becomes “smallest -priority” and pops first.

# Max-heap: negate
heapq.heappush(heap, -x)   # push negative
max_val = -heapq.heappop(heap)   # pop and negate back

Time Complexity

heapify: O(n). heappush, heappop: O(log n) each. Peeking the min is O(1) (heap[0]). So “push n elements then pop n times” is O(n log n). “Top k from n” with a size-k heap: O(n log k)—push each of n elements, heap never exceeds size k.

Example: Top K Elements

Keep a min-heap of size k. For each element: if heap has fewer than k elements, push it; else if the new element is larger than the heap’s minimum, pop the min and push the new one. At the end, the heap contains the K largest. To get them in order, pop repeatedly (smallest of the K first) or use nsmallest on the heap. Alternatively, push all with negated values for “K largest” and pop k times from a max-heap (negated) to get largest first.

def top_k_largest(arr, k):
    heap = arr[:k]
    heapq.heapify(heap)   # min-heap of first k
    for x in arr[k:]:
        if x > heap[0]:
            heapq.heapreplace(heap, x)   # pop min, push x (or heappop + heappush)
    return heap   # k largest (unordered in heap; sort if needed)

heapq.heapreplace(heap, x) pops the smallest and then pushes x—equivalent to heappop then heappush, but slightly more efficient.

Summary

• heapq gives a min-heap on a list: heapify, heappush, heappop; min is always at heap[0].
• For a max-heap, negate values when pushing and when popping.
• Use for top K, merge K sorted lists, and priority-queue algorithms (e.g., Dijkstra).

2.16 bisect

Introduction

The bisect module provides binary search on sorted lists. It finds the position where an element would be inserted to keep the list sorted, or where an existing element sits. All operations are O(log n). Use it when you have a sorted sequence and need “find index of x” or “insert x and keep sorted” without writing binary search by hand.

Why bisect Matters for DSA

Binary search on a sorted array is a core pattern: search for a value, find the first position ≥ x, or the last position ≤ x. bisect_left and bisect_right give you these insertion points; from that you can check “is x present?” or “how many elements are < x?”. Many problems (search in rotated array, lower/upper bound, binary search on answer) build on the same idea—bisect is the standard library way to get the indices right.

Main Functions

• bisect.bisect_left(a, x) — Leftmost index where x can be inserted so the list stays sorted. If x is already in a, returns the index of the first occurrence. So “is x in a?” → i = bisect_left(a, x); i < len(a) and a[i] == x.
• bisect.bisect_right(a, x) (alias bisect.bisect(a, x)) — Rightmost index where x can be inserted. If x is in a, one past the last occurrence. So “count of elements ≤ x” → bisect_right(a, x); “count of elements < x” → bisect_left(a, x).
• bisect.insort_left(a, x) — Insert x in a at the position that keeps order (before equal elements). O(n) for the shift.
• bisect.insort_right(a, x) (alias bisect.insort(a, x)) — Insert x after equal elements. O(n) for the shift.

import bisect

a = [1, 2, 2, 3, 4]
bisect.bisect_left(a, 2)   # 1 — first position for 2
bisect.bisect_right(a, 2) # 3 — one past last 2
bisect.bisect_left(a, 5)  # 5 — would go at end
# Is 2 in a? i = bisect_left(a, 2); i < len(a) and a[i] == 2  → True

Left vs Right

Use bisect_left when you want “first index ≥ x” or “where to insert so x is before equals.” Use bisect_right when you want “first index > x” (one past last x) or “count of elements ≤ x” (the return value is that count when the list is 0-indexed and sorted). For “lower bound” (first ≥ x): bisect_left. For “upper bound” (first > x): bisect_right.

Time Complexity

All bisect functions are O(log n) for the search. insort is O(n) because it shifts elements to make room. So use bisect for search; use insort only when you occasionally need to maintain a sorted list with inserts (e.g., small dynamic set). For many inserts, consider a structure that supports O(log n) insert (e.g., sorted container from a library) or batch sort.

Summary

• bisect_left(a, x) = leftmost insertion point (first ≥ x); bisect_right(a, x) = rightmost (first > x). List must be sorted.
• Use for “is x in sorted list?,” “count in range [L, R],” lower/upper bound. O(log n) search; insort is O(n).

2.17 itertools

Introduction

The itertools module provides iterator building blocks: combinations, permutations, Cartesian product, chaining iterables, grouping, and more. These are lazy (one element at a time) and memory-efficient. For DSA you’ll use them for “all subsets of size k,” “all permutations,” “all pairs from two lists,” and similar enumeration tasks without writing nested loops or recursion by hand.

Why itertools Matters for DSA

Problems that ask for “all combinations,” “all permutations,” or “iterate over pairs (i, j)” can be done with itertools.combinations, permutations, or product. They’re correct, readable, and avoid off-by-one errors. For backtracking you often write your own recursion—but when the problem is “enumerate all,” the standard iterators are a quick and correct option (and you can wrap them in list() if you need the full list, though that uses O(n) space).

Combinations and Permutations

• itertools.combinations(iterable, r) — All subsequences of length r in iterable order; no repeats, (a,b) same as (b,a).
• itertools.combinations_with_replacement(iterable, r) — Same but elements can repeat (e.g., (1,1), (1,2), (2,2)).
• itertools.permutations(iterable, r=None) — All orderings of length r (default: full length). (a,b) and (b,a) both appear.

import itertools

list(itertools.combinations([1,2,3], 2))   # [(1,2), (1,3), (2,3)]
list(itertools.permutations([1,2], 2))      # [(1,2), (2,1)]
list(itertools.combinations_with_replacement([1,2], 2))  # [(1,1),(1,2),(2,2)]

Product

itertools.product(*iterables, repeat=1) — Cartesian product: all pairs (or tuples) from the given iterables. product(A, B) is like nested loops “for a in A: for b in B.” With repeat=k, one iterable is used k times (e.g., all k-tuples from a set of choices).

list(itertools.product([1,2], ['a','b']))  # [(1,'a'),(1,'b'),(2,'a'),(2,'b')]
list(itertools.product([0,1], repeat=3))   # all 3-bit: (0,0,0)..(1,1,1)

Chain and Slice

• itertools.chain(*iterables) — Iterate over the first iterable, then the second, and so on. Flattens one level. chain.from_iterable(iterables) takes one iterable of iterables.
• itertools.islice(iterable, start, stop, step) — Slice an iterator (like list slice but lazy). No negative indices.

list(itertools.chain([1,2], [3,4]))   # [1, 2, 3, 4]
list(itertools.islice(range(10), 2, 6))   # [2, 3, 4, 5]

groupby

itertools.groupby(iterable, key=None) — Group consecutive elements that have the same key. The iterable should be sorted by the key for meaningful groups. Yields (key, group_iterator) pairs. Useful for “run-length” style grouping.

# Consecutive equal elements (sort first if needed)
for k, g in itertools.groupby([1,1,2,2,2,3]):
    print(k, list(g))   # 1 [1,1], 2 [2,2,2], 3 [3]

Other Useful Tools

• itertools.cycle(iterable) — Repeat the iterable forever (use with islice to limit).
• itertools.repeat(x, times=None) — Yield x repeatedly (or times times).
• itertools.count(start=0, step=1) — Infinite counter.

Summary

• combinations / permutations / product — enumerate subsets, orderings, or Cartesian product; lazy iterators.
• chain, islice, groupby — combine or slice iterators, group consecutive keys.
• Use itertools for “all combinations/permutations/pairs” to avoid manual loops and keep code clear.

2.18 functools

Introduction

The functools module provides tools for higher-order functions: caching, partial application, and reduction. For DSA the most important is lru_cache—a decorator that memoizes function results by argument (with an optional size limit). It turns a recursive solution with repeated subproblems into an efficient one without writing a cache by hand. Other tools like partial and reduce are useful in general Python but less central to algorithm implementation.

Why functools Matters for DSA

Recursive solutions often recompute the same arguments (e.g., Fibonacci, many DP problems). Memoization stores the return value for each argument so the second time you call with the same args you get the cached value. @lru_cache does this automatically: add the decorator and ensure arguments are hashable (e.g., use tuples instead of lists for state). Then the recursive function runs in time proportional to distinct argument values instead of exploding exponentially.

lru_cache

functools.lru_cache(maxsize=128, typed=False) — Decorator that caches the most recent results of a function. Arguments must be hashable (so use tuples, not lists, for composite state). Call the function with the same arguments again and you get the cached result (O(1) lookup) instead of recomputing.

from functools import lru_cache

@lru_cache(maxsize=None)   # unbounded cache
def fib(n):
    if n <= 1:
        return n
    return fib(n - 1) + fib(n - 2)

fib(100)   # fast — each n computed once

Without the decorator, fib(n) would be exponential. With it, each distinct n is computed once; subsequent calls with the same n return the cached value. Use maxsize=None for unbounded cache (all distinct argument tuples stored). Use a number (e.g., 1000) to limit memory; LRU evicts least recently used entries when full.

Hashable Arguments Only

The cache uses arguments as dict keys, so they must be hashable. If your state includes a list, convert it to a tuple for the call: f(tuple(arr), i). For multiple arguments, lru_cache treats the whole (arg1, arg2, ...) as the key.

partial

functools.partial(func, *args, **kwargs) — Returns a new callable that fixes some arguments of func. Useful when you need to pass a function that takes one argument (e.g., to sorted or map) but your function takes two—fix the second with partial.

from functools import partial
def power(base, exp):
    return base ** exp
square = partial(power, exp=2)
square(5)   # 25

reduce

functools.reduce(function, iterable, initializer=None) — Apply a two-argument function cumulatively: function(function(...(function(initial, first), second), ...), last). Classic example: reduce(lambda a, b: a + b, [1,2,3], 0) → 6. For “fold” or “reduce a sequence to one value” it’s built-in; for readability, a simple loop is often clearer. Use when it fits the problem (e.g., product of a list: reduce(operator.mul, arr, 1)).

Summary

• lru_cache — Memoization decorator; use for recursive/DP functions with hashable args to avoid recomputation.
• partial — Fix some arguments of a function; useful for callbacks. reduce — Fold an iterable to a single value.
• For DSA, lru_cache is the main tool; ensure arguments are hashable (e.g., tuple, int).

2.19 Python Memory Management (Reference Counting & GC)

Introduction

Python manages memory mainly through reference counting: each object keeps a count of how many references point to it. When that count drops to zero, the object is reclaimed immediately. A garbage collector (GC) also runs to break reference cycles (e.g., A refers to B, B refers to A) that reference counting alone cannot free. For DSA you rarely tune this—but understanding “variables are references” and “when is an object freed?” helps you reason about aliasing, copies, and space.

Why This Matters for DSA

You don’t usually optimize Python memory at the level of refcounts. What matters is: assignment doesn’t copy objects (a = b means both names point to the same object); mutating that object affects all names; when you need a copy, use .copy() or list() or slicing. Knowing that “no more references = object can be freed” explains why a big structure can be garbage-collected when you leave the only reference to it (e.g., reassign the variable or return from a function and drop the local reference).

Reference Counting

Every object has a reference count: how many names, containers, or other objects point to it. When you assign x = [1, 2], the list’s refcount is 1. When you do y = x, it becomes 2—same object, two names. When y is reassigned or goes out of scope, the count drops. When it reaches 0, Python frees the object (and any objects it alone referenced). This is automatic and immediate for non-cyclic structures.

import sys
a = [1, 2, 3]
sys.getrefcount(a)   # 2 (a + the argument to getrefcount)
b = a
sys.getrefcount(a)   # 3
del b
# refcount drops; when a is no longer used, list is freed

getrefcount is for curiosity; the exact number includes the temporary reference from passing a to the function. The idea: more references = higher count; when all references are gone, the object is reclaimed.

Cycles and the Garbage Collector

If object A holds a reference to B and B holds a reference to A, their refcounts never become 0 even if no external reference exists. Python’s cycle detector (in the gc module) periodically finds such cycles and reclaims them. So you can have circular references (e.g., a graph node pointing to neighbors) and they will still be collected when unreachable. You normally don’t need to call gc.collect(); the runtime runs it when needed.

Implications for Your Code

• Aliasing: a = b does not copy; both refer to the same object. Changes via a are visible via b. For DSA this is why “pass by reference” for lists matters: mutating inside a function affects the caller’s list.
• When objects are freed: When the last reference disappears (variable reassigned, scope left, container holding the reference cleared), the object becomes eligible for reclamation. So a large local list is freed when the function returns (assuming you don’t return it or store it elsewhere).
• No explicit free: You don’t manually free memory; dropping references is enough. To release a large structure early, delete the name (del big_list) or reassign it so nothing else points to the object.

Summary

• Python uses reference counting plus a garbage collector for cycles. When refcount hits 0 (and no cycle keeps it alive), the object is reclaimed.
• Assignment copies the reference, not the object—so aliasing and mutation matter. Use explicit copy when you need an independent copy.
• For DSA, this backs your mental model of “pass by reference” and when big structures get freed (when no reference remains).

2.20 Internals of list & dict (Dynamic Resizing & Amortized Analysis)

Introduction

Lists and dicts in Python grow dynamically: they allocate more memory when needed instead of fixing a size upfront. That growth is done in chunks (list) or by resizing the hash table (dict), so individual operations stay fast on average. Understanding dynamic resizing and amortized analysis explains why we say list.append is O(1) amortized and dict get/set is O(1) average—and why the occasional "expensive" resize doesn't make the whole sequence slow.

Why This Matters for DSA

When we state "append is O(1) amortized," we mean: over n appends, total time is O(n), so average per append is O(1). You don't need to implement resizing yourself—but knowing that lists over-allocate and resize in steps (and that dicts resize when load factor gets high) lets you trust the complexity we use in analysis and avoid micro-optimizations like preallocating list size "to be safe."

List: Dynamic Array

A list is implemented as a dynamic array: a contiguous block of pointers to objects. When you append and the current block is full, Python allocates a larger block, copies the pointers over, and frees the old one. It doesn't grow by one slot each time—it uses a growth strategy (in CPython, the new capacity is computed so that the sequence of appends does O(n) total work). So most appends just write into an existing empty slot (O(1)); occasionally one triggers a resize (O(current size)), but that cost is "spread" over the next many cheap appends. That's amortized O(1) per append.

Dict: Hash Table and Resizing

A dict is a hash table: an array of buckets, each holding key-value entries. On get/set, the key is hashed and the bucket is found; collisions are handled (e.g., by probing or chaining). When the table gets too full (high load factor = number of entries / number of buckets), the table is resized (e.g., doubled), all entries are rehashed into the new table. That resize is O(n), but it happens only when the load factor crosses a threshold, so over many insertions the average cost per insertion stays O(1). Deletes don't shrink the table immediately in CPython; the table can grow and stay large. So we say get/set/delete are O(1) average (assuming a good hash function and normal load).

Amortized Analysis Intuition

For a list: suppose every time we double the capacity we do work proportional to the current size. So we do 1 + 2 + 4 + 8 + … + (capacity at some point) units of copy work. That sum is less than 2 × final size. So for n appends, total copy work is O(n), hence O(1) per append on average. The "expensive" resizes are rare and their cost is amortized over the many cheap appends that follow. Same idea for dict: occasional O(n) resize, but averaged over all insertions, each insertion is O(1).

Summary

• List = dynamic array; grows in chunks so append is O(1) amortized (total work for n appends is O(n)).
• Dict = hash table; resizes when load factor is high so get/set are O(1) average.
• Amortized = total cost of n operations is O(n), so average per operation is O(1). Occasional expensive resize is spread over many cheap operations.

2.21 Object Internals (slots, is vs ==)

Introduction

Two concepts that come up in Python object model: is vs == (identity vs value equality), and __slots__ (restricting instance attributes to save memory and speed attribute access). For DSA you'll mostly care about is vs ==—especially using is None instead of == None by convention—and rarely __slots__ unless you create huge numbers of small objects.

is vs ==

is checks identity: do two names refer to the exact same object in memory? == checks value equality: do the objects compare equal (via __eq__)? Two different list instances with the same elements are == but not is. The same object is both is and ==.

a = [1, 2, 3]
b = [1, 2, 3]
a == b   # True  — same values
a is b   # False — different objects

c = a
a is c   # True  — same object

Use is None (and is not None) to check for None—it's the conventional and correct way, because there's only one None object. Using == None works but can be overridden by a custom __eq__; is None cannot. For other values, use == when you care about equality of content.

__slots__

By default, each instance of a class has a __dict__ that stores its attributes—flexible but uses extra memory per instance. __slots__ is a class attribute that lists the only attribute names the instances are allowed to have. The class then uses a fixed-size structure (like a tuple) instead of a dict for those attributes, saving memory and giving slightly faster attribute access. You can't add new instance attributes beyond those in __slots__.

class Point:
    __slots__ = ('x', 'y')
    def __init__(self, x, y):
        self.x, self.y = x, y

p = Point(1, 2)
# p.z = 3   # AttributeError — not in __slots__

Use __slots__ when you have many small instances (e.g., graph nodes, events) and want to reduce memory. For DSA it's an optimization, not required for correctness.

Summary

• is = identity (same object); == = value equality. Use is None / is not None for None checks.
• __slots__ = fixed set of instance attributes; saves memory and speeds access when you have many small objects.

2.22 Concurrency Basics (GIL & Asyncio Overview)

Introduction

Python has a Global Interpreter Lock (GIL): only one thread can execute Python bytecode at a time in a single process. So multithreading doesn't give you parallel execution of CPU-bound Python code—threads take turns. For I/O-bound work (network, disk), threads can still help because the lock is released while waiting. Asyncio is a different model: cooperative concurrency with async/await, one thread, many tasks that yield during I/O. For DSA and algorithm interviews you rarely need concurrency; this section is an overview so you know the landscape.

The GIL (Global Interpreter Lock)

The GIL is a mutex that protects access to Python objects. Only one thread holds it at a time, so only one thread runs Python bytecode at a time. That simplifies the interpreter (no fine-grained locking on every object) but means:

• CPU-bound: Multiple threads running pure Python computation don't run in parallel—they serialize. To use multiple CPU cores for CPU-bound work, use multiprocessing (separate processes, each with its own interpreter and GIL) or offload to C extensions that release the GIL.
• I/O-bound: While a thread is waiting for I/O (network, file), it typically releases the GIL. So other threads can run. Multithreading can still speed up I/O-bound programs (e.g., many simultaneous HTTP requests) because waiting is overlapped.

Asyncio Overview

Asyncio provides cooperative concurrency: you define coroutines with async def and await other coroutines or I/O operations. One thread runs an event loop; when a coroutine hits await (e.g., waiting for a socket), the loop can run another coroutine. So many I/O operations can be in flight without blocking the thread—useful for servers handling many connections or scripts making many HTTP requests. It's not parallelism (one thread); it's concurrency (overlapping I/O wait).

Key ideas: async def defines a coroutine; await yields until the awaited thing completes; asyncio.run(main()) runs the top-level coroutine. For DSA you don't need to write async code; just know that asyncio is for I/O-bound concurrency on one thread, not for making CPU-bound algorithms faster.

When to Use What

• Single-threaded: Default for DSA, scripts, most interview code.
• Multithreading: I/O-bound work where you want to overlap waits (e.g., many URLs); limited by GIL for CPU.
• Multiprocessing: CPU-bound work to use multiple cores; no GIL sharing (each process has its own).
• Asyncio: I/O-bound, many concurrent operations; one thread, cooperative scheduling.

Summary

• The GIL allows only one thread to run Python bytecode at a time; multithreading doesn't parallelize CPU-bound Python code.
• Asyncio = cooperative concurrency with async/await; good for I/O-bound, one thread.
• For DSA and interviews, single-threaded code is the norm; concurrency is for system design and I/O-heavy applications.

3.1 Big-O Notation

Introduction

Big-O notation describes how the time or space used by an algorithm grows as the input size grows. We use it to compare algorithms and to state guarantees: "this runs in O(n) time" means the running time is at most proportional to n (for large n), up to a constant factor. Big-O is the language of complexity analysis in interviews and in practice—you must be able to state it, derive it from code, and compare two Big-O expressions.

Why Big-O Matters

We care about growth rate, not the exact number of nanoseconds. When n doubles, does the time double (O(n)), quadruple (O(n²)), or stay roughly the same (O(1))? That tells us whether an algorithm will scale. Problem constraints (e.g., n ≤ 10⁵) combined with Big-O tell you if a solution will pass (e.g., O(n log n) is fine, O(n²) might be too slow). In interviews, you're expected to state complexity and justify it.

Formal Definition (Intuitive)

We say T(n) is O(g(n)) if, for large enough n, T(n) is at most a constant multiple of g(n). In symbols: there exist constants c > 0 and n₀ such that for all n ≥ n₀, T(n) ≤ c · g(n). So we ignore constant factors and lower-order terms: 5n + 3 is O(n), 2n² + n is O(n²). Big-O is an upper bound: "no worse than this growth."

Common Complexity Classes

From best to worst (for large n):

• O(1) — Constant. Same cost regardless of n (e.g., hash lookup, array index).
• O(log n) — Logarithmic. Doubling n adds a constant amount of work (e.g., binary search, balanced tree operations).
• O(n) — Linear. Doubling n doubles the work (e.g., one pass over an array).
• O(n log n) — Linearithmic. Typical for efficient sorting (e.g., merge sort, heapsort).
• O(n²) — Quadratic. Nested loops over n (e.g., two loops over the array).
• O(n³), O(2ⁿ), O(n!) — Higher; often too slow for large n unless the problem size is tiny.

How to Derive Big-O from Code

1. Count basic steps (or representative operations) as a function of input size n.
2. Drop constant factors (e.g., 3n → n).
3. Keep the dominant term (the one that grows fastest as n grows). So 5n + 10 → O(n); n² + n → O(n²).

Single loop that does O(1) work per iteration → O(n). Two nested loops, each over n → O(n²). Loop that halves n each time → O(log n). Recursion with one call per level and n levels → O(n); with two calls per level and n levels (like naive Fibonacci) → exponential.

# O(n): one loop
for i in range(len(arr)):
    process(arr[i])

# O(n²): two nested loops
for i in range(n):
    for j in range(n):
        do_something()

Summary

• Big-O = upper bound on growth rate; we ignore constants and lower-order terms.
• Common: O(1), O(log n), O(n), O(n log n), O(n²); know what they mean and when they arise.
• Derive from code by counting steps, dropping constants, keeping the dominant term.

3.2 Theta & Omega Notation

Introduction

Big-O gives an upper bound: "the algorithm is no worse than this growth." Sometimes we need a lower bound ("at least this much work") or a tight bound ("exactly this growth, up to constants"). Omega (Ω) is the lower bound; Theta (Θ) means "both upper and lower bound"—the growth is tightly characterized. Together, O, Ω, and Θ are the standard asymptotic notation; in interviews you'll mostly use O, but knowing Θ and Ω helps you state "optimal" precisely and understand lower-bound arguments.

Why This Topic Matters

When we say "this algorithm is optimal," we often mean its running time is Θ(f(n)) and that any algorithm for the problem must take Ω(f(n))—so we've matched the lower bound. For example, comparison-based sorting is Ω(n log n); merge sort is O(n log n), so merge sort is optimal (Θ(n log n) for that problem). Omega is also used in proofs: "you must look at every element at least once, so the algorithm is Ω(n)." Theta is the right way to say "the complexity is exactly this order of growth."

Omega (Ω): Lower Bound

We say T(n) is Ω(g(n)) if, for large enough n, T(n) is at least a constant multiple of g(n). Formally: there exist constants c > 0 and n₀ such that for all n ≥ n₀, T(n) ≥ c · g(n). So Ω describes a lower bound: "the algorithm does at least this much work." Example: finding the maximum in an unsorted array by comparison is Ω(n), because you must look at every element (otherwise an unseen element might be the max). So any correct algorithm is at least linear.

Theta (Θ): Tight Bound

We say T(n) is Θ(g(n)) if T(n) is both O(g(n)) and Ω(g(n)). So for large n, T(n) is sandwiched between two constant multiples of g(n): c₁·g(n) ≤ T(n) ≤ c₂·g(n) for some c₁, c₂ > 0 and n ≥ n₀. That means the growth rate is exactly g(n), up to constants—no faster, no slower. Example: merge sort does Θ(n log n) comparisons—we can prove both O(n log n) and Ω(n log n) for comparison-based sorting on n elements, so Θ(n log n) is the tight bound.

Relationship: T(n) = Θ(g(n)) if and only if T(n) = O(g(n)) and T(n) = Ω(g(n)). So when you've proved both an upper and a lower bound with the same g(n), you write Θ(g(n)).

Comparison of O, Ω, and Θ

Examples

• Linear scan to find max: We do n−1 comparisons → O(n). We must touch every element → Ω(n). So the algorithm is Θ(n).
• Binary search (success): Each step halves the range → O(log n) comparisons. We need at least log₂ n steps to narrow from n to 1 → Ω(log n). So Θ(log n).
• Bubble sort: O(n²) (nested loops). It can be Ω(n²) in the worst case (e.g., reverse order), so worst case is Θ(n²). Best case (already sorted) is O(n) but not Θ(n²)—Theta is for a specific scenario (e.g., worst case).

When to Use Which

• O: Most common. "My algorithm runs in O(n²) time." Safe and correct as long as you've proved an upper bound.
• Ω: When you're proving "you can't do better" or "at least this much work is required." Used in lower-bound proofs and to justify optimality.
• Θ: When you've proved both upper and lower bounds with the same growth. "Merge sort is Θ(n log n) in the comparison model." More precise than O when you know the bound is tight.

Summary

• Ω(g(n)) = lower bound: T(n) ≥ c·g(n) for large n. "At least this much work."
• Θ(g(n)) = tight bound: T(n) = O(g(n)) and T(n) = Ω(g(n)). "Exactly this growth."
• Use O for upper bounds (most common); use Ω for lower-bound proofs; use Θ when you have both and they match.

3.3 Best / Worst / Average Case

Introduction

So far we've talked about Big-O, Theta, and Omega as ways to describe how an algorithm's cost grows with input size. But the same algorithm can take different amounts of time depending on what the input looks like. Linear search might find the target in the first cell (one step) or in the last (n steps). That's why we distinguish best case, worst case, and average case: they tell us how the algorithm behaves under different input scenarios. Master this and you'll know exactly what to say in interviews and how to choose algorithms in practice.

Real-World Analogy

Imagine searching for your keys at home. Best case: they're on the table by the door—you find them in one look. Worst case: they're in the last drawer you check, after searching every room. Average case: sometimes they're in the first place, sometimes the middle, sometimes the last—over many days, you do "about half" the possible search. The method (where you look) is the same; only the input (where the keys actually are) changes the cost. Algorithms work the same way: same code, different inputs → different runtimes.

Formal Definitions

Best Case

The best case is the scenario (or type of input) that minimizes the algorithm's work. For a given input size n, it's the minimum number of steps (or comparisons, or operations) the algorithm can take over all valid inputs of that size. We express it with Big-O (or Theta when the bound is tight) and call it "best-case time complexity."

• Example: Linear search in an array of size n. Best case = target is at index 0 → 1 comparison → O(1) best case.

Worst Case

The worst case is the scenario that maximizes the algorithm's work. For input size n, it's the maximum number of steps over all valid inputs. We almost always care about worst case when we say "time complexity" because it's a guarantee: the algorithm will never do more than this, no matter how unlucky the input.

• Example: Linear search, target not present or at last index → n comparisons → O(n) worst case.

Average Case

The average case is the expected (average) number of steps over some distribution of inputs—usually we assume all inputs of size n are equally likely, or we use a probability distribution that matches reality. Average case is harder to define and analyze than best/worst, but it often reflects "typical" performance.

• Example: Linear search with target equally likely at any index (1 to n) or absent. If present: average position ≈ n/2 → about n/2 comparisons. Often we still say O(n) average, since the constant factor doesn't change the growth class.

Why This Topic Matters

In interviews and in production, you need to know: (1) Worst case—so you can guarantee latency and avoid surprises. (2) Best case—so you know when an algorithm can be very fast (e.g., early exit). (3) Average case—when inputs are random or typical, average often predicts real behavior. Many algorithms (e.g., quicksort) have a bad worst case but a good average case; choosing them means accepting that worst case in exchange for speed on average.

Mental Model

Think of the algorithm as a fixed procedure. For each input size n, imagine all possible inputs of that size. Each input leads to some number of steps. Best case = minimum over those inputs; worst case = maximum; average case = average (with a defined distribution). The three cases can have different Big-O classes (e.g., best O(1), worst O(n), average O(n)).

Step-by-Step: Analyzing an Algorithm for All Three Cases

1. Identify the "cost" you're measuring (e.g., comparisons, swaps, or total operations).
2. Ask: what input minimizes this cost? That gives the best case; count steps and express in Big-O.
3. Ask: what input maximizes this cost? That gives the worst case.
4. For average case, define a distribution over inputs (e.g., uniform), compute expected cost, then simplify to Big-O.

Example: Linear Search

Problem: find index of target in list arr, or return -1 if not present.

def linear_search(arr, target):
    for i in range(len(arr)):
        if arr[i] == target:
            return i
    return -1

• Best case: target at index 0. One comparison, then return. → O(1).
• Worst case: target at last index or absent. n comparisons. → O(n).
• Average case (target in array, uniform position): Position k (1-based) with probability 1/n; comparisons = k. Expected comparisons = (1 + 2 + … + n) / n = n(n+1)/(2n) = (n+1)/2 → Θ(n).

Example: Binary Search (on sorted array)

We repeatedly compare with the middle and discard half. Best case: target is the middle element → 1 comparison → O(1). Worst case: target absent or at a leaf of the "decision tree"—we halve until size 1 → about log₂ n comparisons → O(log n). Average case (assuming target equally likely in any position): also on the order of log n → O(log n). So for binary search, best is O(1), worst and average are O(log n).

Example: Bubble Sort

Compare adjacent pairs and swap if out of order; repeat until no swaps. Best case: array already sorted → one pass, no swaps, early exit possible → O(n). Worst case: array reverse sorted → every pass does maximum swaps, O(n) passes → O(n²). Average case: random order → still O(n²) comparisons and swaps on average. So we say: best O(n), worst O(n²), average O(n²).

ASCII Diagram: Where Do the Cases Come From?

Input size n = 5. "Cost" = number of comparisons (e.g., linear search).

  Input:   [T, ?, ?, ?, ?]  [?, ?, ?, ?, T]  [?, T, ?, ?, ?]  ...
           (target at 0)    (target at 4)    (target at 1)

  Cost:        1                 5                 2             ...

  Best case  = min(1, 5, 2, ...) = 1   →  O(1)
  Worst case = max(1, 5, 2, ...) = 5   →  O(n)
  Average    = (1+5+2+...)/#inputs     →  O(n)

Comparison Table: Best / Worst / Average

Edge Cases and Assumptions

• Empty input: Often O(1) or a single check. Sometimes counted separately or as part of best case.
• Average case depends on distribution: If keys are usually at the start, "average" for linear search is better than n/2. We usually assume uniform or "random" when we say "average."
• Worst case can be rare: Quicksort's O(n²) worst case happens for specific inputs (e.g., already sorted with first-element pivot); random shuffling makes it unlikely.

Interview Insight

Summary

• Best case = input that minimizes work; worst case = input that maximizes work; average case = expected work over a chosen input distribution.
• Same algorithm can have different O(·) for each case (e.g., linear search: O(1) best, O(n) worst and average).
• Default "time complexity" usually means worst case. State best/worst/average explicitly when it matters.
• Analyze by asking: what input minimizes/maximizes cost? Then express each with Big-O (or Θ when tight).

3.4 Loop & Nested Loop Analysis

Introduction

Most iterative algorithms are built from loops. To get their time complexity, we need a reliable way to count how many times the loop body runs and how much work each run does. A single loop that runs n times with O(1) work per iteration gives O(n). Two nested loops, each over n, give O(n²). This section teaches you to analyze single loops, nested loops, and common variations (loop bounds that change, multiple sequential loops, loop with a shrinking range) so you can derive Big-O from code quickly and correctly.

Why This Topic Matters

Loop structure is the main source of complexity in non-recursive code. If you can analyze loops, you can analyze most iterative algorithms—searching, sorting, matrix traversal, and many DP and greedy solutions. Interview problems often have obvious loop structure; being able to say "two nested loops over n → O(n²)" or "outer n, inner halves each time → O(n log n)" shows you understand complexity at a glance.

Mental Model

Total time = (number of iterations) × (work per iteration), where "work per iteration" is itself in Big-O. So we (1) figure out how many times the loop runs as a function of n, (2) figure out the cost per iteration (constant, or another loop, etc.), (3) multiply (or sum if different iterations do different work), then (4) simplify to Big-O by keeping the dominant term.

Single Loop Analysis

Fixed number of iterations

If the loop runs a constant number of times (e.g., 10, 100) independent of input size n, total cost is O(1). Example: a loop that runs exactly 5 times doing O(1) work each time → 5 × O(1) = O(1).

Loop runs n times

Most common: for i in range(n): or for i in range(len(arr)):. If the body does O(1) work per iteration, total is n × O(1) = O(n). If the body does O(log n) work per iteration (e.g., a binary search or a balanced-tree operation inside), total is n × O(log n) = O(n log n).

# O(n): n iterations, O(1) per iteration
for i in range(n):
    do_something_constant()

# Still O(n): body is O(1) in terms of n
for i in range(n):
    x = arr[i] + 1
    result.append(x)

Loop runs a fraction or multiple of n

If the loop runs n/2, 2n, or 5n times, we still get O(n)—constant factors are dropped. So for i in range(0, n, 2): runs n/2 times → O(n).

Loop that halves the range each time

If the loop variable doubles each time (e.g., i = 1; while i < n: i *= 2), the number of iterations is about log₂ n. With O(1) work per iteration → O(log n). If the body does O(n) work (e.g., process all elements), total is O(n log n).

# O(log n): i takes values 1, 2, 4, 8, ... up to n
i = 1
while i < n:
    do_something_constant()
    i *= 2

Nested Loop Analysis

Two loops, both run n times

Outer loop runs n times. For each outer iteration, inner loop runs n times. Inner body is O(1). So total = n × n × O(1) = O(n²). Classic pattern for "check every pair" or full matrix traversal.

# O(n²): n * n iterations, O(1) per inner iteration
for i in range(n):
    for j in range(n):
        process(i, j)

Inner loop depends on outer index

Very common: inner loop runs from 0 to i (or i to n). Example: for i in range(n): for j in range(i): .... Inner iterations: when i=0 → 0, i=1 → 1, …, i=n-1 → n-1. Total inner iterations = 0+1+2+…+(n-1) = n(n-1)/2 = Θ(n²). So still O(n²).

# O(n²): inner runs i times for each i; total 0+1+...+(n-1) = n(n-1)/2
for i in range(n):
    for j in range(i):
        do_something()

Why the sum 0+1+…+(n-1) is Θ(n²)

Sum of first k integers = 1+2+…+k = k(k+1)/2. So 0+1+…+(n-1) = (n-1)n/2 = (n² - n)/2. For large n, the n² term dominates; we drop the linear term and constant factor → Θ(n²). So any time the inner loop runs "up to the outer index" (or "from outer index to n"), expect O(n²).

Three nested loops, each O(n)

n × n × n = O(n³). Same idea: multiply the number of iterations of each loop when they're independent (or sum over outer index if inner bounds depend on it).

Step-by-Step: Deriving Complexity from Loops

1. Identify the loops and their bounds (as functions of n or input size).
2. For each loop, determine how many times it runs (exact or in Big-O). If the bound depends on another loop variable, express it in terms of that variable first.
3. Multiply iteration counts for nested loops when the inner count doesn't depend on the outer index, or sum over the outer index when it does (e.g., inner runs 0 to i → sum i from 0 to n-1).
4. Multiply by work per iteration if it's not O(1).
5. Simplify to a single Big-O (drop constants, keep dominant term).

ASCII Diagram: Nested Loop Iterations

Outer i:    0    1    2    3   ...  n-1
Inner j:   (0)  (0,1) (0,1,2) (0..3) ... (0..n-1)
Count:      0    1      2      3    ...  n-1

Total inner iterations = 0 + 1 + 2 + ... + (n-1) = n(n-1)/2  →  O(n²)

Common Loop Patterns (Quick Reference)

Python Examples with Line-by-Line Complexity

Example 1: Sequential loops

def two_loops(arr):
    total = 0
    for x in arr:           # n iterations, O(1) each → O(n)
        total += x
    for x in arr:           # n iterations, O(1) each → O(n)
        total += x * 2
    return total            # O(n) + O(n) = O(n)

Two independent loops, each O(n). We add their costs: O(n) + O(n) = O(n). So overall O(n).

Example 2: Nested loops (full pair check)

def count_pairs(arr):
    count = 0
    for i in range(len(arr)):      # n times
        for j in range(len(arr)):  # n times each → n * n
            if i != j and arr[i] + arr[j] == 0:
                count += 1
    return count                   # O(n²)

Outer n, inner n, body O(1). Total O(n²).

Example 3: Inner loop from 0 to i

def prefix_sums(arr):
    n = len(arr)
    result = [0] * n
    for i in range(n):              # i = 0, 1, ..., n-1
        for j in range(i + 1):      # j runs 1, 2, ..., n times
            result[i] += arr[j]     # inner: 1+2+...+n = n(n+1)/2
    return result                   # O(n²)

Inner iterations: 1 + 2 + … + n = n(n+1)/2 → O(n²).

Edge Cases

• Loop with early break/return: Complexity is the worst case over all inputs—assume the loop runs as many times as the bound allows unless you're analyzing best/average separately.
• Loop bound is min(n, k) or similar: If k is a constant, effective iterations ≤ k → O(1). If k is a parameter, express complexity in both n and k (e.g., O(n × k)).
• Inner loop that doesn't start at 0: Same idea—count how many times it runs for each outer value and sum (e.g., j from i to n-1 gives (n-i) iterations for outer i; sum = n(n-1)/2 → O(n²)).

Summary

• Single loop: (iterations) × (work per iteration). n iterations with O(1) body → O(n); log n iterations → O(log n).
• Nested loops: Multiply when bounds are independent (n × n → O(n²)). When inner bound depends on outer index, sum over outer index (e.g., 0+1+…+(n-1) = Θ(n²)).
• Sequential loops: Add their costs: O(n) + O(n) = O(n).
• Derive by: count iterations per loop, multiply or sum as appropriate, multiply by work per iteration, then simplify to Big-O.

3.5 Recursion Tree Method

Introduction

Recursive algorithms are described by recurrence relations: equations that express T(n) in terms of T on smaller inputs (e.g., T(n) = 2T(n/2) + n). To find their time complexity, we need to solve the recurrence. The recursion tree method does this by drawing a tree: each node represents the cost at one level of recursion, and we sum the costs level by level (or across leaves) to get the total. It gives intuition, works for many recurrences that don't fit the Master Theorem, and is a standard tool in interviews and coursework.

Why This Topic Matters

Merge sort, quicksort, binary search, and many divide-and-conquer algorithms have runtimes given by recurrences like T(n) = 2T(n/2) + Θ(n). The recursion tree method lets you see why the total is O(n log n): you draw the tree, sum the work at each level, and observe that there are log n levels with O(n) work per level. When the Master Theorem doesn't apply (e.g., T(n) = T(n/3) + T(2n/3) + n), the tree method still works. It also builds the intuition you need for the Master Theorem and for substitution proofs.

What Is a Recurrence?

A recurrence is an equation that defines T(n) in terms of T on smaller arguments. Example: T(n) = 2T(n/2) + n says: "to solve a problem of size n, we do n work and then solve two subproblems of size n/2." We need a base case (e.g., T(1) = 1 or T(1) = O(1)) to stop the recursion. The goal is to find a closed form or Big-O for T(n).

Recursion Tree: The Idea

Imagine the recurrence as a tree:

• Root = one call of size n; we charge the "extra" work at this level (the + n part) to the root.
• Children = the recursive calls. So T(n) = 2T(n/2) + n gives two children, each of size n/2; we write the cost "n" at the root.
• Each child again has its own cost (n/2 at that level) and its children (size n/4), and so on until we hit the base case (size 1).
• Total cost = sum of the costs written at every node. We can sum by level: level 0 has 1 node with cost n; level 1 has 2 nodes with cost n/2 each → total n; level 2 has 4 nodes with cost n/4 each → total n; … So each level sums to n, and there are log₂ n levels → total Θ(n log n).

Step-by-Step: Recursion Tree Method

1. Write the recurrence in the form T(n) = (sum of T(subproblems)) + (work at this level). Put the "work at this level" as the node cost.
2. Draw the tree (at least conceptually): root cost, then children with their costs, and so on. Identify the pattern: how many children per node? What size? What cost per node at level i?
3. Find the number of levels until base case. For T(n/2), size halves each level → about log₂ n levels. For T(n/3), about log₃ n levels.
4. Sum the cost per level, then sum over levels. Or sum over all leaves (if each leaf has the same cost and you know the count) and add the internal cost—whichever is easier.
5. Simplify to Big-O. Often the sum is a geometric series or "constant per level × number of levels."

Worked Example: T(n) = 2T(n/2) + n

This is the merge sort recurrence. Assume T(1) = Θ(1).

Level 0:        [n]                    cost = n
               /   \
Level 1:    [n/2]   [n/2]              cost = n/2 + n/2 = n
            /  \     /  \
Level 2:  [n/4][n/4][n/4][n/4]        cost = 4·(n/4) = n
            ...
Level k:  2^k nodes, each cost n/2^k   cost = 2^k · (n/2^k) = n

Every level has total cost n. Number of levels: size goes n → n/2 → … → 1, so k levels when n/2^k = 1 → k = log₂ n. So total = n × log₂ n = Θ(n log n).

Worked Example: T(n) = 2T(n/2) + Θ(1)

Same structure, but only constant work at each node. Level 0: 1; level 1: 2; level 2: 4; … level k: 2^k nodes, each Θ(1). Total = 2^0 + 2^1 + … + 2^(log n) = 2^(log n + 1) − 1 = Θ(n). So T(n) = Θ(n)—the tree is a full binary tree with Θ(n) leaves, and constant work per node.

When the Per-Level Cost Changes

If the work at level i is not constant (e.g., it decreases with depth), write the cost at each level and sum. Example: T(n) = 2T(n/2) + n². Root cost n²; level 1: 2 × (n/2)² = n²/2; level 2: 4 × (n/4)² = n²/4; … level k: 2^k × (n/2^k)² = n²/2^k. Total = n²(1 + 1/2 + 1/4 + …) ≤ 2n² → O(n²). Here the root dominates; the series is geometric and sums to a constant.

ASCII: Summing by Levels vs by Leaves

Sum by levels (T(n)=2T(n/2)+n):
  L0: n
  L1: n
  L2: n
  ...
  L_log n: n
  Total: n * (log n + 1) = Θ(n log n)

Sum by leaves (for T(n)=2T(n/2)+Θ(1)):
  Number of leaves = 2^(log n) = n
  Work per leaf = Θ(1)
  Total = Θ(n)
  (Internal work is also Θ(n), same order.)

Common Recurrence Patterns (Recursion Tree View)

Edge Cases and Tips

• Uneven splits: T(n) = T(n/3) + T(2n/3) + n. The tree is not full; the "depth" is determined by the branch that shrinks slowest (2n/3). After k steps, size is at most (2/3)^k n; so depth ≈ log_{3/2} n. Per-level cost is still O(n). Total O(n log n).
• More than two subproblems: T(n) = 3T(n/2) + n. Each node has 3 children of size n/2. Level k has 3^k nodes, cost (n/2^k) each → total n×(3/2)^k. Sum over k = 0 to log₂ n: geometric with ratio 3/2 → dominated by last term, O(n^(log₂ 3)).
• Base case: Use T(1) = c or T(0) = c. For asymptotic result, the exact base constant doesn't change Big-O.

Summary

• Recurrence = equation like T(n) = 2T(n/2) + n; recursion tree = picture of cost at each level of recursion.
• Method: Draw tree (root = current work, children = subcalls); find number of levels and cost per level; sum over levels (or use geometric series); simplify to Big-O.
• Classic: T(n) = 2T(n/2) + n → log n levels, n work per level → Θ(n log n).
• Use the tree when the Master Theorem doesn't apply or when you want a clear visual derivation.

3.6 Master Theorem

Introduction

The Master Theorem (or Master Method) gives a cookbook solution for recurrences of the form T(n) = aT(n/b) + f(n): we split the problem into a subproblems of size n/b, and do f(n) extra work. By comparing f(n) with the quantity n^{log_b a} (the "critical exponent"), the theorem tells you whether the total time is dominated by the base-case work, balanced, or dominated by the top-level work—and gives the answer in one step. It's the fastest way to solve many divide-and-conquer recurrences without drawing the recursion tree.

Why This Topic Matters

Merge sort, binary search, and many recursive algorithms fit T(n) = aT(n/b) + f(n). The Master Theorem lets you state their complexity in seconds: "a=2, b=2, f(n)=n → n^(log_b a)=n, so Case 2 → Θ(n log n)." In interviews, knowing the three cases and when they apply is enough to justify divide-and-conquer runtimes. When the recurrence doesn't fit (e.g., a or b not constant, or f(n) has a different form), you fall back to the recursion tree or substitution method.

Standard Form and the Critical Exponent

Recurrence in standard form:

T(n) = aT(n/b) + f(n)

• a ≥ 1: number of subproblems per step.
• b > 1: factor by which the problem size shrinks (so each subproblem has size n/b).
• f(n): cost of dividing and combining (the "non-recursive" work). We assume f(n) is asymptotically positive.

The critical exponent is log_b a. So the "size" of the recursive work (ignoring f) is like n^{log_b a}: that's the number of leaves in the recursion tree (a^(log_b n) = n^(log_b a)). The theorem compares f(n) with n^{log_b a} to decide which dominates.

The Three Cases

Case 1: f(n) is polynomially smaller than n^{log_b a}

If f(n) = O(n^{log_b a − ε}) for some constant ε > 0, then the recursive part dominates. Result: T(n) = Θ(n^{log_b a}).

Intuition: the work at the root (and each level) is tiny compared to the growth of the leaf count; total is dominated by the leaves.

Case 2: f(n) is the same order as n^{log_b a}

If f(n) = Θ(n^{log_b a} log^k n) for some k ≥ 0 (usually k = 0 or 1), then work is balanced across levels. Result: T(n) = Θ(n^{log_b a} log^k+1 n). For k = 0: T(n) = Θ(n^{log_b a} log n).

Intuition: every level does about the same total work; there are Θ(log n) levels, so we get an extra log factor.

Case 3: f(n) is polynomially larger than n^{log_b a}

If f(n) = Ω(n^{log_b a + ε}) for some ε > 0, and if the regularity condition holds (a·f(n/b) ≤ c·f(n) for some c < 1 and large n), then the top-level work dominates. Result: T(n) = Θ(f(n)).

Intuition: the root (and upper levels) do so much work that the sum is dominated by f(n). The regularity condition ensures that work decreases as we go down the tree.

Quick Reference Table

Step-by-Step: How to Apply the Master Theorem

1. Identify a, b, and f(n) from T(n) = aT(n/b) + f(n).
2. Compute log_b a. So n^{log_b a} is your comparison benchmark. (Tip: log_b a = (ln a)/(ln b).)
3. Compare f(n) with n^{log_b a}:
   • If f(n) is O(n^{log_b a − ε}) for some ε > 0 → Case 1 → T(n) = Θ(n^{log_b a}).
   • If f(n) is Θ(n^{log_b a} log^k n) → Case 2 → T(n) = Θ(n^{log_b a} log^k+1 n).
   • If f(n) is Ω(n^{log_b a + ε}) and a·f(n/b) ≤ c·f(n) for some c < 1 → Case 3 → T(n) = Θ(f(n)).
4. If none of the cases clearly applies (e.g., f(n) is not polynomial in n, or recurrence has a different form), use recursion tree or substitution.

Worked Examples

Example 1: T(n) = 2T(n/2) + n (merge sort)

a = 2, b = 2, f(n) = n. So n^{log_b a} = n^{log₂ 2} = n¹ = n. We have f(n) = n = Θ(n) = Θ(n^{log_b a} log⁰ n). That's Case 2 with k = 0. So T(n) = Θ(n^{log_b a} log¹ n) = Θ(n log n).

Example 2: T(n) = T(n/2) + Θ(1) (binary search)

a = 1, b = 2, f(n) = Θ(1). So n^{log_b a} = n^{log₂ 1} = n⁰ = 1. We have f(n) = Θ(1) = Θ(n⁰). So f(n) = Θ(n^{log_b a} log⁰ n) → Case 2, k = 0. T(n) = Θ(n⁰ log n) = Θ(log n).

Example 3: T(n) = 2T(n/2) + Θ(1)

a = 2, b = 2, f(n) = Θ(1). n^{log_b a} = n. So f(n) = O(n^1−ε) for any ε in (0,1] (e.g. f(n) = O(n^0.5)). Case 1. T(n) = Θ(n^{log_b a}) = Θ(n).

Example 4: T(n) = 2T(n/2) + n²

a = 2, b = 2, f(n) = n². n^{log_b a} = n. So f(n) = n² = Ω(n^1+ε) for ε = 1. Regularity: 2·(n/2)² = n²/2 ≤ c·n² for c = 1/2 < 1. Case 3. T(n) = Θ(n²).

Example 5: T(n) = 3T(n/2) + n (e.g. Strassen-like split)

a = 3, b = 2, f(n) = n. n^{log_b a} = n^{log₂ 3} ≈ n^1.58. So f(n) = n = O(n^{1.58 − ε}) for small ε. Case 1. T(n) = Θ(n^{log₂ 3}) ≈ Θ(n^1.58).

When the Master Theorem Does Not Apply

• T(n) = 2T(n/2) + n log n: f(n) = n log n is larger than n but not polynomially larger than n (no n^1+ε). Case 2 with k=1 gives T(n) = Θ(n log² n)—check the exact statement for log^k n in f(n).
• T(n) = T(n/2) + T(n/2) + n is really 2T(n/2)+n, so it applies. But T(n) = T(n/3) + T(2n/3) + n has two different fractions; standard form assumes one n/b. Use recursion tree.
• a or b not constant: e.g. T(n) = n·T(√n) + n. Form is different; use substitution or tree.

Summary

• Standard form: T(n) = aT(n/b) + f(n). Compute n^{log_b a} and compare with f(n).
• Case 1: f(n) polynomially smaller → T(n) = Θ(n^{log_b a}).
• Case 2: f(n) = Θ(n^{log_b a} log^k n) → T(n) = Θ(n^{log_b a} log^k+1 n).
• Case 3: f(n) polynomially larger and regularity → T(n) = Θ(f(n)).
• When the recurrence doesn't fit (e.g. different split ratios), use the recursion tree or substitution method.

3.7 Space Complexity

Introduction

Space complexity is how much extra memory an algorithm uses, as a function of input size n (and sometimes other parameters). We express it in Big-O, just like time: O(1), O(n), O(n²), etc. It tells you whether your solution will run in limited memory (e.g. on embedded systems or with huge inputs) and whether you can improve by using less auxiliary space—e.g. in-place algorithms use O(1) extra space. This section covers what to count, how to analyze iterative and recursive code, and how to state space complexity clearly in interviews.

Why This Topic Matters

Interviews often ask "what's the space complexity?" in addition to time. Recursive solutions can be O(n) space just from the call stack; building a copy of the input adds O(n). In-place algorithms are valued when you must not use extra linear space. In practice, running out of memory can be as bad as running too long—so understanding space helps you choose and justify algorithms (e.g. iterative vs recursive DFS, merge sort vs in-place quicksort in terms of space).

Auxiliary Space vs Total Space

• Total space = space for input + output + any extra data structures and call stack. Sometimes we say "the algorithm uses O(n) space" meaning total.
• Auxiliary space = extra space used beyond the input (and sometimes beyond the output, depending on convention). So "O(1) auxiliary space" means we only use a constant number of extra variables; the input itself is not counted.

In DSA and interviews, "space complexity" usually means auxiliary space—we don't count the input (or the output if it's required, e.g. returning a new array). So "merge sort is O(n) space" means O(n) extra for the temporary arrays, not counting the input array. Always clarify if the problem says "space" without specifying: most of the time it's auxiliary.

What to Count

• Extra variables and data structures: A list of size n you build → O(n). A hash map with n keys → O(n). A few integers → O(1).
• Call stack (recursion): Each recursive call uses space for parameters, return address, and local variables. Depth of recursion × space per frame. Example: recursive binary search has depth Θ(log n), so O(log n) space if each frame is O(1).
• Input and output: Usually not counted in auxiliary space. If you must count total space, input is often Θ(n) and output can be too.

Iterative Code: No Recursion

Space is just the extra data structures and variables. One loop with a fixed number of variables → O(1). Building a list of size n → O(n). Two arrays of size n → O(n) (we don't double-count constants in Big-O, but "two arrays" is still O(n)). Nested loops don't add space unless you allocate per iteration (e.g. a new list each time would be dangerous and likely O(n²) or worse).

# O(1) auxiliary: only a few variables
def max_element(arr):
    m = arr[0]
    for x in arr[1:]:
        if x > m:
            m = x
    return m

# O(n) auxiliary: copy of input
def sorted_copy(arr):
    return sorted(arr)   # returns new list of size n

Recursive Code: Call Stack

Space = (maximum depth of recursion) × (space per frame). Frame space is usually O(1) per call (parameters and locals). So if depth is Θ(n), space is O(n); if depth is Θ(log n), space is O(log n).

# O(n) space: depth n, O(1) per frame
def fact(n):
    if n <= 1:
        return 1
    return n * fact(n - 1)

# O(log n) space: depth log n, O(1) per frame
def binary_search_rec(arr, t, lo, hi):
    if lo > hi:
        return -1
    mid = (lo + hi) // 2
    if arr[mid] == t:
        return mid
    if arr[mid] > t:
        return binary_search_rec(arr, t, lo, mid - 1)
    return binary_search_rec(arr, t, mid + 1, hi)

Common Space Complexities (Quick Reference)

In-Place and O(1) Space

An in-place algorithm uses O(1) extra space (or at most O(log n) for recursion). It may overwrite the input. Examples: swapping two elements, reversing an array with two pointers, some sorting algorithms (e.g. quicksort with tail recursion or iterative implementation can be O(log n) stack; "in-place" often means no extra array). Saying "we can do this in O(1) space" is a strong claim—it usually means no extra arrays or maps that grow with n.

Time vs Space Trade-off

Often you can use more space to get faster time: e.g. hash map for O(1) lookup instead of O(n) scan—O(n) space for O(n) or O(1) time. Or memoization (DP): store subproblem results to avoid recomputation—extra space for less time. The reverse: in-place algorithms save space but may be trickier or have the same time. In interviews, stating the trade-off ("we can do O(n) time and O(n) space with a set, or O(n²) time and O(1) space with two loops") shows good understanding.

Summary

• Space complexity = extra memory as a function of n, usually meaning auxiliary space (excluding input, and often output).
• Count: extra data structures (arrays, maps, etc.) and call stack depth for recursion.
• Iterative: space = size of extra variables and structures. Recursive: space = depth × space per frame.
• O(1) = in-place style; O(n) = one extra linear structure or linear recursion depth; O(log n) = logarithmic depth (e.g. binary search recursion).
• Time–space trade-offs: more space can mean faster time (e.g. hash map); less space may mean more time or a more complex algorithm.

3.8 Amortized Analysis

Introduction

Amortized analysis gives a bound on the average cost per operation over a sequence of operations, rather than the cost of a single operation in the worst case. Some operations in the sequence may be expensive (e.g. resizing an array), but if they happen rarely, the average cost per operation can still be low. We say "amortized O(1) per append" meaning: over n appends, total cost is O(n), so each append "costs" O(1) on average. This is how we justify that list.append in Python (and dynamic arrays in general) is O(1) amortized, even though a single append can trigger an O(n) resize.

Why This Topic Matters

Dynamic arrays (Python list, C++ vector, Java ArrayList), hash tables, and many data structures have operations that are cheap most of the time and occasionally expensive. Worst-case per-operation analysis would say "append can be O(n)," which is misleading—we don't pay O(n) every time. Amortized analysis tells the right story: append is O(1) amortized, so building a list of n elements is O(n) total. In interviews, saying "append is O(1) amortized" shows you understand the difference between worst-case per operation and amortized cost.

Amortized vs Worst-Case vs Average Case

• Worst-case (per operation): The cost of a single operation in the worst possible scenario. One append might be O(n) when it triggers a resize.
• Average case (over inputs): Expected cost of one operation when the input is random. That's a different notion—we're not averaging over a sequence of operations.
• Amortized: We fix a sequence of operations (e.g. n appends). Total cost of the sequence is T. Amortized cost per operation = T / n. So we're spreading the cost of expensive operations over the whole sequence.

Classic Example: Dynamic Array (List) Append

Start with an array of capacity 1 (or some constant). When we append and the array is full, we allocate a new array of double the size, copy all elements, then append. So most appends are O(1); every time we double, we do O(current size) work. Question: over n appends, what is the total cost?

We do at most O(1) work for each of the n appends, plus the cost of copies during resizes. Copy sizes: 1, 2, 4, 8, … up to at most n (roughly). So total copy cost ≤ 1 + 2 + 4 + … + n ≤ 2n (geometric series). So total operations = n (appends) + O(n) (copies) = O(n). Hence amortized cost per append = O(n)/n = O(1).

Append #:  1  2  3  4  5  6  7  8  9  ...
Cost:      1  1  2  1  1  1  1  4  1  ...
           (copy 1)  (copy 1,2)       (copy 1..4)

Total after n appends: n + (1+2+4+...+ ≤n) = n + O(n) = O(n)
Amortized per append: O(1)

Three Methods for Amortized Analysis

Aggregate method

Sum the total cost of n operations; show it's T(n); then amortized cost = T(n)/n. We used this for dynamic array: total O(n), so O(1) amortized per append.

Accounting (banker's) method

Assign a "charge" (amortized cost) to each operation. Some of that charge pays for the operation itself; the rest is "credit" stored for later. We require that credit never goes negative. If we charge 2 units per append (so amortized O(1)), then a cheap append uses 1 and saves 1; when we resize we use the saved credit to pay for the copy. So total charge n×2 = O(n) covers all real cost.

Potential method

Define a "potential" function Φ on the data structure state. Let c_i be the real cost of the i-th operation and Φ_i the potential after it. We show that amortized cost â_i = c_i + Φ_i − Φ_i−1 is small (e.g. O(1)). Then sum of â_i = sum of c_i + Φ_n − Φ₀; if Φ is always non-negative and bounded, total real cost is bounded by sum of â_i. For dynamic array, Φ can be "2 × (number of elements) − capacity"; when we double, the drop in potential pays for the copy.

For interviews, the aggregate method is usually enough: "Total cost of n appends is O(n), so O(1) amortized."

Summary of Dynamic Array Resize

• Doubling when full: copy sizes 1, 2, 4, … up to ≈ n. Sum = O(n). So n appends cost O(n) total → O(1) amortized per append.
• If we grew by a constant (e.g. +10 each time): after n appends we'd copy 10 + 20 + 30 + … ≈ O(n²) total → amortized O(n) per append. So doubling (or any constant factor growth) is essential for O(1) amortized.

When Amortized Analysis Applies

Use it when you have a sequence of operations and some are occasionally expensive. Examples: dynamic array append/push, hash table insert (with rehashing), splay tree operations, incrementing a binary counter (flipping bits). We don't use "amortized" for a single standalone operation—we use it for the cost per operation in a long run.

Summary

• Amortized cost = (total cost of a sequence of operations) / (number of operations). We bound the total, then divide.
• Used when some operations are occasionally expensive (e.g. resize); over the sequence, the average cost per operation is small.
• Dynamic array append with doubling: total O(n) for n appends → O(1) amortized per append. Constant-size growth would give amortized O(n).
• Methods: aggregate (sum total, divide), accounting (charge and credit), potential (potential function). For interviews, aggregate is usually enough.
• Amortized is not average case: it's deterministic worst-case over the sequence.

3.9 Recurrence Relations

Introduction

A recurrence relation is an equation that defines a function T(n) in terms of its values on smaller inputs—e.g. T(n) = 2T(n/2) + n. Recurrences appear whenever we analyze recursive algorithms: the cost of solving a problem of size n is the cost of the "current step" plus the cost of solving smaller subproblems. To get Big-O for the algorithm, we must solve the recurrence—find a closed form or an asymptotic bound. This section ties together recurrences as a concept, when they arise, and how to solve them using the tools from earlier topics (recursion tree, Master Theorem, and substitution).

Why This Topic Matters

Merge sort, quicksort, binary search, divide-and-conquer, and many recursive DP or backtracking algorithms have runtimes that naturally express as recurrences. Writing the recurrence is step one; solving it gives you the complexity. Recurrence relations are the bridge between "what the code does" and "what is T(n) in Big-O." In interviews, you might be asked to write a recurrence for your recursive solution and then solve it (or say "it fits the Master Theorem, so Θ(n log n)").

What Is a Recurrence Relation?

Formally, a recurrence for T(n) has the form:

T(n) = (expression involving T(n₁), T(n₂), …) + (non-recursive work)

where n₁, n₂, … are smaller than n (e.g. n−1, n/2, n/3). We also need a base case: T(1) = c or T(0) = c so the recursion stops. The "non-recursive work" is the cost of dividing the problem and combining results (e.g. merging two sorted halves). We want to find a closed form or asymptotic bound for T(n).

Where Recurrences Come From

• Divide-and-conquer: Split into a subproblems of size n/b, do f(n) work. T(n) = aT(n/b) + f(n). Examples: merge sort 2T(n/2)+n, binary search T(n/2)+1.
• Linear reduction: One subproblem of size n−1 plus linear work. T(n) = T(n−1) + n or T(n) = T(n−1) + Θ(1). Examples: factorial, simple recursive scan.
• Multiple subproblems with different sizes: T(n) = T(n/3) + T(2n/3) + n. Doesn't fit the standard Master Theorem form; use recursion tree.

Common Recurrence Types and Their Solutions

How to Solve Recurrences: Method Overview

1. Recursion tree (3.5): Draw the tree, sum cost per level (or over leaves). Works for any recurrence; especially useful when Master Theorem doesn't apply (e.g. T(n/3)+T(2n/3)+n).
2. Master Theorem (3.6): For T(n) = aT(n/b) + f(n). Compare f(n) with n^(log_b a); apply Case 1, 2, or 3. Fast when it fits.
3. Substitution: Guess the form (e.g. T(n) = O(n log n)), then prove by induction. Useful when you have a candidate bound and need to verify, or when the recurrence is non-standard.
4. Expand and sum: For simple recurrences like T(n) = T(n−1) + n, unroll: T(n) = n + (n−1) + … + T(1) = n(n+1)/2 + c = Θ(n²).

Quick Derivation: T(n) = T(n−1) + n

T(n) = n + T(n−1) = n + (n−1) + T(n−2) = … = n + (n−1) + … + 1 + T(0) = n(n+1)/2 + Θ(1) = Θ(n²).

Quick Derivation: T(n) = 2T(n/2) + n

Master Theorem: a=2, b=2, f(n)=n, n^(log_b a)=n. So f(n)=Θ(n)=Θ(n^(log_b a)). Case 2 → T(n)=Θ(n log n). Or recursion tree: each level costs n, log n levels → n log n.

When the Recurrence Doesn't Fit Standard Form

• Uneven splits: T(n) = T(n/3) + T(2n/3) + n. Use recursion tree; depth from slowest branch (2n/3); per-level cost O(n) → O(n log n).
• More than one recursive term with different arguments: Still draw the tree and sum, or try substitution with a guessed bound.
• f(n) not polynomial: e.g. T(n) = 2T(n/2) + n log n. Master Theorem (extended) or recursion tree; result is often Θ(n log² n) or similar.

Summary

• Recurrence relation = equation defining T(n) in terms of T on smaller inputs plus non-recursive work; need a base case.
• Common forms: T(n)=aT(n/b)+f(n) (divide-and-conquer), T(n)=T(n−1)+g(n) (linear reduction).
• Solve with: recursion tree, Master Theorem (when form fits), substitution, or expand-and-sum for simple cases.
• Know the classic solutions: T(n−1)+n → Θ(n²); 2T(n/2)+n → Θ(n log n); T(n/2)+1 → Θ(log n).
• When the recurrence is non-standard (uneven splits, non-polynomial f), use the recursion tree or substitution.

3.10 Proof of Time Complexity (Basic Induction)

Introduction

So far we've derived time complexity using recursion trees and the Master Theorem. To be rigorous, we can prove that our solution is correct—e.g. that T(n) = O(n log n) for the recurrence T(n) = 2T(n/2) + n. The standard way to do that is the substitution method: guess a bound (e.g. T(n) ≤ c·n log n), then use induction to show that the recurrence implies the bound for a suitable constant c and large n. This section introduces basic induction and the substitution method so you can prove (or verify) complexity bounds when needed—and understand why the Master Theorem and recursion tree conclusions are valid.

Why This Topic Matters

In coursework and sometimes in interviews, you may need to justify that your asymptotic bound is correct—not just state it. The substitution method is the standard proof technique for recurrence solutions. It also helps when the recurrence doesn't fit the Master Theorem: you guess the answer from the recursion tree, then prove it by substitution. Understanding induction makes you confident that "T(n) = 2T(n/2) + n ⇒ T(n) = Θ(n log n)" is not just a formula but a provable fact.

Induction in One Paragraph

Induction proves a statement P(n) for all natural numbers n (or all n ≥ n₀). You show: (1) Base case: P(n₀) is true. (2) Inductive step: For every n > n₀, if P(k) is true for all k < n (or for k = n−1, in simple induction), then P(n) is true. Then by induction, P(n) holds for all n ≥ n₀. For recurrences we often use strong induction: assume the bound holds for all smaller values (T(1), T(2), …, T(n−1)), then prove it for T(n) using the recurrence.

The Substitution Method (Steps)

1. Guess the form of the solution: e.g. T(n) ≤ c·n log n for some constant c > 0.
2. State the inductive hypothesis: Assume T(k) ≤ c·k log k for all k < n (and for k in the range we care about, e.g. k ≥ 2).
3. Plug into the recurrence: T(n) = 2T(n/2) + n ≤ 2(c·(n/2) log(n/2)) + n = c·n log(n/2) + n = c·n log n − c·n + n.
4. Show the right-hand side is ≤ your guess: We want c·n log n − c·n + n ≤ c·n log n. That holds if −c·n + n ≤ 0, i.e. c ≥ 1. So choose c = 1 (or larger).
5. Base case: For n = 1 (or small n), T(1) = O(1). Choose c large enough so that c·1·log 1 = 0 doesn't matter; we may need to set n₀ ≥ 2 and check T(2) by hand, then the inductive step applies for n ≥ 2.

Worked Example: T(n) = 2T(n/2) + n ⇒ T(n) = O(n log n)

Guess: T(n) ≤ c·n log n for n ≥ 2, for some c ≥ 1.

Inductive hypothesis: Assume T(k) ≤ c·k log k for all 2 ≤ k < n.

Recurrence: T(n) = 2T(n/2) + n. By the hypothesis (with k = n/2, and we assume n/2 ≥ 2 so n ≥ 4, or we handle n = 2, 3 separately), T(n/2) ≤ c·(n/2)·log(n/2). So:

T(n) ≤ 2·c·(n/2)·log(n/2) + n = c·n·log(n/2) + n = c·n·(log n − 1) + n = c·n log n − c·n + n.

We need T(n) ≤ c·n log n. So we need −c·n + n ≤ 0, i.e. c ≥ 1. Choose c = 1. Then T(n) ≤ n log n for n ≥ 2 (with base case checked). Hence T(n) = O(n log n).

When the Guess Needs a Lower-Order Term

Sometimes a "plain" guess like T(n) ≤ c·n log n doesn't work because the recurrence gives an extra positive term that won't disappear. Then we guess with a subtraction: T(n) ≤ c·n log n − dn. Substituting, we get something like c·n log n − dn − (d − 1)n; we choose d so that (d − 1)n ≥ 0 (e.g. d ≥ 1), and then the right-hand side is ≤ c·n log n − dn. So the inductive step goes through. The Master Theorem and recursion tree already tell us the correct form; the subtraction trick is just to make the algebra work in the proof.

Proving Θ (Upper and Lower Bounds)

To prove T(n) = Θ(n log n), we prove both T(n) = O(n log n) and T(n) = Ω(n log n). For the upper bound we use substitution as above. For the lower bound, we guess T(n) ≥ c·n log n and show that the recurrence implies it (with a suitable c > 0 and possibly a lower-order term added in the guess). The idea is the same; the inequalities are reversed.

Summary of the Substitution Method

• Guess the asymptotic form (from recursion tree or Master Theorem).
• Assume by (strong) induction that the bound holds for all smaller inputs.
• Substitute into the recurrence and show that the bound holds for n.
• Choose constants (c, n₀, and sometimes a subtracted term) so that the base case and inductive step both work.

Summary

• Induction proves a statement for all n: base case + inductive step (assume for smaller k, prove for n).
• Substitution method for recurrences: guess a bound (e.g. T(n) ≤ c·n log n), assume it holds for all smaller inputs, substitute into the recurrence, and show the bound holds for n; choose c and base case appropriately.
• Sometimes the guess needs a lower-order term (e.g. c·n log n − dn) so the inequality works out.
• To prove Θ, prove both O and Ω.
• In practice, recursion tree and Master Theorem are enough for stating complexity; substitution is for rigor or when asked to prove.

4.1 Number Systems

Introduction

When you see the number 42, you instantly read it as “forty-two.” When a computer stores that same value, it uses a different representation: 101010 in binary. The value is the same; the way we write it depends on the number system (base) we choose. In algorithms and programming, you will constantly meet binary (bits, masks, XOR), hexadecimal (memory addresses, colors, hashes), and sometimes octal. Understanding how number systems work—and how to convert between them—is essential for bit manipulation, low-level reasoning, and many interview problems.

Real-World Analogy

Think of number systems like different languages for writing the same quantities.

• Decimal (base 10): What we use every day. Ten symbols: 0–9. “42” means 4×10 + 2×1.
• Time: Clocks use base 60 for seconds and minutes (60 seconds in a minute), and base 12 for hours (12 on the clock face). So “1:30” is one way of writing a duration—same idea as a different base.
• Binary (base 2): Only two symbols: 0 and 1. Computers use it because hardware is built from switches that are either on or off. “101010” is the same quantity as decimal 42, just written in base 2.
• Hexadecimal (base 16): Sixteen symbols: 0–9 and A–F. Shorthand for binary (one hex digit = four bits). Used in memory addresses, color codes (#FF5733), and when debugging.

The value (how many) doesn’t change; only the notation (how we write it) changes with the base.

Formal Definition

A number system (or numeral system) is a way of representing numbers using a fixed set of symbols and a base (radix). In a positional system, the value of a digit depends on its position in the number.

The radix is the number of distinct digits. Base 2 → digits 0,1. Base 10 → 0–9. Base 16 → 0–9, A(10), B(11), C(12), D(13), E(14), F(15).

Why This Topic Matters

In DSA and interviews, number systems show up everywhere:

• Bit manipulation: AND, OR, XOR, shifts—all operate on the binary representation. You need to think in base 2 to set/clear/test bits.
• Fast exponentiation / modular arithmetic: The binary expansion of the exponent drives “square and multiply” algorithms (covered later).
• Encoding and hashing: Hex strings represent raw bytes (e.g., SHA hashes). Parsing and generating such strings requires comfort with base 16.
• Problem constraints: Some problems ask for “the number of 1s in the binary representation” or “convert to base k”—direct number-system questions.

Building a clear mental model of bases and conversion will make later topics (Bit Manipulation, Fast Exponentiation, Modular Arithmetic) much easier.

Mental Model

In any base b, each position is a power of b. The rightmost digit is the “ones” place (b⁰), the next is “bs” (b¹), then “b²s,” and so on. So:

  Base 10:  ... thousands  hundreds  tens   ones
            ... 10³        10²       10¹    10⁰
  Example:  4    2     →   4×10 + 2×1 = 42

  Base 2:   ... 32  16  8  4  2  1
            ... 2⁵  2⁴  2³ 2² 2¹ 2⁰
  Example:  1   0   1  0  1  0  →  32+8+2 = 42

  Base 16:  ... 256  16  1
            ... 16²  16¹ 16⁰
  Example:  2    A       →  2×16 + 10×1 = 42
        

Same number, different “columns.” Conversion is just re-expressing the same total in a different column system.

Decimal (Base 10)

We use digits 0–9. Each place is a power of 10. You already do this intuitively: 347 = 3×100 + 4×10 + 7×1. Nothing new here except naming: this is our default radix.

Binary (Base 2)

Only two digits: 0 and 1. Every number is a sum of distinct powers of 2. That’s why binary is natural for computers: each bit is one “switch” (on/off).

• Rightmost bit = 2⁰ = 1 (least significant bit, LSB).
• Next left = 2¹ = 2, then 4, 8, 16, … (most significant bit, MSB, on the left).

Counting in binary: 0, 1, 10, 11, 100, 101, 110, 111, 1000, … (same idea as rolling over digits when you pass 9 in decimal).

Octal (Base 8)

Digits 0–7. Each octal digit corresponds to exactly three binary digits (because 8 = 2³). So conversion between binary and octal is trivial: group bits in threes from the right. Less common today than hex, but you may see file permissions (e.g., chmod 755) expressed in octal.

Example: 42₁₀ = 101010₂. Group as 101 | 010 → 5 and 2 in octal → 52₈. Check: 5×8 + 2 = 42 ✓.

Hexadecimal (Base 16)

Digits 0–9 and A–F (A=10, B=11, C=12, D=13, E=14, F=15). One hex digit = four bits (16 = 2⁴). So binary ↔ hex is a simple grouping by fours. Hex is compact and easy to read; that’s why memory addresses, RGB values (e.g., #FF5733), and many hashes are shown in hex.

Step-by-Step: Converting Between Bases

Two main directions: from decimal to base b, and from base b to decimal.

Decimal → Base b (e.g., Decimal → Binary)

Method: repeated division by b; remainders (read in reverse) give digits from LSB to MSB.

1. Divide the number by b. The remainder is the rightmost digit (LSB).
2. Take the quotient and divide by b again. The new remainder is the next digit to the left.
3. Repeat until the quotient is 0. The sequence of remainders (last to first) is the number in base b.

Example: 42 → binary (base 2). 42÷2=21 rem 0; 21÷2=10 rem 1; 10÷2=5 rem 0; 5÷2=2 rem 1; 2÷2=1 rem 0; 1÷2=0 rem 1. Remainders (bottom to top): 1,0,1,0,1,0 → 101010.

Base b → Decimal

Method: expand by place value. Multiply each digit by its place power and add: d_k·b^k + … + d₀·b⁰. For binary, that’s “add the powers of 2 where the bit is 1.”

Example: 101010₂ = 1×32 + 0×16 + 1×8 + 0×4 + 1×2 + 0×1 = 32+8+2 = 42.

Binary ↔ Hex (Shortcut)

Binary → Hex: group bits in fours from the right; replace each group with the corresponding hex digit (0–9, A–F). Hex → Binary: replace each hex digit with its 4-bit binary form (e.g., A → 1010).

ASCII Diagram: Place Value Across Bases

  Decimal 42 in different bases (same value, different digits):

  Base 10:    4    2     → 4×10¹ + 2×10⁰
  Base  2:  1  0  1  0  1  0  → 1×2⁵ + 1×2³ + 1×2¹
  Base  8:    5    2     → 5×8¹ + 2×8⁰
  Base 16:    2    A     → 2×16¹ + 10×16⁰

  Position:  (MSB) ............ (LSB)
  Value:     b^(n-1) ... b^1  b^0
        

Python Implementation

Python has built-in support for number systems. You can parse strings in a given base and format integers in binary, octal, or hex.

Parsing: String in Base b → Integer

# int(string, base) — base 2 to 36
int("101010", 2)    # 42
int("52", 8)        # 42
int("2A", 16)       # 42
int("2a", 16)       # 42 (hex digits case-insensitive)

# Optional prefix: 0b, 0o, 0x (base inferred)
int("0b101010", 0)  # 42
int("0o52", 0)      # 42
int("0x2A", 0)      # 42

Formatting: Integer → String in Base b

# bin(), oct(), hex() return strings with prefix
bin(42)   # '0b101010'
oct(42)   # '0o52'
hex(42)   # '0x2a'

# Without prefix: use format() or slice
format(42, 'b')   # '101010'
format(42, 'o')   # '52'
format(42, 'x')   # '2a'
format(42, 'X')   # '2A' (uppercase hex)

# Generic base (2–36) — no built-in; implement with repeated division

Custom Base Conversion (Decimal to Base b)

def to_base(n: int, b: int) -> str:
    if n == 0:
        return "0"
    digits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    result = []
    neg = n < 0
    n = abs(n)
    while n:
        result.append(digits[n % b])
        n //= b
    if neg:
        result.append("-")
    return "".join(reversed(result))

# Examples
to_base(42, 2)   # "101010"
to_base(42, 16)  # "2A"
to_base(255, 16) # "FF"

Custom Base Conversion (Base b String to Decimal)

def from_base(s: str, b: int) -> int:
    s = s.strip().upper()
    if not s:
        return 0
    start = 1 if s[0] in "-+" else 0
    sign = -1 if s[0] == "-" else 1
    n = 0
    digits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    for c in s[start:]:
        n = n * b + digits.index(c)
    return sign * n

# Examples
from_base("101010", 2)  # 42
from_base("2A", 16)     # 42

Line-by-Line Explanation: to_base

• if n == 0: return "0" — edge case: zero in any base is "0".
• digits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" — one character per value 0 to 35; supports bases 2–36.
• result = [] — we collect remainder digits; we'll reverse at the end because we get LSB first.
• neg, n = abs(n) — handle negative by converting to positive and tracking sign; then we process positive n.
• while n: — repeat until quotient is 0. n % b is the next digit (LSB first); n //= b is the quotient for the next iteration.
• return "".join(reversed(result)) — remainders were collected LSB-first, so reversing gives MSB-first (normal written form). Prepend "-" if the number was negative.

This is exactly the “repeated division” algorithm we described earlier; the code just automates it.

Edge Cases

• Zero: In any base, 0 is written "0". Your to_base(0, b) should return "0".
• Negative numbers: Standard conversion is for non-negative integers. If you need negative, handle the sign separately (e.g., convert abs(n) and prepend "-").
• Empty string: int("", 2) raises ValueError. In custom from_base, you might return 0 or raise; document the choice.
• Invalid digits: e.g. int("12", 2) — digit 2 is invalid in base 2. Python raises ValueError. In your own parser, validate each character against the base.
• Large numbers: Python integers are arbitrary-precision, so no overflow; int("1"*1000, 2) works.

Common Mistakes

• Reading binary/hex backwards: The rightmost bit is the LSB (2⁰). Don’t treat the leftmost as “first” when computing value—left is MSB, highest power.
• Confusing prefix with value: 0b101010 is a literal in code; the value is 42. When you bin(42) you get the string '0b101010'—the 0b is a prefix, not part of the mathematical value.
• Off-by-one in bit position: “Bit 0” usually means the LSB (2⁰). “Bit i” often means the coefficient of 2ⁱ. Check problem wording (0-indexed vs 1-indexed).
• Using int(string) without base for non-decimal: int("101010") is 101010 in decimal, not 42. For binary you must use int("101010", 2).

Pattern Recognition

Base conversion fits two patterns you’ll reuse:

1. Decimal → base b: Repeated division by b; remainders (reverse order) = digits. Same idea as “extract digits” in decimal (n % 10, n // 10).
2. Base b → decimal: Horner-style expansion: start at 0, then for each digit do value = value * b + digit. One pass left-to-right.

Many “digit” problems (e.g., “sum of digits in base k,” “palindrome in base b”) use these same building blocks.

Practice Problems

• Convert a decimal number to binary and to hex by hand for small values (e.g., 0–255).
• Implement to_base(n, b) and from_base(s, b) for bases 2–36 without using int(s, b) or bin/hex (practice the algorithm).
• Given a positive integer, count the number of 1s in its binary representation (Hamming weight).
• Check if a number’s binary representation is a palindrome (e.g., 9 = 1001 is; 10 = 1010 is not).
• Convert a number from base A to base B (e.g., base 7 → base 5) by going through decimal or by direct conversion if you know the place values.

Summary

• A number system is a way of writing numbers using a base (radix) and a set of digits. In a positional system, value = sum of digit × base^position.
• Decimal (base 10), binary (base 2), octal (base 8), and hexadecimal (base 16) are the most common in programming. Binary is fundamental for bits; hex is a compact shorthand for binary (one hex digit = four bits).
• Decimal → base b: Repeated division by b; remainders (reverse order) give the digits. Base b → decimal: Expand by place value, or use Horner: value = value * b + digit.
• In Python: int(string, base) parses; bin(), oct(), hex() and format(n, 'b'/'o'/'x') format. Use these for correctness and speed; implement by hand when practicing the algorithm.
• Number systems underpin bit manipulation, fast exponentiation, and encoding—master them early for a smooth path through Mathematics for Algorithms and Bit Manipulation.

4.2 Prime Numbers

Introduction

A prime number is a natural number greater than 1 that has exactly two positive divisors: 1 and itself. Primes are the building blocks of integers—every integer greater than 1 can be written uniquely (up to order) as a product of primes. In DSA, primes appear in hashing (hash table sizes), cryptography, factorization problems, and counting (e.g., divisors, coprime pairs). This section covers the definition, why primes matter, how to test if a number is prime, and how to count primes in a range—setting you up for the Sieve of Eratosthenes (next topic) and number-theoretic algorithms.

Real-World Analogy

Think of primes as indivisible building blocks. Just as you can’t split an atom into smaller pieces of the same substance (in classical chemistry), you can’t factor a prime into smaller positive integers other than 1 and itself. Composite numbers are “molecules” made of primes: 12 = 2×2×3, 42 = 2×3×7. Once you know the primes up to a limit, you can build and analyze all integers in that range—same idea as having a periodic table of elements.

Formal Definition

By convention, 1 is neither prime nor composite. The first primes are 2, 3, 5, 7, 11, 13, 17, 19, 23, … Note that 2 is the only even prime; every other even number is divisible by 2.

Why This Topic Matters

• Primality testing: “Is n prime?” appears in problems and interviews. You need a correct, efficient check—often trial division first, then optimizations.
• Counting primes: “How many primes ≤ N?” or “List all primes ≤ N” lead to the Sieve of Eratosthenes (next section)—one of the most common algorithms in number theory.
• Divisors and factorization: GCD, LCM, divisor count, and “sum of divisors” rely on prime factorization. Primes are the first step.
• Hashing and randomization: Prime-sized hash tables and prime moduli reduce collisions. Choosing a “large enough prime” is a standard trick.

Mental Model

For a number n to be composite, it must have a divisor d with 2 ≤ d ≤ √n. Why? If n = a×b with a ≤ b, then a ≤ √n (otherwise a×b > n). So we only need to check divisors up to √n—that’s the core of trial-division primality testing and of “find one factor” for composites.

  n = a × b   with  a ≤ b
  ⇒  a² ≤ a·b = n
  ⇒  a ≤ √n

  So: if no divisor in [2, √n], then n is prime.
        

Primality Testing: Is n Prime?

We want a function is_prime(n) that returns True if n is prime and False otherwise.

Brute Force: Trial Division

Check every integer d from 2 to n−1: if d divides n, then n is composite. If none divide, n is prime. Correct but slow—O(n) divisions for n.

Better: Check Only Up to √n

If n has a divisor d > √n, then n/d is a divisor < √n. So we only need to check d from 2 to ⌊√n⌋. If n % d == 0 for any such d, n is composite; otherwise prime. This reduces the loop to O(√n) iterations.

Optimal for Trial Division: Check 2, Then Odd Numbers

If n is even, we can immediately return n == 2. Then check only odd d: 3, 5, 7, … up to √n. This halves the number of divisions (still O(√n) but with a smaller constant). For very large n, probabilistic tests (e.g., Miller–Rabin) are used in practice; for DSA and interviews, trial division up to √n is usually enough.

Python Implementation

is_prime(n) — Trial Division

def is_prime(n: int) -> bool:
    if n < 2:
        return False
    if n == 2:
        return True
    if n % 2 == 0:
        return False
    d = 3
    while d * d <= n:
        if n % d == 0:
            return False
        d += 2
    return True

Line-by-Line Explanation

• n < 2 — Primes are ≥ 2; 0 and 1 are not prime.
• n == 2 — 2 is the only even prime; return early.
• n % 2 == 0 — Any other even number is composite.
• while d * d <= n — Check only up to √n. We use d * d <= n to avoid floating-point √n. We start d = 3 and add 2 each time (odd divisors only).
• If any such d divides n, return False. If the loop finishes, no divisor was found, so return True.

Time and Space Complexity

Time: O(√n). The loop runs at most about √n times (only odd d, so roughly √n/2 iterations). Each iteration does a remainder and a multiply; both O(1) for fixed-size integers. For arbitrary-precision integers, cost per division grows with the number of digits; we still say O(√n) in terms of the value n when analyzing algorithm design.

Space: O(1). Only a few variables.

Edge Cases

• n < 2: Return False. 0 and 1 are not prime.
• n == 2: Return True. Handle before the “even” check so we don’t incorrectly label 2 as composite.
• Large n: Trial division is fine for n up to around 10¹² in practice (√n ≈ 10⁶). For bigger n, use a probabilistic test or accept that trial division is slow.

Counting Primes and Listing Primes

To answer “how many primes ≤ N?” or “list all primes ≤ N,” testing each number with is_prime would take about O(N √N) time—too slow for large N. The Sieve of Eratosthenes (next topic, 4.3) does this in O(N log log N) by marking composites once. Here we only note the goal; the algorithm is in the next section.

Common Mistakes

• Treating 1 as prime: 1 has only one positive divisor (itself). By definition, primes have exactly two divisors (1 and n). So 1 is not prime.
• Checking up to n−1 instead of √n: That leaves the algorithm correct but O(n) instead of O(√n). Always use the √n bound.
• Using floating-point sqrt(n): int(n**0.5) can have rounding errors for large n. Prefer while d * d <= n so all arithmetic is integer.
• Forgetting to handle 2: If you do “if n % 2 == 0: return False” before “if n == 2: return True”, you’ll incorrectly say 2 is not prime.

Interview Insight

Practice Problems

• Implement is_prime(n) with trial division (up to √n, odd divisors after 2).
• Given N, count how many primes are in the range [2, N] using trial division for each (then compare with the Sieve in the next section).
• Find the smallest prime factor of n (or return n if prime)—same loop as primality test, but return the first d that divides n.
• Check if n is a “prime power” (n = p^k for some prime p and k ≥ 1): after confirming n > 1, find the smallest prime factor p; then check if n is a power of p.

Summary

• A prime is an integer ≥ 2 whose only positive divisors are 1 and itself. 1 is not prime. 2 is the only even prime.
• Primality test: Trial division—check if any d in [2, √n] divides n. If yes, composite; if no, prime. Optimize by checking 2 then only odd divisors. Time O(√n), space O(1).
• For “list/count primes ≤ N,” use the Sieve of Eratosthenes (next topic)—O(N log log N)—not N separate trial divisions.
• Edge cases: n < 2 → false; n == 2 → true; use integer comparison d*d <= n instead of floating-point √n to avoid rounding issues.

4.3 Sieve of Eratosthenes

Introduction

The Sieve of Eratosthenes is an ancient algorithm that finds all prime numbers up to a given limit N by repeatedly marking multiples of primes as composite. Instead of testing each number with trial division (which would cost about O(N √N) for the whole range), the sieve marks each composite number once—or a small number of times—yielding a total cost of O(N log log N) time and O(N) space. It is the standard way to “list all primes ≤ N” or “count primes ≤ N” and is a must-know for number theory, competitive programming, and interviews.

Real-World Analogy

Imagine a long list of numbers from 2 to N written on a board. You go through the list in order. When you see a number that hasn’t been crossed out, it’s prime—so you cross out all its multiples (4, 6, 8, … for 2; then 6, 9, 12, … for 3; and so on). When you’re done, every number still left is prime. You never “test” a number by dividing it; you only erase multiples of numbers you’ve already declared prime. That’s the sieve: “sift out” composites by marking multiples of each prime.

Formal Definition

We only need to start crossing multiples from p = 2 up to p = √N, because any composite ≤ N has a prime factor ≤ √N—so it will already be marked when we process that factor.

Why This Topic Matters

• Count primes / list primes: Many problems ask “how many primes ≤ N?” or “output all primes ≤ N.” The sieve answers both in one pass.
• Precomputation: Once you’ve sieved up to N, you have O(1) primality checks for any number ≤ N (just look up the array). Useful when you need many such checks.
• Prime factorization, divisors: A common variant stores the smallest prime factor (SPF) for each number instead of just a boolean. That allows fast factorization and divisor enumeration.
• Interview staple: “Count primes less than n” is a classic LeetCode-style problem; the expected solution is the sieve.

Mental Model

Think of the sieve as “every composite number has a smallest prime factor.” When we process prime p, we mark all multiples of p. The first time a composite m gets marked is when we process its smallest prime factor. So we’re not “testing” each number—we’re just ensuring each composite gets marked once (by its smallest prime factor). That’s why the total work is much less than N × √N.

  For each prime p in [2, √N]:
      Mark 2p, 3p, 4p, ... (multiples of p) as composite.

  After the loop: any unmarked number in [2, N] is prime.
  Key: we only iterate p up to √N; multiples beyond N are skipped.
        

Step-by-Step Breakdown

1. Create: An array is_prime of length N+1. Set is_prime[0] = is_prime[1] = False (0 and 1 are not prime). Set is_prime[2..N] = True (assume prime until marked composite).
2. Outer loop: For p from 2 to N (or to √N for the optimized version): if is_prime[p] is still True, then p is prime.
3. Mark composites: For each multiple of p—i.e. p*2, p*3, p*4, ...—set is_prime[k] = False for k = 2*p, 3*p, ... (stop when k > N).
4. Optional optimization: Start marking from p*p instead of 2*p. Why? Because 2*p, 3*p, ..., (p-1)*p have already been marked when we processed smaller primes (2, 3, …, p−1). So we only need to mark p², p²+p, p²+2p, ... up to N.
5. Result: After the loop, collect all i in [2, N] such that is_prime[i] is True—those are the primes. Or count them.

ASCII Diagram

Sieving primes up to 30. We mark multiples of 2, then 3, then 5 (we don’t need to go beyond √30 ≈ 5). X = composite, . = prime (unchanged).

  p=2: mark 4,6,8,10,12,14,16,18,20,22,24,26,28,30
  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
  .  .  X  .  X  .  X  .  X  .  X  .  X  .  X  .  X  .  X  .  X  .  X  .  X  .  X  .  X

  p=3: mark 9,15,21,27
  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
  .  .  X  .  X  .  X  X  X  .  X  .  X  X  .  X  .  X  .  X  .  X  .  X  .  X  X  .  X

  p=5: mark 25
  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
  .  .  X  .  X  .  X  X  X  .  X  .  X  X  .  X  .  X  .  X  .  X  .  X  X  .  X  .  X

  Primes: 2, 3, 5, 7, 11, 13, 17, 19, 23, 29
        

Evolution: Brute Force → Sieve → Optimized Sieve

Brute Force

For each number i from 2 to N, call is_prime(i) (trial division). Time: O(N √N). Space: O(1) per call. Too slow for large N.

Sieve of Eratosthenes (Basic)

One array; for each p from 2 to N, if p is prime, mark all multiples of p. Time: O(N log log N) (see below). Space: O(N). Much faster than brute force.

Optimized Sieve

(1) Outer loop only to √N—once we’ve processed primes up to √N, all composites ≤ N are already marked. (2) Start marking multiples at p²—smaller multiples were marked by smaller primes. (3) Optional: use a list of booleans for odd numbers only (half the memory, index mapping). The asymptotic time remains O(N log log N); constants improve.

Python Implementation

Basic Sieve: List All Primes ≤ N

def sieve(n: int) -> list[int]:
    if n < 2:
        return []
    is_prime = [True] * (n + 1)
    is_prime[0] = is_prime[1] = False
    p = 2
    while p * p <= n:
        if is_prime[p]:
            for k in range(2 * p, n + 1, p):
                is_prime[k] = False
        p += 1
    return [i for i in range(2, n + 1) if is_prime[i]]

Count Primes Only (Same Idea)

def count_primes(n: int) -> int:
    if n < 2:
        return 0
    is_prime = [True] * (n + 1)
    is_prime[0] = is_prime[1] = False
    p = 2
    while p * p <= n:
        if is_prime[p]:
            for k in range(2 * p, n + 1, p):
                is_prime[k] = False
        p += 1
    return sum(1 for i in range(2, n + 1) if is_prime[i])

Optimized: Start Marking at p²

def sieve_optimized(n: int) -> list[int]:
    if n < 2:
        return []
    is_prime = [True] * (n + 1)
    is_prime[0] = is_prime[1] = False
    p = 2
    while p * p <= n:
        if is_prime[p]:
            for k in range(p * p, n + 1, p):
                is_prime[k] = False
        p += 1
    return [i for i in range(2, n + 1) if is_prime[i]]

The only change is range(2 * p, ...) → range(p * p, ...). Correct because multiples p×2, p×3, …, p×(p−1) were already marked when we processed primes 2, 3, …, p−1.

Line-by-Line Explanation (Basic Sieve)

• if n < 2: return [] — No primes below 2.
• is_prime = [True] * (n + 1) — Index 0..n; we’ll use indices 0 and 1 for 0 and 1, and 2..n for numbers 2..n.
• is_prime[0] = is_prime[1] = False — 0 and 1 are not prime.
• while p * p <= n — We only need to consider p up to √n. If p > √n, then any multiple p×k ≥ p² > n, so we’d mark nothing new.
• if is_prime[p] — If p was already marked composite (by a smaller prime), skip; we’ve already marked its multiples.
• for k in range(2*p, n+1, p) — Mark 2p, 3p, 4p, … up to n. Step size p gives exactly the multiples of p.
• return [i for i in range(2, n+1) if is_prime[i]] — Collect all indices that are still True; those are the primes.

Time Complexity

For each prime p ≤ √N, we mark about N/p numbers (multiples of p). Total operations is roughly:

N/2 + N/3 + N/5 + N/7 + … (sum over primes p ≤ N). Factor out N: N × (1/2 + 1/3 + 1/5 + …). The sum of 1/p over primes ≤ N is known to be about log log N. So total is O(N log log N).

Stopping the outer loop at √N doesn’t change the dominant term (most marks come from small primes). Starting the inner loop at p² reduces the number of marks per prime but the same asymptotic holds. So the sieve is O(N log log N) time.

Space Complexity

We use one boolean array of length N+1 → O(N) space. If we pack one bit per number (or only store odd indices), we can get O(N) bits, i.e. O(N/8) bytes—same big-O, smaller constant.

Edge Cases

• n < 2: No primes. Return [] or count 0.
• n == 2: One prime (2). The outer loop runs p=2, 2×2=4 > 2 so the inner loop runs no times; is_prime[2] stays True. Correct.
• Large N: O(N) space can be an issue for N in the hundreds of millions. Use a segmented sieve or bit-packed storage if needed.

Common Mistakes

• Including 0 or 1 as prime: Set is_prime[0] = is_prime[1] = False at the start.
• Looping p to N instead of √N: Correct but wasteful. Stopping at √N is enough and faster.
• Starting the inner loop at 2*p but forgetting p² optimization: Correct either way; starting at p² is a performance improvement.
• Off-by-one in range: Use range(2, n + 1) so that n itself is included when we collect primes. Use range(p * p, n + 1, p) so the last multiple ≤ n is marked.

Pattern Recognition

“Mark multiples of each prime” is a recurring idea: when you want to eliminate all numbers that are “multiples of something,” iterate over the “something” (here, primes) and mark multiples in one batch. The same pattern appears in segmented sieves (sieving a range [L, R]) and in some divisor-sum precomputations.

Interview Insight

Practice Problems

• Implement the sieve and return the list of primes ≤ N; then implement count-only without building the list.
• LeetCode 204: Count Primes — use the sieve; handle n ≤ 2.
• Precompute a “smallest prime factor” (SPF) array: for each number, store its smallest prime divisor. (When marking multiples of p, if spf[k] is not yet set, set spf[k] = p.) Then use SPF to factor any number ≤ N quickly.
• Segmented sieve: find primes in an interval [L, R] without building a full sieve up to R (useful when R is huge but R−L is moderate).

Summary

• The Sieve of Eratosthenes finds all primes ≤ N by marking multiples of each prime; unmarked numbers are prime. Use a boolean array and iterate p from 2 to √N; for each prime p, mark multiples (start at 2p or p²).
• Time: O(N log log N). Space: O(N).
• Evolution: brute force (N × trial division) → sieve → optimized sieve (outer loop to √N, inner from p²).
• Edge cases: n < 2 → empty list / 0; always set is_prime[0] = is_prime[1] = False.
• Standard for “list/count primes ≤ N”; variant with smallest-prime-factor array enables fast factorization.

4.4 GCD & LCM

Introduction

The Greatest Common Divisor (GCD) of two integers is the largest positive integer that divides both. The Least Common Multiple (LCM) is the smallest positive integer that both divide. GCD and LCM are fundamental in number theory, modular arithmetic, simplifying fractions, and many DSA problems (e.g., “minimum steps to reach,” “repeat pattern,” “coprime pairs”). This section covers definitions, the Euclidean algorithm for GCD (and why it’s fast), the link between GCD and LCM, and clean Python implementations—including edge cases and interview-ready phrasing.

Real-World Analogy

Imagine you have two ropes of length 12 and 18. You want to cut both into equal pieces (no leftover), with each piece as long as possible. The piece length must divide both 12 and 18—so it’s a common divisor. The greatest such length is 6. That’s the GCD(12, 18) = 6. Now imagine two gears that turn every 12 and 18 seconds. When do they align (both at the start)? The first time is at the smallest positive time that both 12 and 18 divide—that’s the LCM(12, 18) = 36. So: GCD = “largest common measure”; LCM = “earliest common multiple.”

Formal Definition

For negative numbers, we usually work with absolute values: GCD(−12, 18) = GCD(12, 18) = 6. So in code we often take gcd(abs(a), abs(b)).

Why This Topic Matters

• Modular arithmetic and cryptography: Extended GCD (next topic) is used to compute modular inverses—essential for RSA and many algorithms.
• Simplifying fractions: A fraction a/b in lowest terms is (a/gcd(a,b)) / (b/gcd(a,b)).
• Periodicity and “when do two events align?”: Problems like “two runners with periods 12 and 18—when do they meet?” reduce to LCM (or GCD of periods in special cases).
• Interview problems: “Minimum steps to make two numbers equal,” “number of coprime pairs,” “split array into groups with GCD K”—all use GCD/LCM.

Mental Model

Think of a and b in terms of their prime factorizations. The GCD takes the minimum exponent for each prime (the “overlap”). The LCM takes the maximum exponent (the “union”). So for 12 = 2²×3 and 18 = 2×3²: GCD = 2¹×3¹ = 6, LCM = 2²×3² = 36. And indeed 6 × 36 = 12 × 18. That’s the intuition behind GCD × LCM = a × b (for positive a, b).

  a = 12 = 2² × 3¹    b = 18 = 2¹ × 3²
  GCD = common part    = 2¹ × 3¹ = 6
  LCM = combined part  = 2² × 3² = 36
  Check: 6 × 36 = 216 = 12 × 18
        

GCD: Brute Force → Euclidean Algorithm

Brute Force

List all divisors of a and b; take the largest number that appears in both. Finding divisors of n costs O(√n), so total is about O(√a + √b). Works for small numbers but is slow and clumsy.

Euclidean Algorithm (Optimal)

Key identity: GCD(a, b) = GCD(b, a mod b). Why? Any common divisor of a and b also divides a − b, a − 2b, …, so it divides a mod b. So the set of common divisors of (a, b) and (b, a mod b) is the same; hence the GCD is the same. Base case: GCD(a, 0) = a (the largest divisor of a is a). So we repeatedly replace (a, b) with (b, a mod b) until the second number is 0; then the first number is the GCD.

Each step roughly halves the larger number (a mod b < b, and often much smaller), so the number of steps is O(log min(a, b)). No factorization needed—just remainders.

Step-by-Step: Euclidean Algorithm

1. Start with (a, b). If b = 0, return a (base case).
2. Otherwise compute r = a mod b and return GCD(b, r).
3. Repeat until the second argument is 0. The first argument at that point is the GCD.

  GCD(48, 18)
  = GCD(18, 48 mod 18) = GCD(18, 12)
  = GCD(12, 18 mod 12) = GCD(12, 6)
  = GCD(6, 12 mod 6)   = GCD(6, 0)
  = 6
        

LCM from GCD

For positive integers a, b: LCM(a, b) = a × b / GCD(a, b). So compute GCD first, then LCM = (a // g) * b (or a * (b // g)) to avoid overflow: divide one number by the GCD first, then multiply by the other. In Python, integers are arbitrary size, but (a // gcd) * b keeps the intermediate values smaller and is good practice for other languages.

Python Implementation

GCD — Recursive

def gcd(a: int, b: int) -> int:
    a, b = abs(a), abs(b)
    if b == 0:
        return a
    return gcd(b, a % b)

GCD — Iterative

def gcd_iter(a: int, b: int) -> int:
    a, b = abs(a), abs(b)
    while b:
        a, b = b, a % b
    return a

LCM (Using GCD)

def lcm(a: int, b: int) -> int:
    if a == 0 or b == 0:
        return 0
    return (a // gcd(abs(a), abs(b))) * abs(b)

We use (a // g) * b so the division happens first (no overflow). We take absolute values so LCM is defined for negatives in a consistent way (e.g., LCM(−12, 18) = LCM(12, 18) = 36).

Using the Standard Library

import math
math.gcd(a, b)   # GCD; handles 0 and negatives sensibly
# Python 3.9+:
math.lcm(a, b)   # LCM; returns 0 if either argument is 0

In practice, use math.gcd and math.lcm when available; implement by hand when practicing the algorithm or when you need extended GCD (next topic).

Line-by-Line Explanation (Iterative GCD)

• a, b = abs(a), abs(b) — GCD is defined for non-negative numbers; negatives are handled by taking absolute values so the result is non-negative.
• while b: — Loop until the second argument is 0. When b becomes 0, the current a is the GCD.
• a, b = b, a % b — Replace (a, b) with (b, a mod b). This is one step of the Euclidean algorithm; the GCD is unchanged.
• return a — When b = 0, GCD(a, 0) = a, so we return a.

Time Complexity

GCD: Each step reduces the larger argument. It can be shown that in two steps, the larger number is at least halved (e.g., if a > b, then (a mod b) < b, and after one more step we have (b, a mod b) with both values bounded by the previous larger value). So the number of steps is O(log min(a, b)). Each step is O(1) for fixed-size integers; for big integers, cost per step is proportional to the number of digits. So O(log min(a, b)) iterations; with cost per iteration O(log n) for big integers, total is O(log² min(a, b)) in that model. We usually state it as O(log min(a, b)) for the number of iterations.

LCM: One GCD plus one division and one multiplication → same as GCD, O(log min(a, b)).

Space Complexity

Iterative GCD: O(1) extra space. Recursive GCD: O(log min(a, b)) stack depth. Prefer iterative if you care about stack or want to avoid recursion limits for huge inputs.

Edge Cases

• One or both zero: GCD(0, b) = b, GCD(a, 0) = a, GCD(0, 0) = 0 (by convention). LCM(0, b) = LCM(a, 0) = 0 (no positive multiple of 0 in the usual sense; 0 is the standard convention).
• Negatives: GCD and LCM are usually defined for positive integers; in code we take absolute values so GCD(−12, 18) = 6, LCM(−12, 18) = 36.
• Equal numbers: GCD(a, a) = a, LCM(a, a) = a.

Common Mistakes

• Forgetting to handle zero: GCD(a, 0) must return a, not crash or loop forever. In iterative code, while b: correctly stops when b = 0.
• LCM overflow: Using a * b // gcd(a, b) in a language with fixed-width integers can overflow. Use (a // gcd(a, b)) * b.
• LCM when one is zero: Define LCM(a, 0) = 0 (or avoid calling LCM with 0). Don’t divide by zero.
• Assuming a and b are positive: If the problem allows negatives, use abs(a), abs(b) for a non-negative GCD.

Optimization Insight

Pattern Recognition

“Reduce (a, b) to (b, a mod b)” is the same idea as “reduce the problem size using the remainder.” Many number-theoretic algorithms (extended GCD, modular inverse, continued fractions) build on this. When you see “greatest common divisor,” “simplify fraction,” or “coprime,” think GCD first.

Interview Insight

Practice Problems

• Implement gcd and lcm (iterative and recursive for GCD) and test with (0, b), (a, 0), negatives, and (48, 18).
• Simplify a fraction (a, b) to lowest terms: divide numerator and denominator by GCD(a, b).
• Count pairs (i, j) in an array such that GCD(arr[i], arr[j]) = 1 (coprime pairs)—use GCD in the inner check or use inclusion–exclusion with prime factors.
• Given two numbers and a step, “minimum steps to make both equal” often involves GCD of the steps and the difference.
• LCM of an array: LCM(a, b, c) = LCM(LCM(a, b), c); use a loop and the two-argument LCM.

Summary

• GCD(a, b) = largest positive integer that divides both. LCM(a, b) = smallest positive integer that both divide. For positive a, b: LCM × GCD = a × b.
• Euclidean algorithm: GCD(a, b) = GCD(b, a mod b); base case GCD(a, 0) = a. Iterative: while b ≠ 0, (a, b) = (b, a % b); return a. Time O(log min(a, b)), space O(1) iterative.
• LCM: LCM(a, b) = (a // GCD(a, b)) × b (or use math.lcm in Python 3.9+). Handle zero: LCM(a, 0) = 0.
• Use abs(a), abs(b) when inputs can be negative. Prefer math.gcd and math.lcm in production; implement by hand for learning and for extended GCD (next topic).

4.5 Extended Euclidean Algorithm

Introduction

The Extended Euclidean Algorithm not only computes GCD(a, b) but also finds integers x and y such that ax + by = gcd(a, b). This identity is called Bézout’s identity. The coefficients x, y are essential for computing modular inverses (next topic), solving linear Diophantine equations, and many cryptographic algorithms. This section builds on the ordinary Euclidean algorithm and shows how to “extend” it to recover the coefficients.

Real-World Analogy

Imagine you have two piles of coins with a and b coins. You’re allowed to add or remove multiples of each pile. The Euclidean algorithm tells you the “unit” of value you can always form (the GCD). The extended algorithm tells you how to form it: “take x copies of the first pile and y copies of the second (with removal meaning negative copies), and you get exactly gcd(a, b).” So you get both the number (GCD) and a “recipe” (x, y) to express it as a combination of a and b.

Formal Definition

We usually want one concrete solution (x, y). The extended algorithm gives one by propagating coefficients backward through the Euclidean steps.

Why This Topic Matters

• Modular inverse: Finding x such that a·x ≡ 1 (mod m) reduces to solving a·x + m·y = 1. That has a solution iff gcd(a, m) = 1; the extended algorithm gives x (mod m). Critical for RSA and many algorithms.
• Linear Diophantine equations: Equations like a·x + b·y = c have integer solutions (x, y) iff gcd(a, b) | c. The extended algorithm gives a particular solution when c = gcd(a, b); scale to get one for general c.
• Interview and contest problems: “Compute modular inverse,” “find one solution to a·x + b·y = c”—both rely on extended GCD.

Mental Model

In the Euclidean algorithm we repeatedly replace (a, b) with (b, a mod b). At each step we have a = q·b + r, so r = a − q·b. If we already know how to write the GCD as a combination of b and r (i.e., b·x₁ + r·y₁ = g), then we can substitute r = a − q·b and get a combination of a and b: a·y₁ + b·(x₁ − q·y₁) = g. So we propagate coefficients backward: from (b, r) we get coefficients for (a, b). Base case: when b = 0, we have a·1 + 0·0 = a = gcd(a, 0).

  At step: a = q·b + r   ⇒   r = a − q·b
  If  b·x₁ + r·y₁ = g  then  b·x₁ + (a − q·b)·y₁ = g
  ⇒  a·y₁ + b·(x₁ − q·y₁) = g   →  new (x, y) = (y₁, x₁ − q·y₁)
        

Step-by-Step: Recursive Form

1. Base case: If b = 0, then gcd(a, 0) = a and a·1 + 0·0 = a. So return (a, 1, 0) meaning (gcd, x, y).
2. Recursive step: Compute (g, x₁, y₁) = extended_gcd(b, a % b). So g = b·x₁ + (a % b)·y₁. We have a = q·b + (a % b) with q = a // b. So (a % b) = a − q·b. Substitute: g = b·x₁ + (a − q·b)·y₁ = a·y₁ + b·(x₁ − q·y₁). So coefficients for (a, b) are x = y₁, y = x₁ − q·y₁. Return (g, x, y).

Python Implementation

Recursive Extended GCD

def extended_gcd(a: int, b: int) -> tuple[int, int, int]:
    """Returns (g, x, y) such that a*x + b*y = g = gcd(a, b)."""
    a, b = abs(a), abs(b)
    if b == 0:
        return (a, 1, 0)
    q = a // b
    g, x1, y1 = extended_gcd(b, a % b)
    # g = b*x1 + (a % b)*y1, and a % b = a - q*b
    # so g = a*y1 + b*(x1 - q*y1)
    x, y = y1, x1 - q * y1
    return (g, x, y)

Iterative Extended GCD

Keep track of (a, b) and coefficients (x_a, y_a) for current a and (x_b, y_b) for current b, such that a = orig_a·x_a + orig_b·y_a and b = orig_a·x_b + orig_b·y_b. Initially a = orig_a, b = orig_b → (x_a, y_a) = (1, 0), (x_b, y_b) = (0, 1). When we replace (a, b) with (b, a − q·b), we update the coefficient vectors the same way: new (x_b, y_b) becomes (x_a − q·x_b, y_a − q·y_b). When b = 0, (g, x, y) = (a, x_a, y_a).

def extended_gcd_iter(a: int, b: int) -> tuple[int, int, int]:
    """Returns (g, x, y) such that a*x + b*y = g = gcd(a, b)."""
    a, b = abs(a), abs(b)
    x_a, y_a = 1, 0
    x_b, y_b = 0, 1
    while b:
        q = a // b
        a, b = b, a % b
        x_a, y_a, x_b, y_b = x_b, y_b, x_a - q * x_b, y_a - q * y_b
    return (a, x_a, y_a)

Line-by-Line Explanation (Recursive)

• if b == 0: return (a, 1, 0) — Base case: a·1 + 0·0 = a = gcd(a, 0).
• q = a // b — Quotient so that a = q·b + (a % b).
• g, x1, y1 = extended_gcd(b, a % b) — Get GCD and coefficients for (b, a % b): g = b·x1 + (a % b)·y1.
• x, y = y1, x1 - q * y1 — Substitute (a % b) = a − q·b to get g = a·y1 + b·(x1 − q·y1), so x = y1, y = x1 − q·y1.
• return (g, x, y) — One solution to a·x + b·y = g.

Time and Space Complexity

Time: Same as the Euclidean algorithm—O(log min(a, b)) steps. Each step does a division and a few arithmetic operations. So O(log min(a, b)).

Space: Recursive version O(log min(a, b)) stack depth. Iterative version O(1) extra variables.

Edge Cases

• b = 0: Return (a, 1, 0). Correct: a·1 + 0·0 = a.
• a = 0, b ≠ 0: extended_gcd(0, b) → (b, 0, 1). Check: 0·0 + b·1 = b = gcd(0, b) ✓.
• Negative inputs: We take abs(a), abs(b) so we work with non-negative numbers. The GCD is non-negative; x, y can be negative (e.g., 35·1 + 15·(−2) = 5).

Application: Modular Inverse (Preview)

We want x such that a·x ≡ 1 (mod m). That means a·x + m·y = 1 for some y. This has a solution iff gcd(a, m) = 1. Run extended_gcd(a, m) to get (1, x, y). The x you get might be negative; reduce modulo m: x % m (or (x % m + m) % m) is the modular inverse of a modulo m. This is covered in detail in the next topic (Modular Inverse).

Common Mistakes

• Wrong order of (x, y): The equation is a·x + b·y = g. So the first coefficient multiplies a, the second multiplies b. Don’t swap x and y when returning or when using the result for modular inverse (inverse of a mod m uses the coefficient that multiplies a).
• Forgetting to reduce x modulo m for inverse: extended_gcd returns an x that might be negative or larger than m. The modular inverse is (x % m + m) % m (or x % m in Python since % is non-negative when divisor is positive).
• Assuming inverse exists: a has an inverse mod m only if gcd(a, m) = 1. Always check g == 1 before using x as the inverse.

Interview Insight

Practice Problems

• Implement recursive and iterative extended GCD and verify a·x + b·y = g for (35, 15), (48, 18), (0, 7).
• Using extended GCD, compute the modular inverse of a modulo m when gcd(a, m) = 1 (handle negative x).
• Solve a·x + b·y = c in integers: first run extended_gcd(a, b). If g does not divide c, no solution. Otherwise one solution is (x·(c//g), y·(c//g)); describe the full solution set.

Summary

• Bézout’s identity: There exist integers x, y with a·x + b·y = gcd(a, b). The Extended Euclidean Algorithm computes (g, x, y).
• Recursive: Base case (b = 0) → (a, 1, 0). Otherwise (g, x1, y1) = extended_gcd(b, a % b); then x = y1, y = x1 − (a//b)·y1; return (g, x, y).
• Iterative: Maintain (a, b) and coefficient vectors (x_a, y_a), (x_b, y_b); update with the same recurrence as (a, b) when doing (a, b) = (b, a % b).
• Time O(log min(a, b)), space O(1) iterative. Use extended GCD to get the modular inverse of a mod m (x mod m when g = 1)—next topic.

4.6 Fast Exponentiation

Introduction

Fast exponentiation (also called binary exponentiation or square-and-multiply) computes a^b using only O(log b) multiplications instead of b−1. The idea is to use the binary representation of the exponent: write b in base 2, and combine precomputed powers a^2⁰, a^2¹, a^2², … by squaring repeatedly and multiplying when the current bit is 1. This is essential for modular exponentiation (a^b mod m)—used in RSA, hashing, and many algorithms—where we must keep intermediate results modulo m to avoid overflow. This section covers the idea, the algorithm, and clean Python implementations.

Real-World Analogy

To get 3¹³, you could multiply 3 by itself 12 times. Instead, build powers by doubling: 3¹, 3², 3⁴, 3⁸ (each is the previous squared). Then 13 in binary is 1101, so 3¹³ = 3⁸ × 3⁴ × 3¹—multiply only the powers that correspond to 1-bits. You do about four squarings and a few multiplications instead of 12. Same idea as “repeated doubling” in any setting: grow exponentially, then combine.

Formal Definition

For modular exponentiation we compute a^b mod m: after each multiplication or squaring, take the result mod m so numbers stay in [0, m−1].

Why This Topic Matters

• Modular exponentiation: RSA and many crypto primitives need a^b mod m for huge b. Naive a × a × … mod m would take b steps; fast exponentiation does O(log b) steps.
• Python’s pow(a, b, m): The built-in three-argument form does exactly this. Knowing the algorithm explains why it’s fast and how to implement it when needed.
• Matrix exponentiation, recurrence relations: The same “binary exponentiation” idea applies to raising a matrix to power b (e.g., Fibonacci in O(log n))—covered later in the course.

Mental Model

Scan the bits of b from right to left (LSB first). Maintain a running power base = a^(2^i) (start with base = a, then square each time we move left). Maintain a result res (start 1). When the current bit of b is 1, multiply res by base. After each step, square base (for the next bit) and shift b right. When b becomes 0, res is a^b.

  b in binary:  ... b₂ b₁ b₀
  a^b = (a^1)^b₀ × (a^2)^b₁ × (a^4)^b₂ × ...
  So: res = 1; base = a; for each bit of b (LSB first):
      if bit is 1: res *= base
      base *= base; b //= 2
  Return res.
        

Evolution: Naive → Fast Exponentiation

Naive (Brute Force)

Multiply res = 1 by a exactly b times. Time O(b), space O(1). For b in the millions or more, this is impractical.

Fast Exponentiation (Square-and-Multiply)

Repeated squaring + multiply when bit is 1. Number of iterations = number of bits of b = ⌊log₂ b⌋ + 1. Each iteration does one or two multiplications (squaring base, and possibly multiplying res by base). Time O(log b), space O(1) iterative.

Step-by-Step Example

Compute 3¹³. 13 in binary is 1101 (LSB = 1).

  res=1, base=3, b=13
  b&1=1 → res=1*3=3, base=3²=9,   b=6
  b&1=0 → res=3,   base=9²=81,  b=3
  b&1=1 → res=3*81=243, base=81²=6561, b=1
  b&1=1 → res=243*6561=1594323, base=..., b=0
  Return 1594323. Check: 3^13 = 1594323 ✓
        

We did four “bit” steps (squaring each time, and three multiplies into res). So about 4 squarings + 3 multiplies instead of 12 multiplies.

Modular Exponentiation

To compute a^b mod m, reduce modulo m after every multiplication and squaring. That keeps all intermediates in [0, m−1] and avoids overflow. The algorithm is the same; only the multiplication and squaring are done mod m.

# In code: res = (res * base) % m; base = (base * base) % m

Python’s pow(a, b, m) does exactly this when m is provided. Use it in production; implement by hand when learning or when you need a custom variant (e.g., matrix exponentiation).

Python Implementation

Iterative: a^b (no mod)

def power(a: int, b: int) -> int:
    """Returns a^b for non-negative b."""
    if b == 0:
        return 1
    res = 1
    base = a
    while b:
        if b & 1:
            res *= base
        base *= base
        b >>= 1
    return res

Iterative: a^b mod m

def power_mod(a: int, b: int, m: int) -> int:
    """Returns (a^b) % m for non-negative b. Assumes m >= 1."""
    if b == 0:
        return 1 % m
    a %= m
    res = 1
    base = a
    while b:
        if b & 1:
            res = (res * base) % m
        base = (base * base) % m
        b >>= 1
    return res

Recursive (Alternative)

def power_mod_rec(a: int, b: int, m: int) -> int:
    if b == 0:
        return 1 % m
    a %= m
    half = power_mod_rec(a, b // 2, m)
    half = (half * half) % m
    if b & 1:
        half = (half * a) % m
    return half

Recursive idea: a^b = (a^b//2)² × a^b%2. Same O(log b) steps; uses O(log b) stack.

Line-by-Line Explanation (Iterative with Mod)

• if b == 0: return 1 % m — a⁰ = 1; reduce mod m for consistency (handles m = 1).
• a %= m — Work with a in [0, m−1] so base stays small.
• res = 1, base = a — Result starts 1; current power of a is a¹.
• while b: — Process each bit of b until b = 0.
• if b & 1: res = (res * base) % m — If the LSB is 1, multiply result by current base (that bit contributes to the exponent).
• base = (base * base) % m — Square: a^{2^i} → a^{2^(i+1)}.
• b >>= 1 — Shift right to process the next bit.

Time and Space Complexity

Time: O(log b) iterations. Each iteration does O(1) multiplications (with mod, each multiplication is O(log m) for fixed-size integers, or O((log m)²) for naive big-int). We state O(log b) in terms of the number of steps; with big integers, total can be O(log b · (log a + log m)²) or similar depending on model.

Space: O(1) for the iterative version (a few variables). Recursive version O(log b) stack depth.

Edge Cases

• b = 0: a⁰ = 1. Return 1 (or 1 % m).
• a = 0, b > 0: 0^b = 0. The loop will leave res = 0 after the first 1-bit (base becomes 0). Or handle explicitly: if a % m == 0 and b > 0, return 0.
• m = 1: Any integer mod 1 is 0. So a^b mod 1 = 0 for any a, b. The code 1 % 1 = 0 and (res * base) % 1 = 0 is correct.
• Negative exponent: a^−b = 1/(a^b). Not usually needed for modular exponentiation in DSA; if needed, use modular inverse (when gcd(a, m) = 1, a⁻¹ mod m exists and pow(a, -b, m) = pow(a⁻¹, b, m) in Python 3.8+).

Common Mistakes

• Forgetting to reduce mod m at each step: If you only take mod at the end, intermediates can overflow (in other languages) or become huge (slow in Python). Always do res = (res * base) % m and base = (base * base) % m.
• Using b - 1 instead of b >>= 1: We must process bits, not decrement b. Use b >>= 1 (or b //= 2).
• Wrong order of operations: Check the bit (b & 1) before squaring base and shifting b. Multiply into res when bit is 1, then always square base and shift.

Pattern Recognition

“Binary decomposition of the exponent” is a recurring pattern: any operation that is associative (like multiplication, matrix multiplication) can be raised to power b in O(log b) “multiplications” by using the binary expansion of b. Same idea underlies matrix exponentiation for Fibonacci and linear recurrences.

Interview Insight

Practice Problems

• Implement power_mod(a, b, m) iteratively and verify against pow(a, b, m) for small and large b.
• Compute the last k digits of a^b (i.e., a^b mod 10^k) using fast exponentiation.
• LeetCode-style: “Count good numbers” or problems that need (base)^(exponent) mod mod—use fast exponentiation.
• Later: apply the same “binary exponentiation” idea to matrices (e.g., compute the n-th Fibonacci number in O(log n) using matrix power).

Summary

• Fast exponentiation computes a^b in O(log b) multiplications by using the binary expansion of b: a^b = product of a^{2^i} over bits that are 1. Algorithm: res = 1, base = a; for each bit (LSB first): if bit 1, res *= base; base *= base; b //= 2.
• Modular exponentiation: Reduce mod m after every multiplication and squaring so intermediates stay in [0, m−1]. Use pow(a, b, m) in Python.
• Time O(log b), space O(1) iterative. Same idea extends to matrix exponentiation and other associative operations.
• Edge cases: b = 0 → 1; reduce a mod m first; handle a = 0 or m = 1 if needed.

4.7 Modular Arithmetic

Introduction

Modular arithmetic is arithmetic on integers where we only care about the remainder when dividing by a fixed positive integer m (the modulus). Two integers a and b are congruent modulo m, written a ≡ b (mod m), if they leave the same remainder when divided by m—equivalently, if m divides (a − b). Addition, subtraction, and multiplication “work” modulo m: you can reduce before or after the operation and get a consistent result. Division modulo m is different: it requires the modular inverse (next topic). Modular arithmetic is foundational for cryptography, hashing, cyclic structures, and almost every problem that asks for “answer modulo 10⁹+7” in competitive programming.

Real-World Analogy

Think of a clock with 12 hours. If it’s 9 o’clock and you add 5 hours, you get 2 o’clock—not 14. So 9 + 5 ≡ 2 (mod 12). The clock “wraps around” at 12. Similarly, “what day of the week is it 100 days from now?” is modular arithmetic mod 7. The modulus is the size of the cycle; we only care which position we’re in, not how many full cycles have passed.

Formal Definition

In code, “a mod m” usually means the remainder when a is divided by m. In Python, a % m returns a value in [0, m−1] when m > 0 (so −17 % 5 = 3, because −17 = (−4)·5 + 3). In C/Java, a % m can be negative when a is negative; the “canonical” representative is then (a % m + m) % m.

Why This Topic Matters

• Competitive programming and interviews: Problems often ask for the answer “modulo 10⁹+7” or “mod 998244353” to avoid big integers and focus on the algorithm. You must add, subtract, multiply (and sometimes divide) correctly under the modulus.
• Cryptography: RSA and many protocols work in modular arithmetic (mod a large composite or prime).
• Hashing: Hash tables use hash(key) % capacity. Understanding mod avoids off-by-one and negative-index bugs.
• Cyclic behavior: Sequences that repeat (e.g., state machines, periodic signals) are naturally described with modular arithmetic.

Mental Model

Work with numbers as their “remainder when divided by m.” Every integer is equivalent to exactly one value in {0, 1, …, m−1}. When you add or multiply, do the operation and then take the remainder (or take remainders first—see below). So the universe of values is finite: only m “slots,” and everything wraps around.

  Integers mod m:  ... ≡ -2m ≡ -m ≡ 0 ≡ m ≡ 2m ≡ ...  (all same "slot")
                   ... ≡ -m+1 ≡ 1 ≡ m+1 ≡ ...         (another slot)
  We usually pick representatives 0, 1, ..., m-1.
        

Basic Operations Modulo m

For addition, subtraction, and multiplication, the following hold:

• Addition: (a + b) mod m = ((a mod m) + (b mod m)) mod m. So you can reduce a and b first, then add, then reduce again (avoids overflow in other languages).
• Subtraction: (a − b) mod m = ((a mod m) − (b mod m) + m) mod m. The +m ensures the result is non-negative when (a mod m) < (b mod m).
• Multiplication: (a · b) mod m = ((a mod m) · (b mod m)) mod m. Reduce before multiplying to keep intermediates small.

Division and the Need for Inverses

Division modulo m is not “divide then take remainder.” In integers, we don’t have fractions. Instead, to “divide by b” mod m we multiply by the modular inverse of b: a number b⁻¹ such that b · b⁻¹ ≡ 1 (mod m). Then (a / b) mod m is defined as (a · b⁻¹) mod m. The inverse exists iff gcd(b, m) = 1 (e.g., when m is prime, every non-zero b has an inverse). So:

(a / b) mod m = (a · b⁻¹) mod m, where b⁻¹ is the modular inverse of b (next topic). Never compute (a mod m) / (b mod m) as integers and then take mod—that’s wrong.

Congruence Properties

If a ≡ b (mod m) and c ≡ d (mod m), then:

• a + c ≡ b + d (mod m)
• a − c ≡ b − d (mod m)
• a · c ≡ b · d (mod m)
• a^k ≡ b^k (mod m) for any non-negative integer k

So we can replace any term with a congruent one before doing operations. That justifies reducing mod m at each step to keep numbers small.

Python Implementation

Addition and Multiplication

def add_mod(a: int, b: int, m: int) -> int:
    return (a % m + b % m) % m

def sub_mod(a: int, b: int, m: int) -> int:
    return (a % m - b % m + m) % m

def mul_mod(a: int, b: int, m: int) -> int:
    return (a % m * (b % m)) % m

Adding m in sub_mod ensures the result is in [0, m−1] even when a % m < b % m (since in Python a % m is already in [0, m−1], (a % m - b % m) can be negative; adding m and then % m fixes it).

Power: a^b mod m

# Use fast exponentiation; in Python:
pow(a, b, m)   # computes (a^b) % m efficiently

Negative Numbers and “Canonical” Representative

# In Python, a % m is already in [0, m-1] when m > 0
-17 % 5   # 3

# If you get a from a language where % can be negative:
def to_canonical(a: int, m: int) -> int:
    return (a % m + m) % m

Line-by-Line Explanation (sub_mod)

• a % m, b % m — Bring both into [0, m−1].
• a % m - b % m — Can be negative (e.g., 3 − 5 = −2).
• + m — Add one modulus so the value becomes in [0, m−1] (e.g., −2 + 7 = 5).
• % m — Final reduce (redundant if we only added one m, but safe if we ever add more or if the first % was done in a different way).

Time and Space Complexity

Addition, subtraction, multiplication modulo m: O(1) for fixed-size integers; O(log m) or O(log a + log b) for arbitrary-precision. Power mod m: O(log b) using fast exponentiation (previous topic).

Edge Cases

• m = 1: Every integer is ≡ 0 (mod 1). So a mod 1 = 0 for any a. Operations mod 1 always yield 0.
• m < 1: Modulus is usually defined as positive. In code, avoid m ≤ 0 or handle explicitly (Python’s % with negative divisor has a different convention).
• Negative a: In Python, a % m is in [0, m−1]. So (−17) % 5 = 3. No extra fix needed. In other languages, use (a % m + m) % m.
• Large intermediates: Always reduce before and after multiplication so (a * b) % m is computed as ((a % m) * (b % m)) % m to avoid overflow in C++/Java; in Python it’s for speed and clarity.

Common Mistakes

• Dividing without inverse: (a / b) mod m must be (a · b⁻¹) mod m. Don’t use integer division.
• Subtraction giving negative: (a − b) mod m must be in [0, m−1]. Use (a % m − b % m + m) % m.
• Overflow in multiplication: In C++/Java, (a % m) * (b % m) can still overflow if m is large. Use long or ((a % m) * (b % m)) % m with 64-bit; for very large m, use a type that can hold (m−1)² or use a custom big-int approach.
• Assuming mod distributes over everything: (a − b) mod m ≠ (a mod m) − (b mod m) when the right-hand side is negative; you must add m. And (a / b) mod m ≠ (a mod m) / (b mod m).

Optimization Insight

Pattern Recognition

Whenever the problem says “output modulo 10⁹+7” (or similar), all arithmetic in your solution should be done mod m. Counts, sums, products—reduce at each step. If you need to “divide” (e.g., divide by 2 or by n!), use the modular inverse. The pattern is: work in the ring of integers mod m; addition, subtraction, multiplication are safe; division is multiply by inverse.

Interview Insight

Practice Problems

• Implement add_mod, sub_mod, mul_mod and test with negative numbers and large values.
• Compute (a + b + c) mod m and (a · b · c) mod m by reducing at each step.
• Given a, b, m, compute (a − b) mod m correctly when a < b.
• Problems that ask for “number of ways mod 10⁹+7”: use modular arithmetic throughout; when you need to divide by k!, compute k! mod m and then multiply by its inverse mod m.

Summary

• a ≡ b (mod m) iff m | (a − b); equivalently, same remainder when divided by m. We usually work with representatives in [0, m−1].
• Addition/subtraction/multiplication: (a ± b) mod m and (a · b) mod m can be computed by reducing operands first, then doing the operation, then reducing. For subtraction use (a % m − b % m + m) % m to keep result in [0, m−1].
• Division: (a / b) mod m = (a · b⁻¹) mod m; b⁻¹ is the modular inverse (exists when gcd(b, m) = 1). Never use integer division.
• Congruence is preserved under +, −, ·, and powers. Reduce at each step to avoid overflow and keep intermediates small. Use pow(a, b, m) for a^b mod m.

4.8 Modular Inverse

Introduction

The modular inverse of an integer a modulo m is an integer x such that a · x ≡ 1 (mod m). We write x ≡ a⁻¹ (mod m). It lets us “divide” by a in modular arithmetic: (b / a) mod m = (b · a⁻¹) mod m. The inverse exists if and only if gcd(a, m) = 1 (a and m are coprime). When it exists, it is unique modulo m. This section covers when and why the inverse exists, two ways to compute it (Extended Euclidean and Fermat’s little theorem when m is prime), and how to use it in code.

Real-World Analogy

In normal arithmetic, the inverse of 3 is 1/3 because 3 × (1/3) = 1. Modulo m we only have integers, so we can’t use fractions. The modular inverse is the integer that “plays the role” of 1/a: when you multiply a by it, you get 1 (mod m). For example, mod 7 we have 3 × 5 = 15 ≡ 1 (mod 7), so 5 is the inverse of 3 mod 7. “Dividing by 3” mod 7 means multiplying by 5.

Formal Definition

Why gcd(a, m) = 1? The equation a·x ≡ 1 (mod m) means a·x + m·y = 1 for some integer y. By Bézout’s identity, this has a solution iff gcd(a, m) divides 1, i.e., gcd(a, m) = 1.

Why This Topic Matters

• Division mod m: To compute (a / b) mod m you need b⁻¹ mod m, then (a · b⁻¹) mod m. Essential whenever the problem asks for “answer mod 10⁹+7” and your formula involves division (e.g., n! / (k! (n−k)!) for combinations).
• Combinatorics: nCr mod p, Catalan numbers, partition counts—many use factorials and require dividing by factorials mod p. Precompute factorials and inverse factorials mod p using the inverse.
• Cryptography: RSA decryption and many protocols use the modular inverse.

When the Inverse Exists

The inverse of a mod m exists iff gcd(a, m) = 1. So:

• When m is prime, every a with 1 ≤ a ≤ m−1 has an inverse (since gcd(a, m) = 1). 0 has no inverse.
• When m is composite, a has an inverse iff a and m are coprime. For example mod 10, 3 has inverse 7 (3·7=21≡1); 2 has no inverse because gcd(2, 10) = 2 ≠ 1.

Two Methods to Compute the Inverse

Method 1: Extended Euclidean Algorithm

Solve a·x + m·y = 1. The coefficient x is a modular inverse of a mod m. Run extended_gcd(a, m); if g ≠ 1, no inverse. Otherwise reduce x to [0, m−1]: inv = x % m or inv = (x % m + m) % m if x might be negative. This works for any m and any a with gcd(a, m) = 1. Time O(log min(a, m)).

Method 2: Fermat’s Little Theorem (When m is Prime)

If m is prime and a is not divisible by m, then a^m−1 ≡ 1 (mod m). So a · a^m−2 ≡ 1 (mod m), hence a⁻¹ ≡ a^m−2 (mod m). Compute a^m−2 mod m with fast exponentiation. Time O(log m). Only applies when m is prime.

Python Implementation

Using Extended GCD (Works for Any m)

def mod_inverse_gcd(a: int, m: int) -> int | None:
    """Returns a^(-1) mod m if gcd(a, m) = 1, else None."""
    g, x, _ = extended_gcd(a % m, m)
    if g != 1:
        return None
    return (x % m + m) % m

Assume extended_gcd is from topic 4.5. We take a % m so we work with a in [0, m−1]; then (x % m + m) % m puts the inverse in [0, m−1].

Using Fermat (When m is Prime)

def mod_inverse_fermat(a: int, m: int) -> int | None:
    """Returns a^(-1) mod m using a^(m-2). Only when m is prime."""
    a %= m
    if a == 0:
        return None
    return pow(a, m - 2, m)

Using Python’s Built-in (3.8+)

# When gcd(a, m) = 1:
pow(a, -1, m)   # returns a^(-1) mod m
# Raises ValueError if gcd(a, m) != 1

In practice use pow(a, -1, m) when you know a and m are coprime (e.g., m prime and a not divisible by m).

Line-by-Line Explanation (mod_inverse_gcd)

• a % m — Work with a in [0, m−1]; gcd and inverse are unchanged.
• extended_gcd(a % m, m) — Get (g, x, y) with (a mod m)·x + m·y = g.
• if g != 1: return None — Inverse exists only when gcd is 1.
• (x % m + m) % m — x might be negative; this gives the unique representative in [0, m−1].

Time and Space Complexity

Extended GCD method: O(log min(a, m)). Fermat (m prime): O(log m) for pow(a, m−2, m). Space O(1) for both iterative implementations.

Edge Cases

• a = 0: 0 has no inverse (0·x ≡ 0 ≢ 1 for any x). Return None or handle before calling.
• gcd(a, m) ≠ 1: No inverse. extended_gcd returns g > 1; return None. pow(a, -1, m) raises ValueError.
• m = 1: Every number is ≡ 0 (mod 1); “inverse” isn’t useful. Usually m ≥ 2 in practice.
• Negative a: Reduce a to [0, m−1] first with a % m; the inverse of (a mod m) is the same as the inverse of a mod m.

Common Mistakes

• Not checking gcd: If you assume the inverse exists and it doesn’t, you’ll get wrong results or a crash. Always check g == 1 (or catch ValueError for pow).
• Returning negative x: The inverse should be in [0, m−1]. Use (x % m + m) % m after extended GCD.
• Using Fermat when m is not prime: a⁻¹ ≡ a^m−2 only holds when m is prime. For composite m use extended GCD.

Application: (a / b) mod m

To compute (a / b) mod m when gcd(b, m) = 1: find inv = b⁻¹ mod m, then return (a % m * inv) % m. So “division” is multiply by the inverse.

def div_mod(a: int, b: int, m: int) -> int | None:
    inv = pow(b, -1, m)  # or mod_inverse_gcd(b, m)
    if inv is None:
        return None
    return (a % m * inv) % m

Interview Insight

Practice Problems

• Implement mod_inverse using extended GCD and (when m is prime) using Fermat; verify a · inv ≡ 1 (mod m).
• Compute nCr mod p (p prime): nCr = n! / (k! (n−k)!); precompute factorials and inverse factorials mod p, then combine.
• Given a, b, m, compute (a / b) mod m when possible; return a sentinel or raise when b has no inverse.

Summary

• The modular inverse of a mod m is x with a·x ≡ 1 (mod m). It exists iff gcd(a, m) = 1; when it exists it is unique in [0, m−1].
• Extended GCD: Solve a·x + m·y = 1; x mod m (adjusted for negative) is the inverse. Works for any m. Time O(log min(a, m)).
• Fermat (m prime): a⁻¹ ≡ a^m−2 (mod m). Compute with pow(a, m−2, m). Only when m is prime.
• Use pow(a, -1, m) in Python 3.8+ when gcd(a, m) = 1. (a / b) mod m = (a · b⁻¹) mod m.

4.9 Combinatorics

Introduction

Combinatorics is the study of counting: the number of ways to arrange, select, or partition objects under given rules. In DSA and competitive programming you constantly need “how many ways …?” (paths, subsets, arrangements, valid configurations). This section covers the foundational counting principles—the sum rule and product rule—and the role of factorials in counting. The next topic (4.10) gives the exact formulas for permutations and combinations; here we build the mindset and the modular tools (factorials and inverse factorials mod m) needed to compute those efficiently.

Real-World Analogy

Imagine choosing an outfit: 3 shirts and 4 pants. Any shirt can pair with any pant, so total outfits = 3 × 4 = 12. That’s the product rule: when one choice doesn’t affect the other, multiply the number of options. Now imagine you can either wear a hat (2 choices) or no hat (1 choice), but not both—total hat options = 2 + 1 = 3. That’s the sum rule: when choices are mutually exclusive, add. Most counting problems combine these two rules.

Formal Definition

Many problems require breaking the count into cases (sum rule) and then counting each case by a sequence of choices (product rule).

Why This Topic Matters

• “Number of ways” problems: Count paths, subsets, valid sequences, placements—all use combinatorial reasoning.
• Permutations and combinations: The next topic gives P(n,r) and C(n,r); both rely on factorials and on the product/sum rules for derivation.
• Modular counting: Problems often ask for the answer “mod 10⁹+7.” You need factorials and inverse factorials mod m (from the previous topics) to compute nCr and nPr without overflow.

Mental Model

Ask: “Is this a sequence of independent choices?” → product rule (multiply). “Is this one of several disjoint cases?” → sum rule (add). Often you combine: “For each case i, count the ways (product rule); then sum over cases.” Also: “Order matters” vs “order doesn’t matter” will distinguish permutations from combinations (topic 4.10).

Factorials and Growth

The factorial of a non-negative integer n is n! = n × (n−1) × … × 1, with 0! = 1. It counts the number of ways to arrange n distinct objects in a line (permutations of n objects). Factorials grow very fast: 10! ≈ 3.6×10⁶, 20! ≈ 2.4×10¹⁸. So we almost always work modulo m (e.g., 10⁹+7) when n is large.

def factorial(n: int, m: int | None = None) -> int:
    """n! or n! mod m if m given."""
    if n < 0:
        return 0  # or raise
    res = 1
    for i in range(2, n + 1):
        res *= i
        if m is not None:
            res %= m
    return res

For repeated use (e.g., many nCr queries), precompute factorials 0! .. N! mod m in an array—O(N) time once, then O(1) per lookup.

Precomputing Factorials and Inverse Factorials Mod m

To compute nCr = n! / (k! (n−k)!) mod m (m prime), we need n!, k!, (n−k)! mod m and then divide by (k! (n−k)!) using the modular inverse. Precompute:

• fact[i] = i! mod m for i = 0..N
• inv_fact[i] = (i!)⁻¹ mod m (inverse factorial)

Then nCr mod m = fact[n] × inv_fact[k] × inv_fact[n−k] mod m. Building inv_fact: inv_fact[N] = pow(fact[N], -1, m), then inv_fact[i] = inv_fact[i+1] × (i+1) mod m (so inv_fact[i] = 1/(i!) mod m).

def precompute_factorials(n: int, m: int) -> tuple[list[int], list[int]]:
    fact = [1] * (n + 1)
    for i in range(1, n + 1):
        fact[i] = (fact[i - 1] * i) % m
    inv_fact = [1] * (n + 1)
    inv_fact[n] = pow(fact[n], -1, m)
    for i in range(n - 1, -1, -1):
        inv_fact[i] = (inv_fact[i + 1] * (i + 1)) % m
    return fact, inv_fact

Then C(n, k) mod m = fact[n] * inv_fact[k] % m * inv_fact[n - k] % m (for 0 ≤ k ≤ n). This is the standard setup for combinatorics mod m.

Sum Rule and Product Rule: Examples

Product rule

Number of k-digit strings over an alphabet of size d (repetition allowed): each of k positions has d choices → d^k. Number of ways to order n distinct items: n choices for first, n−1 for second, … → n!.

Sum rule

Number of ways to get a sum of 7 with two dice: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1) → 6 ways. Each outcome is mutually exclusive, so we add.

Combined

Count 5-letter words that start with a vowel (a,e,i,o,u) or end with a consonant. Cases: (1) start vowel, end consonant; (2) start vowel, end vowel; (3) start consonant, end consonant. Each case uses the product rule; the three cases are disjoint, so add the three counts.

Time and Space Complexity

Single factorial mod m: O(n). Precompute factorials 0..N: O(N) time, O(N) space. Each nCr query after precomputation: O(1). So for many nCr queries (e.g., up to N), precomputation is essential.

Edge Cases

• n < 0 or k < 0: Factorial and C(n,k) are usually defined for n, k ≥ 0. Return 0 or handle as invalid.
• k > n: C(n, k) = 0 (no way to choose k items from n). So check 0 ≤ k ≤ n before computing.
• m = 1: Everything mod 1 is 0; factorials and nCr mod 1 are 0. Usually m is a prime > 1 (e.g., 10⁹+7).

Common Mistakes

• Confusing sum and product: If choices are independent (one doesn’t restrict the other), multiply. If they’re mutually exclusive (either A or B), add.
• Computing n! without mod for large n: n! overflows or becomes huge. Always work mod m when the problem says “mod 10⁹+7.”
• Computing nCr as fact[n] // (fact[k] * fact[n-k]): Integer division is wrong in modular arithmetic. Use fact[n] * inv_fact[k] * inv_fact[n-k] mod m.

Pattern Recognition

“In how many ways …?” usually means: identify disjoint cases (sum rule) and/or a sequence of choices (product rule). When the count involves “choose k from n” or “arrange n items,” use permutations and combinations (next topic); the formulas are built from factorials and the rules above.

Interview Insight

Practice Problems

• Precompute fact and inv_fact for n up to 10⁶ mod 10⁹+7; implement nCr(n, k, fact, inv_fact, m) and test.
• Count the number of k-digit numbers with no leading zero: first digit 1..9, remaining k−1 digits 0..9 → 9 × 10^k−1 (product rule).
• Solve a problem that asks for “number of ways” mod 10⁹+7 using sum/product rules and (when needed) nCr from precomputed factorials.

Summary

• Sum rule: Mutually exclusive options → add the counts. Product rule: Sequence of independent choices → multiply the counts.
• Factorial: n! = n×(n−1)×…×1, 0! = 1; grows fast—work mod m for large n. Precompute fact[0..N] and inv_fact[0..N] mod m for O(1) nCr/nPr.
• nCr mod m = fact[n] × inv_fact[k] × inv_fact[n−k] mod m (for 0 ≤ k ≤ n). Never use integer division; use modular inverses.
• Combinatorics problems: identify cases (sum) and choices (product); then plug in permutations/combinations (next topic) when order or selection is involved.

4.10 Permutations & Combinations

Introduction

A permutation is an arrangement of objects in a specific order; a combination is a selection of objects where order does not matter. “How many ways to arrange 3 books from 5?” is P(5, 3) = 60. “How many ways to choose 3 books from 5?” is C(5, 3) = 10. Permutations and combinations are the core counting tools in DSA: subsets, teams, passwords, placements. This section gives the definitions, formulas (with and without repetition), efficient computation mod m using precomputed factorials (from 4.9), and common identities.

Real-World Analogy

Permutation: Picking 3 people for president, vice-president, secretary—order matters (Alice–Bob–Carol is different from Bob–Alice–Carol). So we count arrangements. Combination: Picking 3 people for a committee—order doesn’t matter (the same three people are one committee). So we count subsets. Same “choose 3 from n,” but permutation counts orderings (more), combination counts subsets (fewer); and P(n, 3) = C(n, 3) × 3!.

Formal Definition

By convention, P(n, 0) = C(n, 0) = 1 (one way to arrange or choose nothing). For r > n, P(n, r) = C(n, r) = 0.

Why This Topic Matters

• Subset and selection problems: “How many subsets of size k?” → C(n, k). “How many ways to assign k distinct roles from n people?” → P(n, k).
• Counting paths, sequences, and configurations: Many problems decompose into “choose positions” (combination) and “assign values” (permutation or product rule).
• Competitive programming: nCr and nPr mod 10⁹+7 appear constantly; you need the formulas and fast implementation with precomputed factorials.

Relation Between P and C

To arrange r objects chosen from n: first choose the r objects (C(n, r) ways), then order them (r! ways). So P(n, r) = C(n, r) × r!. Hence C(n, r) = P(n, r) / r! = n! / (r! (n−r)!).

Key Identities

• Symmetry: C(n, k) = C(n, n−k). Choosing k is the same as “leaving out” n−k.
• Pascal’s identity: C(n, k) = C(n−1, k−1) + C(n−1, k). Either the first element is in the subset (C(n−1, k−1)) or it isn’t (C(n−1, k)). This gives a recurrence to build Pascal’s triangle.
• Sum of row: C(n, 0) + C(n, 1) + … + C(n, n) = 2ⁿ (total subsets of an n-element set).

With Repetition (Brief)

Permutations with repetition: Arrange r objects from n types, each type available unlimited times. Each of r positions has n choices → n^r.

Combinations with repetition (multiset choice): Choose r objects from n types (unlimited of each). Formula: C(n+r−1, r) = C(n+r−1, n−1) (“stars and bars”). Example: number of ways to distribute r identical candies to n children = C(n+r−1, r).

Python Implementation

Assume we have precomputed fact and inv_fact (from topic 4.9) for indices 0..N mod m.

nCr (combinations)

def nCr(n: int, k: int, fact: list[int], inv_fact: list[int], m: int) -> int:
    if k < 0 or k > n:
        return 0
    return fact[n] * inv_fact[k] % m * inv_fact[n - k] % m

nPr (permutations)

def nPr(n: int, r: int, fact: list[int], inv_fact: list[int], m: int) -> int:
    if r < 0 or r > n:
        return 0
    return fact[n] * inv_fact[n - r] % m

nPr(n, r) = n! / (n−r)! = fact[n] × inv_fact[n−r]. No need for inv_fact[r] unless you derive from nCr × r!.

Line-by-Line Explanation (nCr)

• if k < 0 or k > n: return 0 — No way to choose k from n when k > n or k < 0. C(n, 0) = 1 is handled by the formula (inv_fact[0] = 1).
• fact[n] * inv_fact[k] % m * inv_fact[n - k] % m — n! / (k! (n−k)!) mod m. Multiply by inverse factorials instead of dividing. Reduce mod m after each multiplication to keep intermediates bounded.

Time and Space Complexity

With precomputed fact and inv_fact: O(1) per nCr or nPr query. Precomputation is O(N) time and O(N) space (topic 4.9). Without precomputation, computing one nCr with a loop (multiply and divide) is O(min(k, n−k)); for many queries, precomputation is better.

Edge Cases

• k > n or k < 0: C(n, k) = 0. Return 0.
• k = 0 or k = n: C(n, 0) = C(n, n) = 1. The formula gives fact[n] * inv_fact[0] * inv_fact[n] = 1 when inv_fact[0] = 1 and inv_fact[n] = 1/fact[n].
• r > n for nPr: P(n, r) = 0. Return 0.

Common Mistakes

• Using P when order doesn’t matter: “Choose a committee of 3” → C(n, 3), not P(n, 3). P counts orderings.
• Using C when order matters: “Rank top 3” or “assign 3 distinct roles” → P(n, 3).
• Off-by-one in formulas: P(n, r) has r factors: n, n−1, …, n−r+1. So the last factor is (n−r+1), not (n−r).
• Integer division for nCr: Use inverse factorials (or modular inverse) when working mod m; never integer division.

Pattern Recognition

Ask: “Does order matter?” Yes → permutation (arrange, rank, assign distinct roles). No → combination (choose, committee, subset). Then check: repetition allowed? If yes, use n^r (permutation) or stars-and-bars (combination with repetition). If no, use P(n, r) or C(n, r).

Interview Insight

Practice Problems

• Implement nCr and nPr with precomputed fact/inv_fact; verify against small values (e.g., C(5,2)=10, P(5,2)=20).
• Count subsets of size k: C(n, k). Count permutations of n objects: n!. Count ways to place k distinct items in n distinct positions (each at most one): P(n, k).
• LeetCode-style: “Unique paths” (grid with C(n+m-2, n-1)), “combinations” (enumerate C(n,k)), or “number of ways” mod 10⁹+7 using nCr.

Summary

• Permutation P(n, r) = n!/(n−r)!: arrange r distinct objects from n; order matters. Combination C(n, r) = n!/(r!(n−r)!): choose r from n; order doesn’t matter. P(n, r) = C(n, r) × r!.
• Key identities: C(n, k) = C(n, n−k); C(n, k) = C(n−1, k−1) + C(n−1, k); sum of C(n,0)..C(n,n) = 2ⁿ.
• With repetition: permutations n^r; combinations (stars and bars) C(n+r−1, r).
• Implement nCr/nPr mod m with precomputed fact and inv_fact; O(1) per query. Edge cases: k > n or k < 0 → 0.

4.11 Probability Basics

Introduction

Probability in DSA appears in randomized algorithms, expected-value analysis, and problems that ask “what is the probability that …?” or “expected number of steps?”. You don’t need measure theory—only discrete probability: sample space, events, and the rule P(event) = number of favorable outcomes / number of total outcomes when outcomes are equally likely. This section covers basic definitions, the addition and multiplication rules, complement, independence, and expected value (including linearity). Enough to reason about probability in interviews and to tie counting (combinatorics) to probability.

Real-World Analogy

Roll a fair die. The sample space is {1, 2, 3, 4, 5, 6}. The probability of rolling a 4 is 1/6 (one favorable outcome out of six). The probability of rolling an even number is 3/6 = 1/2 (outcomes 2, 4, 6). “Probability” here is just “favorable count / total count” when every outcome is equally likely. Same idea in algorithms: “probability a random permutation has property P” = (number of permutations with P) / (total permutations).

Formal Definition (Discrete, Equally Likely)

We have 0 ≤ P(E) ≤ 1. P(Ω) = 1 (something must happen). P(empty) = 0.

Why This Topic Matters

• Expected value: “Expected number of comparisons in quicksort,” “expected steps until random walk hits a state”—linearity of expectation is used everywhere.
• Randomized algorithms: “With probability at least 1/2 the algorithm succeeds”—you need to bound or compute such probabilities.
• Interview problems: “Probability that two people share a birthday,” “expected value of …”—formulate as counting (favorable / total) or as expectation.

Key Rules

Complement

P(not E) = 1 − P(E). So “probability at least one” = 1 − “probability none.”

Addition (Disjoint Events)

If events A and B cannot happen together (disjoint), then P(A or B) = P(A) + P(B). For more than two disjoint events, add all.

Multiplication (Independent Events)

If A and B are independent (one occurring doesn’t change the chance of the other), then P(A and B) = P(A) × P(B). Example: two fair coin flips; P(both heads) = (1/2)×(1/2) = 1/4.

General Addition

For any two events: P(A or B) = P(A) + P(B) − P(A and B). When A and B are disjoint, P(A and B) = 0.

Expected Value

For a discrete random variable X that takes values x₁, x₂, … with probabilities p₁, p₂, …, the expected value is E[X] = Σ x_i · P(X = x_i). Intuition: long-run average if you repeat the experiment many times.

Linearity of expectation: E[X + Y] = E[X] + E[Y] for any X, Y (even if they are not independent). So E[X₁ + X₂ + … + Xₙ] = E[X₁] + … + E[Xₙ]. This is used constantly: break the quantity into indicator or simple random variables, compute each expectation, and add.

Probability as Counting

When outcomes are equally likely, P(E) = (number of outcomes in E) / (total outcomes). So:

• Probability that a random subset of size k from {1..n} contains 1 = C(n−1, k−1) / C(n, k) = k/n (or: 1 is in the subset with probability k/n by symmetry).
• Probability that a random permutation of n is a derangement (no fixed point) = (number of derangements) / n!. Useful in some puzzles.

So combinatorics (nCr, nPr, counting) directly gives probabilities when the sample space is “all subsets” or “all permutations” with uniform distribution.

Python: Simple Probability and Expectation

For small spaces you can enumerate and count. For expectation you can sum value × probability, or simulate (Monte Carlo) for verification.

# Example: P(sum of two fair dice = 7)
# Favorable: (1,6),(2,5),(3,4),(4,3),(5,2),(6,1) → 6 outcomes. Total 36.
p_7 = 6 / 36  # 1/6

# Expected value of one die roll (1 to 6)
E_one_die = sum(x for x in range(1, 7)) / 6  # 3.5

# E[X+Y] = E[X] + E[Y]: two dice → 3.5 + 3.5 = 7

Common Mistakes

• Assuming independence when events aren’t: P(A and B) = P(A)×P(B) only when A and B are independent. If they’re not, use P(A and B) = P(A) P(B|A) (conditional) or count outcomes.
• Adding non-disjoint events without subtracting overlap: P(A or B) = P(A) + P(B) − P(A and B). Forgetting the subtraction double-counts.
• Confusing E[X·Y] with E[X]·E[Y]: Linearity gives E[X+Y] = E[X] + E[Y]. In general E[X·Y] ≠ E[X]·E[Y] unless X and Y are uncorrelated/independent.

Interview Insight

Practice Problems

• Probability that two people in a room of n share a birthday (use complement: 1 − P(all different) = 1 − (365/365)×(364/365)×…×((365−n+1)/365)).
• Expected number of trials until first success (geometric): if P(success) = p, E[trials] = 1/p.
• Given n items and k chosen at random, expected number of “special” items in the chosen set (linearity with indicators).

Summary

• When outcomes are equally likely, P(E) = |E| / |Ω|—probability is counting. Use combinatorics (nCr, nPr) for numerator and denominator.
• Complement: P(not E) = 1 − P(E). Disjoint: P(A or B) = P(A) + P(B). Independent: P(A and B) = P(A)×P(B). General or: P(A or B) = P(A) + P(B) − P(A and B).
• Expected value: E[X] = Σ x·P(X=x). Linearity: E[X+Y] = E[X] + E[Y]; use to break into indicators or simple terms.
• In DSA: randomized algorithms (probability of success), expected running time, and “probability that …” problems (count favorable / total).

4.12 Matrix Basics

Introduction

A matrix is a rectangular grid of numbers (or elements) arranged in rows and columns. An m × n matrix has m rows and n columns. In DSA, matrices appear as 2D arrays (graph adjacency, grid problems), in linear recurrences (matrix exponentiation for Fibonacci—topic 4.17), and in dynamic programming. This section covers representation (row/column indexing), basic operations—addition, scalar multiplication, matrix multiplication, and transpose—and clean Python implementations. Mastery of matrix multiplication (dimensions, loop order) is essential for matrix exponentiation and many advanced topics.

Real-World Analogy

Think of a matrix as a spreadsheet or a data table: rows are entities (e.g., users), columns are attributes (e.g., age, score). The entry in row i and column j is the value for that pair. In graphics or physics, a matrix can represent a transformation (rotation, scaling); multiplying a vector by a matrix gives a new vector. In algorithms, we often multiply matrices to combine transitions (e.g., “one step” in a recurrence becomes a matrix; “n steps” is the matrix raised to power n).

Formal Definition

Two matrices of the same dimensions can be added entry-wise. Matrix multiplication is defined when the number of columns of the first equals the number of rows of the second: (m×n)(n×p) → m×p.

Why This Topic Matters

• 2D arrays and grids: Matrices are the natural structure for grids (maze, board, image). Traversal, DP on grids, and adjacency matrices all use the same indexing.
• Matrix multiplication: Recurrences like Fibonacci can be written as a vector updated by a fixed matrix; then F_n is obtained by “matrix to power n” (topic 4.17). So matrix multiply is the building block.
• Graphs: Adjacency matrix of a graph is an n×n matrix; A[i][j] = 1 if there’s an edge from i to j. Powers of the adjacency matrix count walks of a given length.

Representation in Python

A matrix is typically a list of lists: each inner list is a row. So A[i][j] is row i, column j. Ensure every row has the same length (same number of columns).

# 2×3 matrix
A = [
    [1, 2, 3],
    [4, 5, 6]
]
# A[0][1] = 2, A[1][2] = 6
rows, cols = len(A), len(A[0])

Basic Operations

Addition

For two matrices A and B of the same size (m×n), (A + B)[i][j] = A[i][j] + B[i][j]. Time O(m·n), space O(m·n) for the result.

Scalar Multiplication

(c·A)[i][j] = c × A[i][j]. Time O(m·n).

Transpose

A^T has dimensions n×m with A^T[j][i] = A[i][j]. So the j-th row of A^T is the j-th column of A. Time O(m·n).

Matrix Multiplication

Let A be m×n and B be n×p. The product C = A×B is m×p with:

C[i][j] = Σ_k=0ⁿ⁻¹ A[i][k] · B[k][j]

So the (i,j) entry of the product is the dot product of row i of A and column j of B. The inner dimension (n) must match; the result has dimensions m×p.

ASCII Diagram: Matrix Multiplication

  A (m×n)     ×     B (n×p)     =     C (m×p)
  row i  ──────────────►  dot with col j  ──►  C[i][j]
        [ ... A[i][k] ... ]   [ B[k][j] ]   =  sum_k A[i][k]*B[k][j]
                             [   ...   ]
  Inner dimension n must match; result has outer dimensions m and p.
        

Identity Matrix

The identity matrix I_n is the n×n matrix with I[i][j] = 1 if i = j and 0 otherwise. For any n×n matrix A, A·I = I·A = A. In exponentiation we use I as the base case (A⁰ = I).

Python Implementation

Matrix Addition

def mat_add(A: list[list[float]], B: list[list[float]]) -> list[list[float]]:
    m, n = len(A), len(A[0])
    if len(B) != m or len(B[0]) != n:
        raise ValueError("dimension mismatch")
    return [[A[i][j] + B[i][j] for j in range(n)] for i in range(m)]

Matrix Multiplication

def mat_mul(A: list[list[float]], B: list[list[float]]) -> list[list[float]]:
    m, n, p = len(A), len(A[0]), len(B[0])
    if len(B) != n:
        raise ValueError("dimension mismatch: A is m×n, B must be n×p")
    C = [[0] * p for _ in range(m)]
    for i in range(m):
        for j in range(p):
            for k in range(n):
                C[i][j] += A[i][k] * B[k][j]
    return C

Transpose

def mat_transpose(A: list[list[float]]) -> list[list[float]]:
    if not A:
        return []
    m, n = len(A), len(A[0])
    return [[A[i][j] for i in range(m)] for j in range(n)]

Identity Matrix

def identity(n: int) -> list[list[float]]:
    I = [[0] * n for _ in range(n)]
    for i in range(n):
        I[i][i] = 1
    return I

Line-by-Line Explanation (mat_mul)

• m, n, p = len(A), len(A[0]), len(B[0]) — A is m×n, B must be n×p; result C is m×p.
• if len(B) != n — B must have n rows (same as A’s columns) for the product to be defined.
• C = [[0] * p for _ in range(m)] — Initialize m×p result to zeros.
• for i in range(m): for j in range(p): for k in range(n): — For each (i,j), C[i][j] = sum over k of A[i][k]*B[k][j]. Loop order: i, j, k is standard (good cache behavior when row-major).

Time and Space Complexity

Addition: O(m·n) time, O(m·n) space for result. Transpose: O(m·n) time and space. Matrix multiplication (naive): Three nested loops over m, p, n → O(m·n·p) time. For two n×n matrices, O(n³). Space for result O(m·p). (Strassen and other methods reduce the exponent for large matrices but are rarely needed in interviews.)

Modular Matrix Multiplication

When entries are integers and the problem asks for “result mod M,” reduce modulo M after each multiplication and addition to avoid overflow and keep numbers small. Same loop structure; add % M when accumulating C[i][j].

def mat_mul_mod(A: list[list[int]], B: list[list[int]], M: int) -> list[list[int]]:
    m, n, p = len(A), len(A[0]), len(B[0])
    if len(B) != n:
        raise ValueError("dimension mismatch")
    C = [[0] * p for _ in range(m)]
    for i in range(m):
        for j in range(p):
            for k in range(n):
                C[i][j] = (C[i][j] + A[i][k] * B[k][j]) % M
    return C

Edge Cases

• Empty matrix: A = [] or A = [[]]. Rows = 0; number of columns undefined. Check if not A or not A[0] before using len(A[0]).
• Dimension mismatch: For A×B, A’s columns must equal B’s rows. Validate before looping.
• Single row or column: A 1×n matrix times an n×1 matrix gives a 1×1 matrix (a single number). Handled correctly by the same loops.

Common Mistakes

• Wrong loop order: C[i][j] must sum over k. So the inner loop must be over k (A’s columns / B’s rows). Writing loops in wrong order (e.g., j, i, k without i fixed) gives wrong indices.
• Using A[k][j] instead of B[k][j]: The second matrix is B; row k of A times column j of B uses A[i][k] and B[k][j]. Don’t mix A and B.
• Dimension confusion: (m×n)(n×p) = m×p. The shared dimension n is the one we sum over.

Pattern Recognition

Many problems that look like “apply a linear recurrence n times” can be expressed as “vector × matrix^n”. The recurrence defines the matrix; matrix multiplication combines two steps into one. So implementing mat_mul (and later mat_pow using fast exponentiation) is a recurring pattern.

Interview Insight

Practice Problems

• Implement mat_add, mat_mul, transpose, identity and test on 2×2 examples (hand-compute expected result).
• Implement mat_mul_mod for integer matrices mod M; use it as the “multiply” in matrix exponentiation (next topic 4.17).
• Count walks of length k from vertex i to j in a graph using the adjacency matrix: the (i,j) entry of A^k is the number of walks of length k.

Summary

• A matrix is an m×n grid; A[i][j] is row i, column j. Represent in Python as list of lists (each list is a row).
• Addition: entry-wise, same dimensions. Transpose: (A^T)_j,i = A_i,j. Multiplication: (m×n)(n×p) = m×p; (AB)[i][j] = Σ_k A[i][k]·B[k][j].
• Naive matrix multiply: O(m·n·p) time. For mod M, reduce after each operation. Identity matrix I satisfies A·I = I·A = A.
• Loop order: for i, for j, for k with C[i][j] += A[i][k]*B[k][j]. Use row of A and column of B—don’t mix indices.

4.13 Euler's Totient Function

Introduction

Euler’s totient function φ(n) counts the number of integers in {1, 2, …, n} that are coprime to n (i.e., gcd(k, n) = 1). So φ(n) is also the number of elements in the set of integers modulo n that have a multiplicative inverse—the size of the “unit group” mod n. It appears in Euler’s theorem (a^φ(n) ≡ 1 (mod n) when gcd(a, n) = 1), in RSA cryptography, and in counting problems (e.g., “how many fractions a/b in lowest terms with 0 < a < b ≤ n?”). This section defines φ(n), gives formulas (including from prime factorization and via a sieve), and shows how to compute it in code.

Real-World Analogy

Imagine n chairs in a circle. You want to assign each chair a number from 1 to n so that every chair gets a number that “shares no common factor with n” in a certain sense—think of it as “valid” positions. The number of valid assignments is φ(n). Equivalently: among 1, 2, …, n, how many share no prime factor with n? That count is φ(n). For n = 10, the numbers coprime to 10 are 1, 3, 7, 9 → φ(10) = 4.

Formal Definition

If n has prime factorization n = p₁^a₁ p₂^a₂ … p_k^aₖ, then φ(n) = n × Π (1 − 1/p_i) = n × (1 − 1/p₁)(1 − 1/p₂)…(1 − 1/p_k). Equivalently, φ(n) = Π (p_i^aᵢ − p_i^aᵢ−1) = Π p_i^aᵢ−1(p_i − 1).

Why This Topic Matters

• Euler’s theorem: If gcd(a, n) = 1, then a^φ(n) ≡ 1 (mod n). So a⁻¹ ≡ a^φ(n)−1 (mod n)—another way to compute the modular inverse when n is not prime (Fermat applies only when n is prime).
• RSA: The public and private exponents are chosen using φ(N) where N = p·q. Security relies on the hardness of computing φ(N) without knowing the factors.
• Counting: Problems like “count pairs (a, b) with 1 ≤ a < b ≤ n and gcd(a, b) = 1” or “number of reduced fractions” use sums involving φ.

Mental Model

φ(n) is “how many numbers in 1..n don’t share any prime factor with n.” So start with n and for each distinct prime p dividing n, “remove” a fraction 1/p of the numbers (those divisible by p). What remains is n × (1 − 1/p₁)(1 − 1/p₂)… = φ(n). For a prime p, every number 1..p−1 is coprime to p, so φ(p) = p − 1.

Key Formulas

• Prime: φ(p) = p − 1.
• Prime power: φ(p^k) = p^k − p^k−1 = p^k−1(p − 1).
• Multiplicative: If gcd(m, n) = 1, then φ(m·n) = φ(m)·φ(n). So from prime powers, φ(n) = Π φ(p_i^aᵢ) = Π (p_i^aᵢ − p_i^aᵢ−1).
• Product form: φ(n) = n × Π_p|n (1 − 1/p).

Step-by-Step: Computing φ(n) from Prime Factorization

1. Factor n into primes: n = p₁^a₁ … p_k^aₖ.
2. For each distinct prime p dividing n, multiply the running result by (1 − 1/p), or equivalently compute φ(n) = n × Π (1 − 1/p). Alternatively, φ(n) = Π (p_i^aᵢ − p_i^aᵢ−1).
3. If you only need φ(n) for one n, factor n (trial division up to √n) then apply the formula. If you need φ(1) through φ(N), use a sieve (see below).

Computing φ(1..N) with a Sieve

Initialize phi[i] = i for all i. For each prime p from 2 to N: for each multiple k = p, 2p, 3p, … ≤ N, do phi[k] -= phi[k] / p (or phi[k] *= (1 - 1/p) in integer form: phi[k] = phi[k] * (p - 1) / p). After processing all primes, phi[i] = φ(i). This runs in O(N log log N) like the sieve of Eratosthenes.

Euler’s Theorem

If gcd(a, n) = 1, then a^φ(n) ≡ 1 (mod n). So the order of a modulo n divides φ(n). Corollary: a⁻¹ ≡ a^φ(n)−1 (mod n)—useful to compute the modular inverse when n is composite (e.g., n = 10⁹+7 is prime, so Fermat is simpler; but for composite n, use φ(n) or extended GCD).

Python Implementation

φ(n) from Factorization (Single Value)

def totient(n: int) -> int:
    """Returns φ(n) for n >= 1."""
    if n <= 0:
        return 0
    if n == 1:
        return 1
    res = n
    d = 2
    while d * d <= n:
        if n % d == 0:
            while n % d == 0:
                n //= d
            res -= res // d
        d += 1
    if n > 1:
        res -= res // n
    return res

Idea: start with res = n. For each distinct prime p dividing n, multiply res by (1 − 1/p), implemented as res -= res // p (so res becomes res * (p-1) / p). We iterate d and divide n by d so we only process distinct primes.

φ(1..N) via Sieve

def totient_sieve(n: int) -> list[int]:
    """Returns list [φ(0), φ(1), ..., φ(n)]. φ(0)=0."""
    phi = list(range(n + 1))
    for i in range(2, n + 1):
        if phi[i] == i:
            for j in range(i, n + 1, i):
                phi[j] -= phi[j] // i
    return phi

If phi[i] == i, then i is prime (not yet reduced). For each prime i, update all its multiples j: phi[j] *= (1 - 1/i) via phi[j] -= phi[j] // i.

Line-by-Line Explanation (totient)

• if n <= 0: return 0 — φ is defined for positive n only.
• if n == 1: return 1 — φ(1) = 1 by convention.
• res = n — Start with n; we’ll multiply by (1 − 1/p) for each prime p.
• while d * d <= n — Trial division up to √n. When we exit, n is 1 or a single prime.
• if n % d == 0 — d is a prime factor. while n % d == 0: n //= d removes all factors of d so we process d only once.
• res -= res // d — Same as res = res * (1 - 1/d) = res * (d-1) / d in integers.
• if n > 1: res -= res // n — Remaining n is a prime factor; apply the same.

Time and Space Complexity

Single φ(n) from factorization: O(√n) for trial division. Totient sieve: O(N log log N) time, O(N) space—same as the sieve of Eratosthenes. Use the sieve when you need φ(1)..φ(N); use the factorization method for a single large n.

Edge Cases

• n ≤ 0: φ is defined for positive integers; return 0 or handle as invalid.
• n = 1: φ(1) = 1 (1 is coprime to 1).
• Prime n: φ(n) = n − 1.

Common Mistakes

• Using φ(n) for modular inverse when n is prime: For prime n, Fermat (aⁿ⁻²) is simpler than a^φ(n)−1 (they’re equal when n is prime since φ(n) = n−1). Use φ when n is composite.
• Forgetting distinct primes: In the product φ(n) = n Π (1 − 1/p), each distinct prime p appears once. When factoring, don’t apply (1 − 1/p) multiple times for the same p.
• Sieve: updating phi[j] for composite j: In the sieve we iterate primes i and update multiples j. We do phi[j] -= phi[j] // i; doing this for each prime factor of j gives the correct φ(j).

Optimization Insight

Interview Insight

Practice Problems

• Implement totient(n) and totient_sieve(N); verify φ(12)=4, φ(7)=6.
• Compute the modular inverse of a mod n (composite n) using a^φ(n)−1 mod n when gcd(a, n) = 1.
• Count the number of integers in [1, n] coprime to n (that’s φ(n)); or count pairs (a, b) with 1 ≤ a < b ≤ n and gcd(a, b) = 1 using φ.

Summary

• φ(n) = number of integers in {1, …, n} with gcd(k, n) = 1. φ(1) = 1; for prime p, φ(p) = p−1.
• Formula: φ(n) = n × Π_p|n (1 − 1/p) = Π (p^a − p^a−1) over prime powers in n.
• Euler’s theorem: If gcd(a, n) = 1 then a^φ(n) ≡ 1 (mod n). So a⁻¹ ≡ a^φ(n)−1 (mod n) for composite n.
• Single φ(n): factor n, then apply formula—O(√n). Range φ(1..N): sieve in O(N log log N).

4.14 Chinese Remainder Theorem

Introduction

The Chinese Remainder Theorem (CRT) says that when we have several congruences with pairwise coprime moduli, there is a unique solution modulo the product of the moduli. Specifically: given x ≡ a₁ (mod m₁), x ≡ a₂ (mod m₂), …, x ≡ a_k (mod m_k) with gcd(m_i, m_j) = 1 for i ≠ j, there exists a unique x (mod M) where M = m₁·m₂·…·m_k. CRT lets us combine results computed modulo different primes (e.g., nCr mod several primes) into one result modulo their product, or split a problem into smaller moduli. This section states the theorem, gives the constructive formula, and implements it in Python.

Real-World Analogy

You have a number of items. When you divide by 3 the remainder is 2; when you divide by 5 the remainder is 3; when you divide by 7 the remainder is 2. Is there a number that fits all three? CRT says yes, and that all such numbers differ by a multiple of 3×5×7 = 105. So there is a unique remainder mod 105. Like solving a puzzle: each condition narrows the set; with coprime moduli, the constraints are “independent” and pin down one residue class mod the product.

Formal Definition

If the moduli are not pairwise coprime, a solution may not exist (e.g., x ≡ 0 (mod 2) and x ≡ 1 (mod 2) is impossible). When a solution exists (e.g., x ≡ 1 (mod 2) and x ≡ 1 (mod 4)), it is unique modulo lcm(m₁, …, m_k).

Why This Topic Matters

• Combining results: Compute something mod p₁, mod p₂, …, mod p_k (e.g., nCr mod each prime), then use CRT to get the result mod p₁·p₂·…·p_k or mod a large composite.
• Large modulus: Instead of working mod a huge M, work mod several smaller coprime factors and recombine with CRT.
• Contest problems: “Find x such that x ≡ a (mod m) and x ≡ b (mod n)”—direct CRT application.

Mental Model

Each congruence x ≡ a_i (mod m_i) says “x is in a certain residue class mod m_i.” Because the m_i are coprime, the conditions are independent: there is exactly one residue class mod M that matches all of them. So we “build” x by combining the contributions: for each i, we want a term that is a_i mod m_i and 0 mod m_j for j ≠ i; then add these terms.

Construction (Formula)

Let M = m₁·m₂·…·m_k and M_i = M / m_i. Then gcd(M_i, m_i) = 1 (since m_i is coprime to every other m_j). So M_i has an inverse mod m_i; call it y_i (so M_i·y_i ≡ 1 (mod m_i)). One solution is:

x = Σ_i a_i · M_i · y_i

Reduce x mod M to get the unique representative in [0, M−1]. Check: for each i, all terms a_j·M_j·y_j with j ≠ i are divisible by m_i (because M_j contains m_i), so x ≡ a_i·M_i·y_i ≡ a_i·1 ≡ a_i (mod m_i).

Two Moduli (Special Case)

Solve x ≡ a (mod m) and x ≡ b (mod n) with gcd(m, n) = 1. Write x = a + m·t for some integer t. Substitute into the second: a + m·t ≡ b (mod n) ⇒ m·t ≡ (b − a) (mod n). So t ≡ (b − a)·m⁻¹ (mod n). Compute t mod n, then x = a + m·t is a solution; reduce mod (m·n) for the unique solution in [0, mn−1].

Step-by-Step: General CRT

1. Check moduli are pairwise coprime (or handle non-coprime case separately).
2. Compute M = m₁·m₂·…·m_k and for each i, M_i = M / m_i.
3. For each i, compute y_i = modular inverse of M_i modulo m_i (e.g., pow(M_i, -1, m_i) in Python).
4. x = Σ a_i·M_i·y_i; then x = x % M (and if negative, (x % M + M) % M).

Python Implementation

Two Moduli

import math

def crt2(a: int, m: int, b: int, n: int) -> int | None:
    """Solves x ≡ a (mod m), x ≡ b (mod n). Returns x mod (m*n) or None if no solution."""
    if math.gcd(m, n) != 1:
        return None  # or solve for lcm when solution exists
    t = (b - a) * pow(m, -1, n) % n
    x = a + m * t
    return x % (m * n)

General CRT (List of (remainder, modulus))

def crt(remainders: list[int], moduli: list[int]) -> int | None:
    """Solves x ≡ remainders[i] (mod moduli[i]) for all i. Moduli must be pairwise coprime."""
    if len(remainders) != len(moduli):
        return None
    M = 1
    for m in moduli:
        M *= m
    x = 0
    for a, m in zip(remainders, moduli):
        Mi = M // m
        yi = pow(Mi, -1, m)
        x = (x + a * Mi * yi) % M
    return x

We reduce x mod M after each term (or once at the end) to avoid overflow. Final x is in [0, M−1].

Line-by-Line Explanation (General CRT)

• M = product of moduli — The solution is unique mod M.
• Mi = M // m — M_i = M/m_i; divisible by all m_j except m_i.
• yi = pow(Mi, -1, m) — Inverse of M_i mod m_i (exists because gcd(M_i, m_i) = 1).
• x = (x + a * Mi * yi) % M — Add a_i·M_i·y_i and keep x mod M so the sum doesn’t grow and we stay in [0, M−1].

Time and Space Complexity

For k congruences: k modular inverses (each O(log min(M_i, m_i))), k multiplications and additions. So O(k · log(max moduli)). Space O(1) if we don’t store all M_i, y_i (we can compute and add in one loop).

Edge Cases

• Moduli not pairwise coprime: A solution may not exist (e.g., x ≡ 0 (mod 2) and x ≡ 1 (mod 2)). If it exists, it is unique mod lcm(m₁, …, m_k). For a full implementation, check pairwise gcd and either return None or use the lcm and check consistency.
• Single congruence: Just return a mod m (or a if already in range).
• Empty list: Return 0 or define M = 1, x = 0.

Common Mistakes

• Assuming CRT applies when moduli are not coprime: The uniqueness and existence hold only when moduli are pairwise coprime. For 2 and 4, a solution exists only if a ≡ b (mod 2) (consistency); then solution is unique mod 4.
• Forgetting to reduce x mod M: The formula can produce a large x; the answer is x mod M. Always return x % M (and handle negative if needed).
• Wrong order of arguments: (remainder, modulus) pairs must match: x ≡ a (mod m). Don’t swap a and m when calling.

Optimization Insight

Interview Insight

Practice Problems

• Implement crt2 and crt; verify with the example x ≡ 2 (mod 3), x ≡ 3 (mod 5) → 8 (mod 15).
• Solve a system of three congruences with coprime moduli (e.g., mod 3, 5, 7) and check the result.
• Use CRT to combine nCr mod 2, mod 3, mod 5 into nCr mod 30 (or another small composite).

Summary

• CRT: For pairwise coprime m₁, …, m_k, the system x ≡ a_i (mod m_i) has a unique solution modulo M = m₁·…·m_k.
• Construction: x = Σ a_i·(M/m_i)·y_i where y_i = (M/m_i)⁻¹ mod m_i; then x mod M.
• Two moduli: x = a + m·t with t ≡ (b−a)·m⁻¹ (mod n); solution x mod (m·n).
• Moduli must be pairwise coprime for the standard statement. Time O(k · log(max modulus)); reduce x mod M to get the unique representative.

4.15 Lucas Theorem (Large nCr % Mod)

Introduction

Lucas’s theorem computes C(n, k) mod p when p is prime and n, k can be very large (e.g., 10¹⁸). The idea: write n and k in base p; then C(n, k) ≡ Π_i C(n_i, k_i) (mod p), where n_i and k_i are the base-p digits. So instead of precomputing factorials up to n (impossible when n is huge), we only need factorials up to p−1 and then one small product per digit. This is the standard way to compute nCr mod p for large n in competitive programming when p is prime (e.g., p = 10⁹+7).

Real-World Analogy

To compute “choose k from n” mod p, we could use n! / (k!(n−k)!) but n! is astronomically large. Lucas says: break n and k into “digits” in base p (like writing numbers in base 10). Each digit is at most p−1, so “choose k_i from n_i” for each digit is a small binomial coefficient we can precompute. The answer mod p is the product of these small binomials. So we reduce a huge problem to many tiny ones.

Formal Definition

The theorem follows from the fact that (1+x)^p ≡ 1+x^p (mod p) (by the binomial theorem and the fact that p divides C(p,j) for 0 < j < p). So the coefficient of x^k in (1+x)ⁿ mod p factors as the product over digits.

Why This Topic Matters

• Large n, prime p: Problems often ask for nCr mod 10⁹+7 with n up to 10¹⁸. You cannot compute n! or even store n. Lucas reduces to O(log_p n) binomials each with arguments < p.
• Competitive programming: Standard tool for “n choose k mod prime” when n is huge. Precompute factorials 0..p−1 once, then each query is O(log n).
• When p is not prime: Factor p into prime powers, compute nCr mod each prime power (using Lucas for primes; for prime powers there are extensions), then combine with CRT.

Mental Model

Think of n and k in base p. Each “digit position” contributes independently: we must choose k_i “items” from n_i available in that position. The theorem says the total number of ways (mod p) is the product of the ways at each position. So we only ever need C(a, b) for 0 ≤ a, b ≤ p−1—a small table.

Step-by-Step: Computing C(n, k) mod p (Lucas)

1. Precompute factorials and inverse factorials for 0..p−1 (so we can compute C(n_i, k_i) in O(1)).
2. Get base-p digits of n and k: n = n₀ + n₁·p + …, k = k₀ + k₁·p + … (e.g., repeatedly n % p, n //= p).
3. If len(k_digits) > len(n_digits), pad n with zeros (or treat missing digits of n as 0). For each digit index i, if k_i > n_i, return 0.
4. result = 1. For each i: result = (result * C(n_i, k_i)) % p. Return result.

Example

Python Implementation

Assume we have precomputed fact and inv_fact for indices 0..p−1 (size p).

def nCr_mod_small(n: int, k: int, fact: list[int], inv_fact: list[int], p: int) -> int:
    """C(n,k) mod p when 0 <= n, k < p."""
    if k < 0 or k > n:
        return 0
    return fact[n] * inv_fact[k] % p * inv_fact[n - k] % p

def digits_base_p(x: int, p: int) -> list[int]:
    """Digits of x in base p (LSB first)."""
    if x == 0:
        return [0]
    d = []
    while x:
        d.append(x % p)
        x //= p
    return d

def lucas(n: int, k: int, fact: list[int], inv_fact: list[int], p: int) -> int:
    """C(n, k) mod p using Lucas's theorem. p must be prime."""
    if k < 0 or k > n:
        return 0
    nd = digits_base_p(n, p)
    kd = digits_base_p(k, p)
    if len(kd) > len(nd):
        return 0
    res = 1
    for i in range(len(kd)):
        ni = nd[i] if i < len(nd) else 0
        ki = kd[i]
        if ki > ni:
            return 0
        res = res * nCr_mod_small(ni, ki, fact, inv_fact, p) % p
    return res

Line-by-Line Explanation

• digits_base_p(x, p) — Extract digits of x in base p (LSB first) by repeated x % p and x //= p.
• if len(kd) > len(nd) — If k has more base-p digits than n, then k > n, so C(n,k) = 0.
• for i in range(len(kd)) — We only need to consider digit positions where k has a digit. Where k has no digit (higher positions), k_i = 0 and C(n_i, 0) = 1, so we skip those.
• ni = nd[i] if i < len(nd) else 0 — i-th digit of n. Since we already ensured len(kd) ≤ len(nd), we have i < len(nd), so nd[i] exists. (If we didn’t return 0 earlier, n ≥ k implies n has at least as many digits as k.)
• if ki > ni: return 0 — If any digit k_i > n_i, C(n_i, k_i) = 0, so the whole product is 0.
• res = res * nCr_mod_small(ni, ki, ...) % p — Lucas: multiply the binomial coefficient for each digit position.

Time and Space Complexity

Precomputation: Factorials and inverse factorials for 0..p−1: O(p) time and space. Per query (Lucas): O(log_p n) digit extraction and O(log_p n) small binomial lookups (each O(1) with precomputed fact/inv_fact). So O(log_p n) per query. When n is huge (e.g., 10¹⁸) and p = 10⁹+7, log_p n is about 2, so just a few multiplications.

Edge Cases

• k > n or k < 0: C(n, k) = 0. Return 0.
• k = 0 or k = n: C(n, 0) = C(n, n) = 1. Lucas gives product of C(n_i, 0) = 1 for each digit → 1.
• Any digit k_i > n_i: C(n_i, k_i) = 0, so entire product 0. Return 0.
• n = 0: digits [0]; k must be 0 (else k > n); C(0,0) = 1.

Common Mistakes

• Using Lucas when p is not prime: The theorem holds only for prime p. For composite modulus, use prime-power factorization and CRT, or use a different method.
• Precomputing factorials up to n: The whole point of Lucas is that n can be huge. Precompute only 0..p−1.
• Digit order: Lucas uses base-p digits; product is over the same position i. LSB first is consistent as long as both n and k use the same convention.

Optimization Insight

Interview Insight

Practice Problems

• Implement digits_base_p, nCr_mod_small, and lucas; verify C(10,3) mod 5 = 0, C(7,2) mod 5 = 1.
• Solve a problem that asks for C(n, k) mod 10⁹+7 with n ≤ 10¹⁸ using Lucas.
• Compare: when n < p, direct nCr with fact[0..n] vs Lucas (both work; direct is simpler).

Summary

• Lucas’s theorem (prime p): Write n, k in base p; then C(n, k) ≡ Π_i C(n_i, k_i) (mod p). If any k_i > n_i, result is 0.
• Precompute fact and inv_fact for 0..p−1 (O(p)). Each query: get base-p digits of n and k, multiply C(n_i, k_i) for each digit—O(log_p n).
• Only applies when p is prime. For composite modulus use prime factors + CRT.
• Use when n (or k) can be much larger than p; when n, k < p, direct nCr is enough.

4.16 Inclusion-Exclusion Principle

Introduction

The inclusion-exclusion principle counts the size of a union of finite sets by adding sizes of sets, subtracting sizes of pairwise intersections, adding sizes of triple intersections, and so on—alternating signs so that each element is counted exactly once. It is one of the most useful counting tools in DSA: “how many numbers in [1, N] are not divisible by any of these primes?” “how many permutations have at least one fixed point?” “how many strings avoid certain substrings?” All reduce to union counting. This section states the formula, gives the intuition, and shows how to implement it (often by iterating over subsets of conditions).

Real-World Analogy

You want to count how many people in a room speak English or Spanish or French. If you add “English speakers” + “Spanish” + “French,” you count anyone who speaks two languages twice, and anyone who speaks all three three times. So subtract the counts of “English and Spanish,” “English and French,” “Spanish and French.” Now those who speak all three were added three times and subtracted three times (once in each pair)—so add back “English and Spanish and French.” The result is the count of people who speak at least one of the three. That’s inclusion-exclusion: add singles, subtract pairs, add triples.

Formal Definition

The “inclusion” is adding; the “exclusion” is subtracting to correct overcounts. The sign alternates: odd-sized subsets add, even-sized subtract (or vice versa depending on how you write it; the key is that each element in the union is counted exactly once).

Why This Topic Matters

• “Count numbers not divisible by any of …”: Let A_i = numbers divisible by p_i. Then “not divisible by any” = total − |A₁ ∪ … ∪ A_k|. Intersection of A_i for i ∈ S is “divisible by LCM of those p_i”—size ⌊N / LCM⌋.
• Derangements: Permutations with no fixed point = n! − (permutations with at least one fixed point). The latter is inclusion-exclusion over “position i is fixed.”
• Contest problems: Many “count valid configurations” or “count numbers with property P” use inclusion-exclusion over violating conditions.

Mental Model

We want |A₁ ∪ … ∪ A_n|. If we add all |A_i|, we overcount elements in more than one set. Subtract intersections of two sets; then we undercount elements in three sets. Add intersections of three sets; and so on. The alternating sum makes every element in the union contribute exactly 1 (proved by checking how many times an element in exactly r sets is counted: C(r,1) − C(r,2) + … + (−1)^(r+1) C(r,r) = 1).

Two and Three Sets

Two sets: |A ∪ B| = |A| + |B| − |A ∩ B|.

Three sets: |A ∪ B ∪ C| = |A| + |B| + |C| − |A∩B| − |A∩C| − |B∩C| + |A∩B∩C|.

Pattern: sum of singles, minus sum of pairs, plus sum of triples, …

Step-by-Step: Applying Inclusion-Exclusion

1. Define the sets: Identify conditions (e.g., “divisible by p_i”). Let A_i be the set of elements satisfying condition i.
2. Decide what to count: Often we want “elements in none of the sets” = total − |∪ A_i|, or “elements in at least one” = |∪ A_i|.
3. Compute intersection sizes: For each non-empty subset S, compute |∩_i∈S A_i|. This is problem-specific (e.g., “divisible by LCM of primes in S” → ⌊N / LCM⌋).
4. Combine with alternating signs: |∪ A_i| = Σ (−1)^|S|+1 |∩_i∈S A_i| over non-empty S. Or “none” = total − that sum.

Example: Numbers Not Divisible by 2 or 3

Python Implementation (Subset Iteration)

Iterate over non-empty subsets of {0, 1, …, k−1} using bitmasks 1 to 2^k−1. For each subset S, compute the size of the intersection (problem-specific) and add (−1)^|S|+1 × size to the result.

def inclusion_exclusion_union(n_conditions: int, intersection_size: callable) -> int:
    """
    Returns |A0 ∪ A1 ∪ ... ∪ A_{k-1}|.
    intersection_size(S) returns |∩_{i in S} A_i| for S a set or bitmask.
    S is represented as a bitmask: S has bit i set iff i is in S.
    """
    k = n_conditions
    total = 0
    for mask in range(1, 1 << k):
        pop = bin(mask).count("1")
        sign = 1 if pop % 2 == 1 else -1
        total += sign * intersection_size(mask)
    return total

# Example: count 1..N not divisible by any prime in primes
def count_not_divisible(N: int, primes: list[int]) -> int:
    from math import lcm
    def inter_size(mask: int) -> int:
        prod = 1
        for i in range(len(primes)):
            if (mask >> i) & 1:
                prod = lcm(prod, primes[i])
                if prod > N:
                    return 0
        return N // prod
    return N - inclusion_exclusion_union(len(primes), inter_size)

intersection_size(mask) must return the size of the intersection of sets A_i for which bit i is set in mask. For “divisible by prime p_i,” the intersection is “divisible by LCM of selected primes,” so size = N // LCM (or 0 if LCM > N).

Line-by-Line Explanation

• for mask in range(1, 1 << k) — Non-empty subsets: mask from 1 to 2^k−1; bit i set means set A_i is in the intersection.
• pop = bin(mask).count("1") — |S| = number of sets in this intersection.
• sign = 1 if pop % 2 == 1 else -1 — (−1)^|S|+1: odd |S| → +1, even → −1. So we add singles, subtract pairs, add triples, …
• total += sign * intersection_size(mask) — Add signed size of this intersection.
• In count_not_divisible: “not divisible by any” = N − |union|. inter_size(mask) computes LCM of primes in mask and returns N // LCM (or 0 if LCM > N).

Time and Space Complexity

We iterate 2^k − 1 subsets. For each subset we call intersection_size once. So O(2^k × cost of intersection_size). For the “not divisible by primes” example, each intersection is O(k) for LCM (or O(1) if we precompute LCMs). So total O(2^k · k) or similar. Space O(1) plus the cost of intersection_size. When k is large (e.g., 20), 2^k can be acceptable; when k is very large, we may need a different approach or pruning.

Edge Cases

• No conditions (k = 0): Union of zero sets is empty; |∪| = 0. Or define “not in any” = all N elements.
• Single set (k = 1): |A₁| = one term; no subtraction.
• LCM exceeds N: Intersection “divisible by LCM” has size 0; return 0 to avoid invalid division or wrong count.

Common Mistakes

• Wrong sign: The formula is |∪| = Σ (−1)^|S|+1 |∩_S|. So odd-sized subsets add, even-sized subtract. Reversing the sign gives the complement (e.g., “count in none” vs “count in at least one”)—double-check what you want.
• Forgetting the empty subset: We do not include the empty subset in the sum (empty intersection would be the whole universe; we’re not adding that). So start mask from 1.
• Wrong intersection meaning: For “divisible by p_i,” the intersection over S is “divisible by LCM of p_i for i ∈ S,” not product (product is correct only when primes are pairwise coprime; LCM = product for primes).

Pattern Recognition

When the problem asks “count elements that satisfy none of the bad conditions” or “avoid all of these,” define A_i = “satisfies bad condition i.” Then “satisfies none” = total − |∪ A_i|. When it asks “count elements that satisfy at least one,” you want |∪ A_i| directly. The same subset loop works; only the interpretation (and possibly the final subtraction) changes.

Interview Insight

Practice Problems

• Count integers in [1, N] not divisible by any of 2, 3, 5 using inclusion-exclusion; verify for N = 100.
• Count derangements of n (permutations with no fixed point): n! − (at least one fixed point); expand “at least one fixed point” with inclusion-exclusion over positions.
• Given a list of primes, count coprime integers in [1, N] (coprime to product of primes) = same as “not divisible by any prime.”

Summary

• Inclusion-exclusion: |A₁ ∪ … ∪ A_n| = Σ_∅≠S (−1)^|S|+1 |∩_i∈S A_i|. Add singles, subtract pairs, add triples, …
• Implement by iterating non-empty subsets (e.g., mask 1 to 2^k−1); for each subset compute intersection size and add with sign (−1)^|S|+1.
• “Count in none” = total − |∪|. “Count in at least one” = |∪|. For “not divisible by any of primes,” intersection over S = numbers divisible by LCM(primes in S), size ⌊N / LCM⌋.
• Time O(2^k × cost of intersection). Watch sign and empty subset.

4.17 Matrix Exponentiation for Recurrence

Introduction

Many linear recurrences (e.g., Fibonacci: F_n = F_n−1 + F_n−2) can be written as a state vector updated by a fixed matrix: v_n = M · v_n−1. Then v_n = Mⁿ · v₀, so we get the n-th term by matrix exponentiation (compute Mⁿ with fast exponentiation from topic 4.6, using matrix multiplication from topic 4.12) and then multiplying by the initial vector. This gives the n-th term in O(d³ log n) time where d is the dimension of the state (e.g., d = 2 for Fibonacci), instead of O(n) with a naive loop. This section shows how to build the matrix from a recurrence and how to implement it in code (including modulo).

Real-World Analogy

Think of the recurrence as a state machine: at each step, the current state (e.g., “last two Fibonacci numbers”) is updated by a fixed rule. That rule is linear—it’s a matrix multiplying the state vector. Doing n steps means applying the matrix n times = Mⁿ. Fast exponentiation lets us compute Mⁿ in about log n “matrix multiplications,” so we jump from “step one by one” to “double the number of steps” each time.

Formal Definition

The matrix M encodes the recurrence: the first row is (c₁, c₂, …, c_d); the rest of the rows shift the previous state (identity-like with a shift).

Why This Topic Matters

• Fibonacci and similar: F_n in O(log n) instead of O(n). Essential when n is huge (e.g., 10¹⁸).
• Linear recurrences in contests: Many problems give a recurrence of order 2 or 3; matrix exponentiation is the standard solution.
• Counting paths: Number of walks of length n in a graph = (adjacency matrix)ⁿ; same idea (matrix power).

Fibonacci as a 2×2 Matrix

F_n = F_n−1 + F_n−2. State: v_n = (F_n, F_n−1)^T. We want v_n from v_n−1 = (F_n−1, F_n−2)^T. So F_n = 1·F_n−1 + 1·F_n−2 and F_n−1 = 1·F_n−1 + 0·F_n−2. Hence:

  [ F_n   ]   [ 1  1 ]   [ F_{n-1} ]
  [ F_{n-1} ] = [ 1  0 ] · [ F_{n-2} ]
  So M = [[1,1],[1,0]].  v_n = M · v_{n-1},  so v_n = M^{n-1} · v_1,  with v_1 = (F_1, F_0)^T = (1, 0)^T.
  Thus (F_n, F_{n-1})^T = M^{n-1} · (1, 0)^T;  F_n = (M^{n-1})_{0,0} (top-left of M^{n-1}) or first component of M^{n-1} * (1,0)^T.
  Actually: v_1 = (F_1, F_0) = (1, 0). v_2 = M*v_1 = (1, 1). So v_n = M^{n-1} * v_1. F_n = first entry of v_n = first entry of M^{n-1} * (1,0)^T = (M^{n-1})_{00} * 1 + (M^{n-1})_{01} * 0 = (M^{n-1})_{00}. So F_n = top-left entry of M^{n-1}.
  For n=1: M^0 = I, (1,0) -> F_1 = 1. For n=2: M^1 = M, (1,0) -> (1,1), F_2 = 1. Good.
        

So F_n = first component of Mⁿ⁻¹ · (1, 0)^T, or equivalently the top-left entry of Mⁿ⁻¹. For n = 0 we define F₀ = 0; handle separately.

Step-by-Step: Building the Matrix for a Recurrence

1. Write the recurrence: F_n = c₁ F_n−1 + … + c_d F_n−d.
2. State vector: v = (F_n, F_n−1, …, F_n−d+1)^T (d components).
3. First row of M: (c₁, c₂, …, c_d) — these are the coefficients of the recurrence.
4. Row i (i ≥ 2): has a 1 in column i−1 and 0 elsewhere — shifts F_n−1 into second slot, etc. So M is: row 0 = [c₁, c₂, …, c_d]; row 1 = [1, 0, …, 0]; row 2 = [0, 1, 0, …, 0]; …; row d−1 = [0, …, 0, 1, 0].
5. Initial vector: v_d−1 = (F_d−1, F_d−2, …, F₀)^T. Then v_n = M^n−d+1 · v_d−1 for n ≥ d.
6. Compute M^n−d+1 with binary exponentiation (matrix version); multiply by v_d−1; the first component is F_n.

Python Implementation

Matrix Power (Modulo)

def mat_pow_mod(M: list[list[int]], exp: int, mod: int) -> list[list[int]]:
    """Returns M^exp mod mod. M is square (d×d)."""
    d = len(M)
    if exp == 0:
        I = [[1 if i == j else 0 for j in range(d)] for i in range(d)]
        return I
    base = [row[:] for row in M]
    res = [[1 if i == j else 0 for j in range(d)] for i in range(d)]
    while exp:
        if exp & 1:
            res = mat_mul_mod(res, base, mod)
        base = mat_mul_mod(base, base, mod)
        exp >>= 1
    return res

Assume mat_mul_mod(A, B, mod) from topic 4.12 (multiplies two matrices and reduces mod mod).

Fibonacci F_n mod m

def fib_mod(n: int, m: int) -> int:
    if n <= 0:
        return 0
    if n == 1:
        return 1
    M = [[1, 1], [1, 0]]
    P = mat_pow_mod(M, n - 1, m)
    # v_n = P * (1, 0)^T; F_n = P[0][0]*1 + P[0][1]*0 = P[0][0]
    return P[0][0]

General Linear Recurrence (Order 2)

F_n = a·F_n−1 + b·F_n−2. Matrix M = [[a, b], [1, 0]]. v_n = Mⁿ⁻¹ · (F₁, F₀)^T. F_n = first component of Mⁿ⁻¹ · (F₁, F₀)^T = P[0][0]*F₁ + P[0][1]*F₀.

Line-by-Line Explanation (mat_pow_mod)

• if exp == 0 — M⁰ = identity matrix.
• res = identity — Accumulator for the result; we multiply by base when the current bit of exp is 1.
• while exp: if exp & 1: res = mat_mul_mod(res, base, mod) — Binary exponentiation: when the LSB of exp is 1, multiply res by base.
• base = mat_mul_mod(base, base, mod); exp >>= 1 — Square base and shift exp (same as integer fast exponentiation).

Time and Space Complexity

Matrix multiplication (d×d): O(d³). Matrix power: O(log n) multiplications, so O(d³ log n) time. Space O(d²) for the matrices. For Fibonacci (d = 2), this is O(log n) — much better than O(n) with a loop when n is huge.

Edge Cases

• n = 0 or n = 1: Handle before matrix power (F₀ = 0, F₁ = 1 for standard Fibonacci).
• Negative n: Often undefined for recurrences; return 0 or handle as invalid.
• mod = 1: All entries become 0; return 0.

Common Mistakes

• Wrong matrix: The first row must be the recurrence coefficients in order (c₁, c₂, …). Rows below shift: (1,0,…,0), (0,1,0,…), …. Swapping rows or columns gives wrong results.
• Wrong initial vector or exponent: v_n = Mⁿ⁻¹ · v₁ for Fibonacci (state has F_n, F_n−1). So we need Mⁿ⁻¹, not Mⁿ. Check with n = 2: M¹ · (1,0) = (1,1) → F₂ = 1 ✓.
• Index off by one: F_n = first component of v_n = Mⁿ⁻¹ · v₁. So use exponent n−1 for Fibonacci.

Optimization Insight

Interview Insight

Practice Problems

• Implement mat_pow_mod and fib_mod(n, m); verify F_10 = 55, F_0 = 0, F_1 = 1.
• Solve F_n = 2*F_{n-1} + 3*F_{n-2} with given F_0, F_1 using matrix exponentiation.
• Count the number of ways to tile a 2×n board with 2×1 dominoes (recurrence: a_n = a_{n-1} + a_{n-2}; same as Fibonacci).

Summary

• Linear recurrence F_n = c₁·F_n−1 + … + c_d·F_n−d can be written v_n = M · v_n−1; then v_n = M^n−d+1 · v_d−1. First row of M is (c₁, …, c_d); below that, shift rows.
• Fibonacci: M = [[1,1],[1,0]], v₁ = (1,0)^T; F_n = first component of Mⁿ⁻¹ · v₁ = (Mⁿ⁻¹)_0,0.
• Compute M^exp with matrix binary exponentiation (same as fast exponentiation, with mat_mul_mod). Time O(d³ log n), space O(d²).
• Handle n < d (base cases) separately. Use exponent n−1 (not n) for Fibonacci to get F_n.

5.1 Array Basics

Introduction

An array is a contiguous block of memory that stores a sequence of elements of the same type, each identifiable by an index. In Python, the built-in list is the primary “array” type: it supports indexing (arr[i]), length (len(arr)), and dynamic growth (append). Arrays are the foundation of most data structures and algorithms—strings are arrays of characters, matrices are arrays of arrays, and most problems involve traversing or querying an array. This section covers what an array is, 0-based indexing, basic operations (access, update, traverse), and how Python lists behave so you can reason about time complexity and edge cases.

Real-World Analogy

Think of an array like a row of lockers or parking spots numbered 0, 1, 2, … Each slot holds one item. You can go directly to slot 5 (O(1) “access”) and read or replace what’s there. You can walk the row from start to end (traversal). The “address” of each slot is computed from the base address plus the index—that’s why access by index is constant time. Adding a new slot at the end is cheap (append); inserting in the middle or at the front may require shifting (expensive in a true array; Python lists hide this with amortized cost).

Formal Definition

In a static array, the size is fixed at creation. In a dynamic array (like Python’s list), the size can grow (and sometimes shrink); append is amortized O(1), but insert at position 0 is O(n) because elements must be shifted.

Why This Topic Matters

• Foundation: Almost every data structure (strings, heaps, graphs as adjacency lists) uses arrays. Two-pointer, sliding window, and prefix-sum techniques all operate on arrays.
• Interviews: “Given an array of integers…” is the most common problem start. You must be comfortable with indexing, bounds, and traversal.
• Complexity: Access O(1); search by value O(n); insert at end amortized O(1); insert at front or middle O(n). Choosing the right structure (array vs linked list) depends on these costs.

Mental Model

Picture a row of boxes numbered 0, 1, …, n−1. Each box holds one value. “arr[i]” means “open box i.” “len(arr)” is the number of boxes. Traversal is “visit each box in order.” Slicing arr[start:end] is “the segment from box start up to (but not including) box end.” Negative index −1 means “last box,” −2 means “second to last,” and so on.

Indexing and Slicing in Python

• 0-based: First element is arr[0], last is arr[len(arr)−1] or arr[−1].
• Negative indices: arr[−1] is the last element, arr[−2] the second-to-last; arr[−i] is the same as arr[len(arr)−i] (for valid i).
• Slicing: arr[start:end] gives elements from index start to end−1 (end excluded). arr[:end] means start=0; arr[start:] means end=len(arr). arr[::step] can reverse (step=−1) or skip elements.

arr = [10, 20, 30, 40, 50]
arr[0]    # 10
arr[-1]   # 50
arr[1:4]  # [20, 30, 40]
arr[::-1] # [50, 40, 30, 20, 10] — reverse

Basic Operations

Traversal

# By index
for i in range(len(arr)):
    print(arr[i])

# By value
for x in arr:
    print(x)

# With index and value
for i, x in enumerate(arr):
    print(i, x)

Edge Cases

• Empty array: arr = []; len(arr) = 0. Accessing arr[0] raises IndexError. Check len(arr) or use “if arr” before indexing.
• Index out of bounds: arr[i] when i < 0 or i ≥ len(arr) raises IndexError (for negative i, Python interprets as relative to end; −1 is valid if len ≥ 1).
• Single element: arr[0] and arr[−1] are the same; no special case needed.

Common Mistakes

• Off-by-one: Valid indices are 0 to len(arr)−1. Loop “for i in range(len(arr))” gives 0..len−1. Using range(1, len(arr)) skips the first element; range(len(arr)+1) causes IndexError on the last iteration.
• Modifying list while iterating: Removing or inserting elements during “for x in arr” can skip elements or cause errors. Iterate over a copy (e.g., arr[:]) or use indices and adjust.
• Assuming append returns the list: arr.append(x) returns None and mutates arr. Don’t write result = arr.append(x) and expect result to be the new list.

Interview Insight

Practice Problems

• Given an array, return the maximum element and its index (one pass).
• Reverse an array in place (swap arr[i] and arr[n−1−i] for i in range(len(arr)//2)).
• Check if an array is palindrome (compare arr[i] and arr[n−1−i]).

Summary

• An array is a contiguous sequence of elements indexed 0 to n−1. Access and update by index are O(1).
• Python list: append amortized O(1), insert O(n), “x in arr” O(n). Use 0-based and negative indices (arr[−1] = last).
• Traverse with for i in range(len(arr)), for x in arr, or enumerate(arr). Watch empty array and index bounds.
• Off-by-one and “modify while iterating” are common bugs; clarify 0 vs 1-based when the problem says “position.”

5.2 Searching

Introduction

Searching in an array means finding whether a given value (the target) exists and, often, at which index. The right search strategy depends on whether the array is sorted or unsorted. For an unsorted array, you have no choice but to scan elements until you find the target or reach the end—linear search, O(n). For a sorted array, you can repeatedly eliminate half of the remaining elements—binary search, O(log n). This section builds both from scratch, explains why binary search works only when the array is sorted, and shows how to implement "first index of," "last index of," and related variants so you can handle interview problems confidently.

Real-World Analogy

Imagine finding a name in a phone book (sorted A–Z). You don't read every page: you open the middle, see "M," and know the name is either in the first half or the second half. You throw away one half and repeat. That's binary search—each step halves the search space. Now imagine a pile of unsorted receipts. To find one with a certain date, you must look through them one by one until you find it or run out. That's linear search. The structure of the data (sorted vs not) determines which strategy is possible and how fast it can be.

Formal Definition

Binary search requires a sorted array (or an array that can be treated as sorted with a predicate). The "discard half" step is valid only when you know that all elements in one half cannot contain the target—which follows from the ordering.

Why This Topic Matters

• Interviews: "Find the index of target in a sorted array" is a classic. You must code binary search correctly (bounds, loop condition, when to return) and know first/last occurrence variants.
• Building block: Binary search on answer (Topic 5.9), search in rotated array, and many optimization problems use the same "narrow the range" idea.
• Complexity: Going from O(n) to O(log n) when the array is sorted is a huge win for large n. Knowing when you can use binary search (sorted + comparable) is essential.

Mental Model

Linear search: "Walk from index 0 to n−1; stop when you see the target or run out." Binary search: "Keep a range [left, right] where the target might be. While the range is non-empty, look at the middle element. If it equals the target, you're done. If it's less than the target, the target must be in the right half (if at all). If it's greater, the target must be in the left half. Update the range and repeat." The key is that "left" and "right" are indices, and you always shrink the range; the loop exits when left > right (range is empty) or when you find the target.

Linear Search (Unsorted or Any Array)

Algorithm

1. For each index i from 0 to n−1:
2. If arr[i] == target, return i (or True).
3. If the loop finishes without returning, the target is not present; return −1 (or False).

Python Implementation

def linear_search(arr, target):
    for i in range(len(arr)):
        if arr[i] == target:
            return i
    return -1

Time O(n): in the worst case we check every element. Space O(1). This is optimal for an unsorted array—you cannot avoid looking at every element in the worst case (the target might be last or absent).

Python built-ins: target in arr returns True/False (same idea, O(n)). arr.index(target) returns the first index or raises ValueError; also O(n).

Binary Search (Sorted Array)

Why It Works

If the array is sorted in non-decreasing order, then for any index mid, every element to the left is ≤ arr[mid] and every element to the right is ≥ arr[mid]. So when we compare target to arr[mid]:

• If target == arr[mid], we found it.
• If target < arr[mid], the target cannot be at mid or to the right; search only [left, mid−1].
• If target > arr[mid], the target cannot be at mid or to the left; search only [mid+1, right].

Each step removes at least half of the remaining indices, so after O(log n) steps the range is empty or we find the target.

ASCII Diagram: Binary Search Step

  Sorted array:  [ 2,  5,  7,  9, 12, 15 ]   target = 9
  Index:          0   1   2   3   4   5

  Step 1: left=0, right=5, mid=2 → arr[2]=7 < 9 → search right half
          [ 2,  5,  7, | 9, 12, 15 ]
                       ↑
  Step 2: left=3, right=5, mid=4 → arr[4]=12 > 9 → search left half
          [ 9, 12, 15 ]
            ↑
  Step 3: left=3, right=3, mid=3 → arr[3]=9 == 9 → found at index 3.
        

Standard Binary Search (Any Occurrence)

def binary_search(arr, target):
    left, right = 0, len(arr) - 1
    while left <= right:
        mid = (left + right) // 2
        if arr[mid] == target:
            return mid
        if arr[mid] < target:
            left = mid + 1
        else:
            right = mid - 1
    return -1

Line-by-Line Explanation

• left <= right: When left == right, the range has one element; we must check it. So the loop condition is left <= right. Exiting when left > right means the range is empty—target not found.
• mid = (left + right) // 2: Middle index (integer division). Avoids overflow in other languages; in Python, (left + right) // 2 is standard.
• If arr[mid] < target, every element at index ≤ mid is too small, so set left = mid + 1. If arr[mid] > target, every element at index ≥ mid is too large, so set right = mid - 1.

First Occurrence (Leftmost Index)

When duplicates are allowed, "find the first index where arr[i] == target" requires a small change: when arr[mid] == target, don't return yet—remember mid as a candidate and continue searching the left half (there might be an earlier occurrence).

def first_index(arr, target):
    left, right = 0, len(arr) - 1
    result = -1
    while left <= right:
        mid = (left + right) // 2
        if arr[mid] == target:
            result = mid
            right = mid - 1   # keep looking left
        elif arr[mid] < target:
            left = mid + 1
        else:
            right = mid - 1
    return result

When we find a match, we shrink the range to [left, mid−1] to see if there's another match to the left. If not, result holds the leftmost index we saw.

Last Occurrence (Rightmost Index)

Similarly, for the last occurrence: when arr[mid] == target, set result = mid and search the right half with left = mid + 1.

def last_index(arr, target):
    left, right = 0, len(arr) - 1
    result = -1
    while left <= right:
        mid = (left + right) // 2
        if arr[mid] == target:
            result = mid
            left = mid + 1    # keep looking right
        elif arr[mid] < target:
            left = mid + 1
        else:
            right = mid - 1
    return result

Evolution: Brute Force → Linear → Binary

Time and Space Complexity

• Linear search: Time O(n), space O(1).
• Binary search (iterative): Time O(log n)—each step halves the range, so at most ⌈log₂(n+1)⌉ iterations. Space O(1).
• Binary search (recursive): Same time O(log n), but space O(log n) for the call stack.

Edge Cases

• Empty array: Linear: loop doesn't run, return −1. Binary: left=0, right=-1, so left <= right is false; return −1.
• Target not present: Both return −1 (or your chosen sentinel).
• Single element: Linear: one comparison. Binary: one iteration, left==right, check arr[mid].
• All same value: Linear finds the first. Standard binary finds any; first_index/last_index give the correct boundary.
• Unsorted array: Binary search is wrong—it can miss the target or return an arbitrary index. Always ensure the array is sorted (or use a predicate that preserves the "discard half" property) before using binary search.

Common Mistakes

• Using binary search on an unsorted array: Binary search assumes sorted order. If the array isn't sorted, use linear search or sort first.
• Off-by-one in loop condition: Use left <= right so that when left == right you still check that single element. Using left < right can skip the last candidate.
• Wrong mid update: When arr[mid] < target, the target is in the right half, so left = mid + 1. When arr[mid] > target, right = mid - 1. Don't set left = mid or right = mid without ±1, or the range might not shrink and you can get an infinite loop.
• Integer overflow for mid: In C/Java, mid = (left + right) / 2 can overflow for very large indices. Use mid = left + (right - left) / 2. In Python, (left + right) // 2 is fine.

Python Built-ins: bisect

The bisect module provides binary search for sorted lists:

• bisect.bisect_left(arr, target): leftmost index where arr[i] >= target (insertion point to keep sorted). If target is present, this is the first occurrence.
• bisect.bisect_right(arr, target) (or bisect.bisect): rightmost index where arr[i] <= target is still true before the next element—i.e., one past the last occurrence of target.

So "first index of target" is bisect_left (and check arr[i]==target); "last index" is bisect_right(arr, target) - 1 (and check). Count of target = bisect_right(arr, target) - bisect_left(arr, target).

Practice Problems

• Binary search: Sorted array, return index of target or −1.
• First and last position: Sorted array with duplicates; return [first_index, last_index] of target or [−1, −1].
• Search insert position: Sorted array, return the index where target would be inserted to keep order (same as bisect_left).
• Count occurrences: Sorted array, count how many times target appears (last_index − first_index + 1, or use bisect).

Summary

• Linear search: Scan from 0 to n−1; O(n) time, O(1) space. Use for unsorted arrays or when you need a simple one-off check.
• Binary search: Requires sorted array. Maintain [left, right]; compare target to middle; discard half each time. O(log n) time, O(1) space (iterative).
• Use left <= right and mid = (left+right)//2; update left = mid+1 or right = mid−1 so the range always shrinks.
• First occurrence: when match, search left (right = mid - 1). Last occurrence: when match, search right (left = mid + 1).
• Python: in and index() are linear. For sorted lists, use bisect_left / bisect_right for insertion position and range of target.

5.3 Insertion & Deletion

Introduction

Insertion means adding a new element at a given position; deletion means removing an element (by index or by value). Because array elements are stored in contiguous memory, inserting or deleting in the middle (or at the front) forces the rest of the elements to shift—that’s why these operations are O(n) in the worst case. Appending at the end is the exception: no shift is needed, so it’s O(1) amortized in a dynamic array like Python’s list. This section covers exactly when and why shifting happens, how to implement insertion and deletion correctly, and how to reason about time complexity so you can choose the right structure (array vs linked list) when the problem involves many middle insertions or deletions.

Real-World Analogy

Imagine a row of cars in a parking lot with no gaps. To add a car at spot 2, you must move the car currently at 2 (and every car after it) one spot forward to make room—that’s insertion and shifting. To remove the car at spot 2, you must move every car after it one spot backward to close the gap—that’s deletion and shifting. Adding a car at the end of the row doesn’t require moving anyone. The “contiguous, no gaps” rule is what makes middle insertions and deletions expensive; a linked list is like having each car point to the next, so you can insert or remove by changing pointers without moving everyone.

Formal Definition

In a static array (fixed size), insertion might be impossible if the array is full; deletion only “marks” or overwrites. In a dynamic array (Python list), the structure can grow and shrink; the implementation hides reallocation, but the shift cost remains when inserting or deleting not at the end.

Why This Topic Matters

• Complexity reasoning: You must know that append is cheap and insert(0, x) or insert(i, x) for small i is expensive. Same for pop(0) vs pop().
• Choosing data structures: If the problem has many insertions/deletions at the front or middle, an array (list) may be the wrong choice—deque or linked list can offer O(1) at ends or O(1) at a known node.
• Interviews: “Implement a list that supports insert and delete” or “why is insert at front slow?”—you need to explain shifting and give the correct big-O.

Mental Model

Picture the array as a row of slots. Insert at i: Make room by shifting everything from i to the end one step to the right, then write the new element at i. Delete at i: Remove the element at i and shift everything from i+1 to the end one step to the left. The “shift” is a loop that copies elements; the cost is proportional to how many elements lie after the insertion or deletion point.

Insertion

Insert at End (Append)

No shifting: the new element goes into the next available slot. In Python, arr.append(x) is O(1) amortized (occasional reallocation when capacity is exceeded, but amortized constant).

arr = [10, 20, 30]
arr.append(40)   # arr → [10, 20, 30, 40]

Insert at Position i

Elements at indices i, i+1, …, n−1 must move one place right; then write the new element at i. So we need space for one more element (dynamic array handles this) and a loop that copies from right to left to avoid overwriting.

  Before:  [ 10, 20, 30, 40 ]   insert 25 at index 2
  Index:      0   1   2   3

  Step 1: Shift right from index 2 onward (copy 40→3, 30→2)
  After:   [ 10, 20, 25, 30, 40 ]
                       ↑ new
        

Python: arr.insert(i, x) inserts x at index i; all elements at i and beyond shift right. Time O(n).

arr = [10, 20, 30, 40]
arr.insert(2, 25)   # arr → [10, 20, 25, 30, 40]

Insert at beginning (insert(0, x)) shifts all n elements—O(n). Insert at end is same as append—O(1) amortized.

Deletion

Delete by Index

Remove the element at index i; elements at i+1, …, n−1 shift left by one. Time O(n) because up to n−1 elements may move.

  Before:  [ 10, 20, 30, 40 ]   delete index 1
  After:   [ 10, 30, 40 ]
               ↑ 30 and 40 moved left
        

Python: arr.pop(i) removes and returns the element at index i (default i = last, so pop() is O(1) at end). pop(0) is O(n)—shifts all remaining elements.

arr = [10, 20, 30, 40]
arr.pop(1)    # returns 20, arr → [10, 30, 40]
arr.pop()     # returns 40, arr → [10, 30]   (pop from end, O(1))

Delete by Value

Find the first occurrence of the value (O(n) search) and remove it (shift the rest left, O(n)). Total O(n). Python: arr.remove(x) removes the first occurrence of x; raises ValueError if not found.

arr = [10, 20, 20, 30]
arr.remove(20)   # removes first 20 → [10, 20, 30]

Step-by-Step: Manual Insert and Delete (Conceptual)

Insert x at index i (without using insert):

1. Ensure capacity (or append a dummy so length increases by 1).
2. For j from n−1 down to i: set arr[j+1] = arr[j]. (Shift right from the end so we don’t overwrite.)
3. Set arr[i] = x.

Delete at index i (without using pop):

1. For j from i to n−2: set arr[j] = arr[j+1]. (Shift left.)
2. Decrease length by 1 (or remove last slot).

Python Implementation Summary

# Insertion
arr.append(x)           # end, O(1) amortized
arr.insert(i, x)        # at index i, O(n)

# Deletion
arr.pop()               # remove last, O(1)
arr.pop(i)              # remove at index i, O(n)
arr.remove(x)           # remove first occurrence of x, O(n)

# Length changes
len(arr)                # after insert +1, after delete −1

Line-by-Line Notes

• insert(0, x) and pop(0) are O(n)—avoid in a loop or use collections.deque for O(1) at both ends.
• remove(x) only removes the first match. To remove all occurrences, either loop (careful: indices change) or build a new list with a list comprehension.
• Neither insert nor append nor pop returns the list; they mutate and return the element (pop) or None (insert/append/remove).

Evolution: Many Insertions/Deletions

Time Complexity

• append(x): O(1) amortized.
• insert(i, x): O(n); worst when i=0 (shift all).
• pop(): O(1).
• pop(i): O(n); worst when i=0.
• remove(x): O(n) (find + shift).

Space Complexity

In-place insertion/deletion: O(1) extra space (aside from the list’s own storage). The list may over-allocate for growth; that’s implementation-dependent and amortized.

Edge Cases

• Empty list: pop() and pop(i) raise IndexError; remove(x) raises ValueError. Check if arr before popping.
• Single element: pop(0) or pop() both remove the only element and leave an empty list.
• Index out of range: insert(len(arr), x) is valid (same as append). insert(i, x) for i > len(arr) can raise or append (Python: insert at end if i > len). pop(i) for i ≥ len raises IndexError.
• Value not present: remove(x) raises ValueError if x not in list. Check if x in arr first if you need to avoid the exception.

Common Mistakes

• Using insert(0, x) or pop(0) in a loop: That’s O(n) per call, so k operations become O(kn). Use deque for queue-like behavior.
• Removing while iterating: Deleting elements in a for x in arr loop skips elements (index shifts). Iterate over a copy (e.g. for x in arr[:]) or use a while loop and adjust index when you remove.
• Assuming remove removes all occurrences: It removes only the first. To remove all, use list comprehension arr = [a for a in arr if a != x] or a loop over a copy.
• Expecting insert/append to return the list: They return None; the list is modified in place.

Pattern Recognition

When a problem involves “add element” or “remove element”:

• If only at the end: list append/pop is fine.
• If at the front: consider deque (Topic 9.7).
• If in the middle and you need to preserve order: list insert/pop is O(n) per op; acceptable for small n or few operations.

Practice Problems

• Implement “insert at index i” and “delete at index i” on a list without using insert/pop (use a loop to shift).
• Remove all occurrences of value x from a list in place (one pass with two pointers or build new list and assign back).
• Merge two sorted arrays into one sorted array (compare and push to result; no “insert in middle” needed—just append).

Summary

• Insertion at i: Shift elements from i right, then write. O(n). Append is O(1) amortized.
• Deletion at i: Shift elements from i+1 left. O(n). Pop from end is O(1).
• Python: insert(i,x), pop(i), remove(x); avoid insert(0,x) and pop(0) in a loop—use deque for O(1) at both ends.
• Don’t remove (or insert) while iterating over the same list; use a copy or index-based loop with care.

5.4 Two Pointer Technique

Introduction

The two pointer technique uses two indices (or pointers) that move through an array—often from opposite ends or both from the start—to solve a problem in one pass with O(n) time and O(1) extra space. Instead of nested loops (e.g. checking every pair), you move the pointers based on the current values and the problem condition, so each element is considered at most a constant number of times. Typical uses: finding a pair with a given sum in a sorted array, removing duplicates in place, checking if a sequence is a palindrome, or partitioning (e.g. move zeros to the end). This section covers the main patterns (converging pointers, same-direction pointers) and when to apply each.

Real-World Analogy

Imagine two people walking toward each other from opposite ends of a corridor. They meet in the middle. If the corridor is sorted by some rule (e.g. height), you can decide at each step whether the “left” person or the “right” person should move so you get closer to a goal (e.g. two people whose heights sum to a target). That’s the converging two-pointer pattern. Alternatively, imagine one person walking fast and one slow along the same path—useful for finding the “middle” or a cycle. That’s the fast/slow or same-direction pattern. In both cases, you avoid checking every possible pair by using the structure of the array (e.g. sorted) to rule out large parts of the search space.

Formal Definition

Converging pointers work well when the array is sorted (or has a known structure) so that moving one pointer in one direction has a predictable effect (e.g. sum increases or decreases). Same-direction pointers work for in-place compaction, “read” vs “write” positions, or fast/slow for cycle or middle detection.

Why This Topic Matters

• Interviews: Two Sum (sorted), 3Sum, remove duplicates, move zeros, palindrome, container with most water—all frequently asked and often solved with two pointers.
• Efficiency: Replaces O(n²) “check every pair” with O(n) when the problem allows ruling out ranges using order or invariants.
• In-place: Many two-pointer solutions use O(1) extra space, which is required when you cannot allocate a new array.

Mental Model

Converging: You have a “window” [left, right]. The condition (e.g. sum, or “elements between”) depends on both ends. If the current window is too small (e.g. sum too low), move the pointer that will increase it (e.g. left++ on a sorted array increases the smallest element). If too large, move the pointer that will decrease it (e.g. right--). Same direction: One pointer is the “reader” (scans the array), the other is the “writer” (next position to write a valid element). Or one is “slow” and one “fast” so that when fast has moved 2k steps, slow has moved k (e.g. find middle).

Pattern 1: Converging Pointers (Opposite Ends)

Use when the array is sorted (or can be sorted) and you are comparing or combining values at two positions. Start with left = 0, right = len(arr) - 1. Loop while left < right. Depending on the problem, move left right or right left so the search space shrinks.

Two Sum in Sorted Array

Find two indices such that arr[i] + arr[j] == target. Because the array is sorted, if arr[left] + arr[right] < target, we need a larger sum—move left right. If arr[left] + arr[right] > target, move right left. If equal, return the pair.

def two_sum_sorted(arr, target):
    left, right = 0, len(arr) - 1
    while left < right:
        s = arr[left] + arr[right]
        if s == target:
            return [left, right]
        if s < target:
            left += 1
        else:
            right -= 1
    return [-1, -1]

Each iteration either increases left or decreases right, so the number of steps is at most n−1. Time O(n), space O(1).

ASCII Diagram: Converging Pointers

  Sorted: [ 1,  2,  3,  4,  5 ]   target = 7
  Index:    0   1   2   3   4
            ↑               ↑
          left            right   sum=6 < 7 → left++
               ↑            ↑
             left         right   sum=7 → return (1, 4)
        

Check Palindrome (converging)

Check if arr reads the same from both ends. Move left and right toward the center; if arr[left] != arr[right], return False. Stop when left >= right.

def is_palindrome(arr):
    left, right = 0, len(arr) - 1
    while left < right:
        if arr[left] != arr[right]:
            return False
        left += 1
        right -= 1
    return True

Pattern 2: Same-Direction Pointers (Read/Write)

One pointer scans the array (read); the other marks where to write the “next valid” element. Used for in-place removal or compaction (e.g. remove duplicates, move zeros).

Remove Duplicates In Place (Sorted)

Sorted array: keep one copy of each value, in place. write is the next index to write a unique value; read scans. When arr[read] != arr[write-1] (or write==0), copy arr[read] to arr[write] and advance write. Return write as the new length.

def remove_duplicates_sorted(arr):
    if not arr:
        return 0
    write = 1
    for read in range(1, len(arr)):
        if arr[read] != arr[write - 1]:
            arr[write] = arr[read]
            write += 1
    return write

Move Zeros to End

Keep relative order of non-zeros; put all zeros at the end. write = next position for a non-zero. Scan with read; when arr[read] != 0, swap or copy to arr[write] and increment write. Fill the rest with zeros if needed (or swap and leave zeros at end).

def move_zeros(arr):
    write = 0
    for read in range(len(arr)):
        if arr[read] != 0:
            arr[write], arr[read] = arr[read], arr[write]
            write += 1

Time O(n), space O(1).

Pattern 3: Fast and Slow Pointers

Two pointers both start at the beginning; one moves one step per iteration, the other two steps (or different rates). Used for finding the middle of a list, detecting a cycle (in linked lists), or similar “position relative to length” problems. On arrays, a common use is “slow” = write, “fast” = read, which is the same as the read/write pattern above.

Step-by-Step: Two Sum (Sorted)

1. Set left = 0, right = len(arr) - 1.
2. While left < right: compute s = arr[left] + arr[right].
3. If s == target, return (left, right).
4. If s < target, do left += 1 (we need a larger sum; increasing the left element increases the sum because the array is sorted).
5. If s > target, do right -= 1.
6. If the loop exits, no pair found; return a sentinel.

Evolution: Brute Force → Two Pointers

Two sum (sorted): Brute force: two nested loops, check every pair—O(n²). Two pointers: one pass from both ends—O(n). The key is using sorted order: if the current sum is too small, the only way to get a larger sum is to move the left pointer right; if too large, move the right pointer left.

Time and Space Complexity

• Converging (two sum, palindrome): Each iteration does O(1) work and one of the pointers moves; total iterations O(n). Time O(n), space O(1).
• Same direction (remove duplicates, move zeros): Single loop, each element read once, O(1) writes. Time O(n), space O(1).

Edge Cases

• Empty or single element: For converging, left < right is false when n ≤ 1; handle (e.g. return False or empty result). For read/write, handle n==0 so you don’t use write-1 when write is 0.
• No valid pair: Return a clear value (e.g. [-1,-1], False, or empty list).
• Multiple valid pairs: Clarify whether you need one or all (e.g. two sum usually returns one pair; 3Sum returns all unique triplets).
• Duplicates: In “remove duplicates,” sorted array is assumed. In two sum, if you need distinct indices, (left, right) are always distinct when left < right.

Common Mistakes

• Using two pointers on an unsorted array for “pair sum”: Converging logic assumes order; sort first or use a hash map.
• Wrong loop condition: Use left < right for converging (not left <= right unless you need to handle the same index twice). For “find pair,” left and right must be distinct.
• Moving the wrong pointer: In a sorted array, smaller index → smaller value. So to increase sum, move left right; to decrease, move right left. Reverse if the array is sorted descending.
• Read/write: off-by-one: In remove duplicates, the first element is always kept; write starts at 1 and we compare with arr[write-1].

Pattern Recognition

• Sorted array + pair/triplet sum or “find two”: Think converging pointers (or one pointer + binary search for the second).
• In-place remove/compact (duplicates, zeros, “remove value”): Think same-direction read/write pointer.
• Palindrome, “valid from both ends”: Converging pointers from both ends.
• Linked list middle or cycle: Fast/slow pointers (Topic 8).

Practice Problems

• Two Sum II (sorted): Return indices (1-based) of two numbers that add to target; converging pointers.
• Remove duplicates from sorted array: In place, return new length; same-direction write pointer.
• Move zeros: In place; read/write with swap.
• Valid palindrome: Ignore non-alphanumeric, case-insensitive; converging pointers.
• Container with most water: Heights array; converging pointers (move the shorter line inward).

Summary

• Two pointers = two indices moving in one pass; often O(n) time, O(1) space.
• Converging: left at 0, right at n−1; move toward each other. Use for sorted array pair sum, palindrome, “two from ends.”
• Same direction: read and write (or fast/slow). Use for in-place remove duplicates, move zeros, compaction.
• Loop condition for “distinct pair”: left < right. Move the pointer that will fix the condition (e.g. sum too small → left++, sorted).
• Recognize “sorted + pair” and “in-place remove” as two-pointer problems to avoid O(n²) or extra space.

5.5 Sliding Window

Introduction

The sliding window technique solves problems on contiguous subarrays (or substrings) by maintaining a “window” [left, right] and moving it in one pass. Instead of checking every possible subarray (O(n²)), you expand or shrink the window based on the problem condition so each element is added and removed from the window at most twice—O(n) time. There are two main types: fixed-size window (e.g. max sum of k consecutive elements) and variable-size window (e.g. smallest subarray with sum ≥ target, or longest substring with at most k distinct characters). This section covers both patterns, when to use which, and how to keep window state (e.g. sum or frequency map) updated efficiently.

Real-World Analogy

Imagine a train car with a fixed number of seats (fixed window): as the train moves, one person gets off at the front and one gets on at the back—you always see the same number of people, but the group “slides.” For a variable window, imagine a rope you pull from both ends: you lengthen it until a condition is met (e.g. “contains at least 3 red beads”), then shorten from the left until the condition fails, then lengthen again. You never go backward on the right pointer; you only adjust left. In both cases, you avoid re-scanning the whole array by reusing what you already know about the current window.

Formal Definition

The key invariant: each element enters the window once and leaves at most once (when left advances). So the number of “add to window” and “remove from window” operations is O(n), and if each update is O(1), the whole algorithm is O(n).

Why This Topic Matters

• Interviews: Max sum subarray of size k, smallest subarray with sum ≥ target, longest substring with at most K distinct characters, minimum window substring—all classic sliding window.
• Efficiency: Turns “check every subarray” O(n²) (or O(n·k) for fixed k) into O(n) by reusing window state.
• Pattern: “Contiguous subarray/substring” + “max/min length” or “satisfy condition” often suggests sliding window.

Mental Model

Fixed-size: The window is a “frame” of k elements. Slide one step: subtract the element that just left (left), add the new element (right). Keep a running sum (or other aggregate) and update it in O(1) per slide. Variable-size: Right expands the window; when the condition is met (or violated, depending on the problem), you may record a candidate answer and then shrink from the left until the condition is no longer met, then expand again. The goal is usually “smallest window that satisfies” or “largest window that satisfies”; the expansion/shrink logic depends on that.

Fixed-Size Window

Problem: Maximum Sum of K Consecutive Elements

Given an array and integer k, find the maximum sum of any contiguous subarray of length k.

Algorithm

1. Compute the sum of the first k elements (window [0, k−1]). This is the first candidate.
2. For right from k to n−1: the new window drops arr[left] and adds arr[right], where left = right − k. So new_sum = current_sum − arr[left] + arr[right]. Update current_sum and track the maximum.
3. Return the maximum sum seen.

def max_sum_k(arr, k):
    if not arr or k <= 0 or k > len(arr):
        return 0
    window_sum = sum(arr[:k])
    best = window_sum
    for right in range(k, len(arr)):
        left = right - k
        window_sum = window_sum - arr[left] + arr[right]
        best = max(best, window_sum)
    return best

Line-by-Line Notes

• window_sum is the sum of the current window. When we slide, we subtract the element that exits (arr[left]) and add the element that enters (arr[right]).
• Loop runs (n − k) iterations; each iteration is O(1). Total O(n).

ASCII Diagram: Fixed Window

  arr: [ 2,  1,  5,  1,  3,  2 ]   k = 3
  Step 0: [ 2,  1,  5 ]            sum = 8
  Step 1:      [ 1,  5,  1 ]        sum = 7  (drop 2, add 1)
  Step 2:           [ 5,  1,  3 ]  sum = 9  (drop 1, add 3)  ← max
  Step 3:                [ 1,  3,  2 ]  sum = 6
  Result: max = 9
        

Variable-Size Window

Problem: Smallest Subarray with Sum ≥ Target

Given an array of positive integers and a target, find the length of the smallest contiguous subarray whose sum is ≥ target. If none, return 0.

Idea: Expand the window (right++) until the window sum ≥ target. Then we have a candidate length (right − left + 1). Shrink from the left (left++) until the sum is < target again; each time before shrinking, update the minimum length. Then expand again. Each element is added once and removed once—O(n).

def min_subarray_sum(arr, target):
    if not arr or target <= 0:
        return 0
    left = 0
    window_sum = 0
    min_len = float('inf')
    for right in range(len(arr)):
        window_sum += arr[right]
        while window_sum >= target:
            min_len = min(min_len, right - left + 1)
            window_sum -= arr[left]
            left += 1
    return min_len if min_len != float('inf') else 0

While the sum is ≥ target, we shrink from the left and update the minimum length. Time O(n): right and left each advance at most n times. Space O(1).

Problem: Longest Substring with At Most K Distinct Characters

Given a string (or array of characters) and k, find the length of the longest substring with at most k distinct characters. Idea: Expand (right++) and add the new character to a frequency map. While the number of distinct characters exceeds k, shrink from the left (remove arr[left] from the map, left++). After each shrink step, the window has ≤ k distinct. Track the maximum window size. Time O(n), space O(k) for the map.

def longest_k_distinct(s, k):
    if k <= 0 or not s:
        return 0
    left = 0
    freq = {}
    max_len = 0
    for right in range(len(s)):
        c = s[right]
        freq[c] = freq.get(c, 0) + 1
        while len(freq) > k:
            c_left = s[left]
            freq[c_left] -= 1
            if freq[c_left] == 0:
                del freq[c_left]
            left += 1
        max_len = max(max_len, right - left + 1)
    return max_len

Step-by-Step: Variable Window (Min Subarray Sum)

1. left = 0, window_sum = 0, min_len = ∞.
2. For right from 0 to n−1: add arr[right] to window_sum.
3. While window_sum ≥ target: update min_len = min(min_len, right − left + 1); subtract arr[left] from window_sum; left++.
4. After the loop, return min_len (or 0 if no valid window).

The “while” shrink step ensures that when we leave the inner loop, the window [left, right] has sum < target. So the next expansion (right++) is the only way to get back to ≥ target. No need to re-scan from the beginning.

Evolution: Brute Force → Sliding Window

Max sum of k consecutive: Brute force: for each starting index i, sum arr[i..i+k−1] — O(n·k). Sliding window: one pass, add one and remove one per step — O(n). Smallest subarray sum ≥ target: Brute force: for each (i, j), sum arr[i..j] and compare — O(n²). Sliding window: expand and shrink, each element processed O(1) times — O(n).

Time and Space Complexity

• Fixed-size (max sum k): One pass over the array; each element enters and leaves the window once. Time O(n), space O(1).
• Variable-size (min subarray sum): Left and right each advance at most n times; inner while runs at most n times total. Time O(n), space O(1).
• Variable-size with frequency map (k distinct): Time O(n), space O(k) for the map.

Edge Cases

• Empty array or k > n (fixed window): Return 0 or handle (e.g. no valid window).
• k = 0 or negative: No valid window; return 0 or appropriate value.
• Target unreachable (min subarray): If total sum < target, return 0 (or report no solution).
• All elements same (k distinct): One distinct character; window can be the whole array if k ≥ 1.
• Negative numbers: Min subarray sum with “sum ≥ target” can still use sliding window, but the “shrink while sum ≥ target” logic remains correct. For “max subarray sum” (Kadane), different algorithm (Topic 5.8).

Common Mistakes

• Recomputing the window from scratch each time: That gives O(n·k) or O(n²). Always update the running state (sum, frequency) in O(1) when sliding.
• Shrinking too much (variable window): Shrink only while the condition is (still) satisfied (or violated, depending on the problem). For “smallest subarray with sum ≥ target,” shrink while sum ≥ target; for “longest with ≤ k distinct,” shrink while distinct > k.
• Using “if” instead of “while” when shrinking: After expanding, you may need to shrink multiple steps (e.g. remove several elements from the left) to restore the invariant. Use while for the shrink loop.
• Off-by-one in length: Length of [left, right] inclusive is right - left + 1. Check your problem’s definition (0-based indices vs 1-based length).

Pattern Recognition

• “Contiguous subarray of size k” / “every consecutive k”: Fixed-size window.
• “Smallest/largest subarray such that sum ≥ / ≤ target”: Variable-size window; expand then shrink (or the reverse).
• “Longest substring with at most K distinct”: Variable window + frequency map.
• “Minimum window substring” (containing all chars of T): Variable window + frequency map for T and current window.

Practice Problems

• Max sum of k consecutive elements: Fixed window; running sum.
• Smallest subarray with sum ≥ target: Variable window; expand, then shrink while sum ≥ target.
• Longest substring with at most K distinct characters: Variable window + freq map; shrink while distinct > k.
• Maximum average subarray of length k: Same as max sum of k (fixed window); average = sum/k.
• Minimum window substring: Smallest substring of s containing all characters of t; variable window + two frequency maps.

Summary

• Sliding window = contiguous subarray [left, right] with state updated in O(1) when adding/removing one element; total time O(n).
• Fixed-size: Window length k; slide by subtracting arr[left] and adding arr[right]; one pass O(n).
• Variable-size: Expand (right++) until condition met; shrink (left++) with a while until condition fails; track min/max length (or other measure).
• Use while (not if) when shrinking so the invariant is restored after multiple removals.
• Recognize “contiguous subarray” + “max/min sum or length” or “at most k distinct” as sliding window to get O(n) instead of O(n²) or O(n·k).