SESSION 049 — Batch Processing

CONCEPTS.UNLOCKED

📋

Queue Data Structure

FIFO — First In, First Out. Like a line at a store: first person in line is first served. Items enter at the back and leave from the front.

⚙

Queue Operations

enqueue (add to back), dequeue (remove from front), peek (look at front without removing), isEmpty, size. Simple interface, powerful applications.

⚖

Queue vs Stack

Queue: FIFO — fair, ordered processing. Stack: LIFO — last in, first out (Session 34's undo). Different access patterns, different data structures.

⭐

Priority Queues

Not all tasks are equal. A priority queue processes highest-priority items first, regardless of arrival order. Used in OS task scheduling, network routing, and emergency rooms.

📊

Task Scheduling

Process tasks one by one with progress tracking: completed count, total count, percentage. Report progress after each task so the UI stays responsive.

🚫

Cancelable Operations

A #cancelled flag checked between tasks. The loop stops if cancelled, returns partial results. The user stays in control — no frozen tab, no lost work.

HANDS-ON.TASKS

Implement a TaskQueue Class

▶

class TaskQueue {
  #items = [];
  #processing = false;
  #cancelled = false;

  enqueue(task) { this.#items.push(task); }
  dequeue() { return this.#items.shift(); }
  peek() { return this.#items[0]; }
  isEmpty() { return this.#items.length === 0; }
  size() { return this.#items.length; }
  cancel() { this.#cancelled = true; }

  async processAll(onProgress) {
    this.#processing = true;
    this.#cancelled = false;
    const total = this.size();
    let completed = 0;

    while (!this.isEmpty() && !this.#cancelled) {
      const task = this.dequeue();
      await task();
      completed++;
      onProgress({
        completed, total,
        percent: Math.round(
          (completed / total) * 100
        )
      });
    }

    this.#processing = false;
    return {
      completed, total,
      cancelled: this.#cancelled
    };
  }
}

Private fields (#items, #processing, #cancelled) from Session 34. The processAll method dequeues and executes tasks one by one, reporting progress after each. The #cancelled flag is checked between tasks — cooperative cancellation.

Build the Batch Processor

▶

const queue = new TaskQueue();

// Add 20 images to the queue
selectedImages.forEach(image => {
  queue.enqueue(async () => {
    const processed = await applyFilterToImage(
      image, currentFilter
    );
    await saveProcessedImage(processed);
  });
});

// Process with progress
const result = await queue.processAll(
  ({ completed, total, percent }) => {
    updateProgressBar(percent);
    updateStatusText(
      `Processing ${completed}/${total}...`
    );
  }
);

Each image gets its own task function enqueued. processAll works through them sequentially, calling onProgress after each. The UI stays responsive because each await yields back to the event loop.

Build the BatchProcessor Component

▶

function BatchProcessor({ images, filter }) {
  const [progress, setProgress] = useState({
    percent: 0, completed: 0, total: 0
  });
  const [processing, setProcessing] = useState(false);
  const queueRef = useRef(null);

  const start = async () => {
    setProcessing(true);
    queueRef.current = new TaskQueue();
    images.forEach(img =>
      queueRef.current.enqueue(
        () => processImage(img, filter)
      )
    );
    await queueRef.current.processAll(setProgress);
    setProcessing(false);
  };

  const cancel = () => queueRef.current?.cancel();

  return (
    <div>
      <ProgressBar percent={progress.percent} />
      <p>{progress.completed}/{progress.total}
        images processed</p>
      {processing ? (
        <button onClick={cancel}>Cancel</button>
      ) : (
        <button onClick={start}>
          Process All
        </button>
      )}
    </div>
  );
}

useRef stores the queue instance (survives re-renders without triggering them). useState tracks progress for the UI. The cancel button calls queue.cancel() — cooperative cancellation between tasks.

Test: Queue, Progress, Cancel

▶

Queue 20 images, watch the progress bar fill, cancel midway, verify:

Test	Expected
Start processing	Progress bar fills 5%, 10%, 15%...
Cancel at 50%	Stops after current image, returns {completed: 10, cancelled: true}
Partial results	10 processed images saved, 10 unprocessed
Restart	New queue, fresh progress, starts from beginning

Close the Ticket

▶

git switch -c feature/PIXELCRAFT-037-batch-processing
git add src/
git commit -m "Add batch processing with queue, progress, cancel (PIXELCRAFT-037)"
git push origin feature/PIXELCRAFT-037-batch-processing
# PR → Review → Merge → Close ticket ✅

REF.MATERIAL

VIDEO

Queues — CS50

CS50 / Harvard

Clear visual explanation: FIFO ordering, enqueue, dequeue, and the differences between queues and stacks.

QUEUECS50ESSENTIAL

VIDEO

Stacks and Queues in 100 Seconds — Fireship

Fireship

Rapid overview: LIFO vs FIFO, push/pop/enqueue/dequeue, real-world uses, and when to choose which.

QUEUESTACKQUICK

ARTICLE

Queue — Wikipedia

Wikipedia

Theory: FIFO semantics, implementations (array, linked list, circular buffer), priority queues, and applications in OS and networking.

QUEUETHEORYCS

ARTICLE

Array.shift() — MDN

Mozilla Developer Network

The dequeue operation in JavaScript: removes and returns the first element. Understand the O(n) cost and when it matters.

ARRAYMDNPERFORMANCE

ARTICLE

Message Queue — Wikipedia

Wikipedia

How queues scale to production: RabbitMQ, Kafka, SQS. The same FIFO pattern from your batch processor, handling millions of events at global scale.

MESSAGE QUEUEARCHITECTURESCALE

BATCH PROCESSING