065
LVL 03 — MID DEVELOPERSESSION 065DAY 65

PAGINATION & INDEXING

🎫 PIXELCRAFT-052
Performance | 🟠 Hard | Priority: 🔴 Critical

The gallery has 10,000 test images. Loading /api/images returns ALL of them — 8 second response. Search scans the entire collection. Add pagination and database indexing.
CONCEPTS.UNLOCKED
📇
Database Indexes
Making queries fast: from O(n) full scan to O(log n) index lookup. Like a book index — instead of reading every page, jump directly to the right one.
🌳
B-Trees
The data structure behind database indexes. Balanced tree where each node has hundreds of children. 10 million records → only 3-4 levels deep. 3-4 disk reads to find any record.
📄
Pagination
limit and skip, or cursor-based. Return 20 items at a time, not 10,000. The client requests pages as needed. Dramatically less data over the wire.
🎯
Query Optimization
Projection: only fetch needed fields (.select()). Sorting at DB level: don't sort in JavaScript. Let the database do the work — it has indexes for this.
🔍
explain()
See how MongoDB executes your query. Documents examined, index used, execution time. Before index: 10,000 docs scanned. After: 20 docs via index scan.
Infinite Scroll
useInfiniteQuery from React Query. Load 20 images, scroll to bottom, load 20 more. Seamless experience without manual "page 2, page 3" buttons.
HANDS-ON.TASKS
01
Measure Before Optimization
// This scans ALL 10,000 documents const images = await Image.find({ owner: userId }); // Response time: ~800ms
02
Add Pagination
app.get('/api/images', authenticate, async (req, res) => { const page = parseInt(req.query.page) || 1; const limit = parseInt(req.query.limit) || 20; const skip = (page - 1) * limit; const sort = req.query.sort || '-createdAt'; const [images, total] = await Promise.all([ Image.find({ owner: req.userId }) .select('name thumbnail width ' + 'height format createdAt') .sort(sort) .skip(skip) .limit(limit), Image.countDocuments({ owner: req.userId }), ]); res.json({ images, pagination: { page, limit, total, totalPages: Math.ceil(total / limit), hasMore: page * limit < total, }, }); });
03
Create Indexes
// Compound index imageSchema.index({ owner: 1, createdAt: -1 }); // Text search index imageSchema.index({ name: 'text', tags: 'text' });
04
Measure After & Use explain()
// Before index: 800ms for 10,000 docs // After index: 2ms for 20 docs (paginated) const explanation = await Image.find({ owner: userId }).explain('executionStats'); console.log('Docs examined:', explanation.executionStats .totalDocsExamined); // Before index: 10000 (full scan) // After index: 20 (index scan)
ONE index made the database 400× faster. Zero code changes to application logic — just smarter data organization.
05
Frontend: Infinite Scroll

Implement infinite scroll with React Query's useInfiniteQuery. Load 20 images, scroll to bottom, automatically load the next page.

06
Close the Ticket
git switch -c perf/PIXELCRAFT-052-pagination-indexes git add server/ src/ git commit -m "Add pagination + DB indexes — 400x faster (PIXELCRAFT-052)" git push origin perf/PIXELCRAFT-052-pagination-indexes # PR → Review → Merge → Close ticket ✅
CS.DEEP-DIVE

B-trees are the data structure behind virtually all database indexes.

A B-tree indexing 10 million records has a depth of only 3-4 levels.

// B-tree: each node has hundreds of
// children → incredibly shallow depth

10M records → 3-4 disk reads
  (instead of scanning 10 million)

// The same principle applies to:

File systems    → finding files
Search engines  → finding documents
Library catalogs → finding books

// Understanding indexing is understanding
// why databases can be fast — and why
// they're slow when indexes are missing.
"Performance Lab"
[A]Seed the database with 50,000 images. Benchmark pagination with and without indexes. Graph the response times. Present the business case for indexing.
[B]Implement cursor-based pagination (using _id or createdAt as cursor instead of skip/limit). Compare performance at page 1 vs page 500. Why is cursor-based faster for deep pages?
[C]Add full-text search using the text index: search images by name or tags. Build a search bar in the gallery that queries the API with debounce (from Session 44's useDebounce hook).
REF.MATERIAL
ARTICLE
MongoDB
Complete indexing guide: single field, compound, text, geospatial, and how indexes improve query performance.
INDEXESOFFICIALESSENTIAL
VIDEO
Hussein Nasser
Visual B-tree explanation: how indexes work internally, when they help, when they hurt, and how to choose which fields to index.
B-TREEVISUAL
ARTICLE
Wikipedia
The data structure: self-balancing tree, multi-way branching, disk-optimized. The CS theory powering every database index.
B-TREETHEORYCS
ARTICLE
MongoDB
Query analysis: execution stats, index usage, documents examined. The essential tool for understanding and optimizing query performance.
EXPLAINPROFILINGOFFICIAL
VIDEO
Fireship
Offset vs cursor pagination: tradeoffs, implementation, and when each approach is appropriate.
PAGINATIONCOMPARISON
// LEAVE EXCITED BECAUSE
The gallery went from 8 seconds to under 100ms. ONE index made the database 400× faster. Zero code changes — just smarter data organization. This is the real-world impact of CS theory.