Linux Memory Management

Linux memory management is divided into architecture-specific early memory initialization and an architecture-independent high-level memory manager (mm).

On x86 systems, the CPU starts in real mode, where memory addressing uses a segment:offset scheme and is limited to 1 MB. During early boot, the kernel transitions the CPU into protected mode from real mode, which enables paging, and establishes a flat virtual address space.

In protected mode, memory is managed using paging, where physical memory is divided into fixed-size page frames (typically 4 KB on x86). Virtual addresses are translated to physical addresses via page tables by mapping virtual pages to physical frames.

The bootable kernel image (bzImage) contains compressed kernel code (vmlinux) along with bootstrap code responsible for early hardware initialization, enabling protected mode and paging.

The architecture-specific memory initialization code:

• Creates a struct page instance for each physical page frame

• Organizes memory into zones (e.g., DMA, Normal, HighMem on 32-bit x86)

• Initializes data structures required by the high-level memory manager

Zone layout and limits are architecture dependent and differ between 32-bit and 64-bit systems.

Real mode: Physical address is the offset address

Protected mode: Physical address = Frame number + Offset (The physical address format is changed from Offset to frame number + offset)

Responsibilities of Low level Memory initializer:

- Allocate a page list with instances of struct page representing each physical frame
- Categorize pages into appropriate zones
- Zone DMA: category of DMA zone from 0 - 16 MB
- Zone Normal: 16 - 896 MB
- Zone High: > 896 MB
- Initialize high level Memory Manger

1. Big Picture: Linux Memory Management Stack

+--------------------------------------------------+
|                Kernel Subsystems                 |
|   (Drivers, Filesystems, Networking, VM, etc.)  |
+----------------------+---------------------------+
                       |
                       v
+--------------------------------------------------+
|            Slab / SLUB Allocator Layer           |
|   - kmalloc()                                   |
|   - kmem_cache_alloc()                          |
|   - Object caches (fixed-size objects)          |
+----------------------+---------------------------+
                       |
                       v
+--------------------------------------------------+
|             Page Allocator (Buddy)               |
|   - Allocates 2^n contiguous pages               |
|   - Primary allocator                            |
+----------------------+---------------------------+
                       |
                       v
+--------------------------------------------------+
|          Physical Memory (Page Frames)           |
|   - struct page per frame                        |
|   - Organized into zones                         |
+--------------------------------------------------+

2. Early Boot vs Runtime Memory Management

Boot-Time Memory Allocation

System Boot
   |
   v
+------------------------------+
|  Boot Memory Allocator       |
|  (memblock)                  |
|  - Variable-size allocations |
|  - Used before paging        |
+------------------------------+
   |
   v
+------------------------------+
|  Page Allocator Initialized  |
+------------------------------+

• memblock is used only during early boot.

• Once the page allocator is ready, memblock is mostly retired.

3. Page Allocator (Primary Allocator)

Virtual Request: alloc_pages(order = N)
                  |
                  v
+-------------------------------------+
|        Buddy Allocator               |
|  - Manages free lists by order       |
|  - order 0 = 1 page                  |
|  - order 1 = 2 pages                 |
|  - order 2 = 4 pages                 |
+-------------------------------------+
                  |
                  v
+-------------------------------------+
|     Contiguous Physical Pages        |
+-------------------------------------+

Properties:

• Allocates page-sized or power-of-two page blocks

• All higher-level allocators depend on this

• Handles external fragmentation via buddy merging

4. Slab / SLUB Allocator (Object Allocation)

Kernel Request: kmalloc(100 bytes)
        |
        v
+----------------------------------+
|     Slab Cache Selection          |
|     (e.g., kmalloc-128)           |
+----------------------------------+
        |
        v
+----------------------------------+
|   Slab Cache                      |
|   - Objects of same size          |
|   - Backed by one or more pages   |
+----------------------------------+
        |
        v
+----------------------------------+
|     Page Allocator                |
+----------------------------------+

Key points:

• Slab allocators do not allocate arbitrary sizes

• Requests are rounded to predefined cache sizes

• Reduces:

  • Internal fragmentation

  • Allocation overhead

5. Memory Zones (Physical Memory Layout)

Physical Address Space
+-------------------------------+
| ZONE_DMA                      |
| (Legacy DMA devices)          |
+-------------------------------+
| ZONE_NORMAL                   |
| (Directly mapped memory)      |
+-------------------------------+
| ZONE_HIGHMEM (32-bit only)    |
| (Not permanently mapped)     |
+-------------------------------+

On 64-bit systems, ZONE_HIGHMEM usually does not exist.

Summary

Early Boot:
[memblock] --> Page Allocator Ready

Runtime:
Kernel --> Slab/SLUB --> Page Allocator --> Physical Memory

Critical Paths:
Kernel --> mempool --> Slab/Page Allocator