Deep Dive into Assembly

⚡ Direct Machine Mastery: At the lowest software abstraction layer, Assembly Language (ASM) provides a direct 1:1 human-readable mnemonic representation of binary machine instructions executed by the CPU. Understanding registers, stack frame allocation, addressing modes, and calling conventions is the ultimate foundation for compiler engineering, kernel programming, reverse engineering, and extreme high-performance computing.

1. The Execution Model: Silicon, Opcodes & Mnemonics

When a compiler processes high-level code, it emits Assembly instructions. An Assembler (such as NASM, GAS, or LLVM) converts these mnemonics directly into binary Opcodes (machine bytes):

+-----------------------------------------------------------------------------------+
|                           THE INSTRUCTION TRANSLATION CHAIN                       |
+-----------------------------------------------------------------------------------+
  C / C++ Source:     int add(int a, int b) { return a + b; }
                                    │
                                    v (Compiler: gcc -S)
  Assembly (x86-64):  lea eax, [rdi + rsi]
                      ret
                                    │
                                    v (Assembler: nasm / as)
  Machine Code (Hex): 8D 04 37 C3   (Opcodes stored in .text memory segment)
                                    │
                                    v (Hardware Execution)
  CPU Execution:      Instruction Decoder ──► ALU Execution Unit ──► Register Writeback
+-----------------------------------------------------------------------------------+

2. x86-64 CPU Architecture & Register Hierarchy

The x86-64 architecture features sixteen 64-bit General-Purpose Registers. Each register can be accessed in 32-bit, 16-bit, and 8-bit slices for backward compatibility:

+-----------------------------------------------------------------------------------+
|                        x86-64 REGISTER SLICE PARTITIONING (RAX)                   |
+-----------------------------------------------------------------------------------+
  63                               31               15       7        0
  +────────────────────────────────+────────────────+────────+────────+
  |              RAX (Full 64-Bit Quadword Register)                  |
  +────────────────────────────────+────────────────+────────+────────+
                                   |    EAX (Lower 32-Bit Doubleword) |
                                   +────────────────+────────+────────+
                                                    |  AX (16-Bit Word)|
                                                    +────────+────────+
                                                    | AH (8) | AL (8) |
                                                    +────────+────────+
+-----------------------------------------------------------------------------------+

Core Register Roles (System V AMD64 ABI)

• RAX (Accumulator): Holds function return values and primary math results.
• RDI, RSI, RDX, RCX, R8, R9: Used to pass the first 6 integer/pointer arguments to functions.
• RSP (Stack Pointer): Always points to the current top address of the active execution stack.
• RBP (Base / Frame Pointer): Points to the base of the current function’s stack frame.
• RIP (Instruction Pointer / Program Counter): Stores the memory address of the next instruction to be executed.
• RFLAGS (Status Register): Stores conditional status flags:
  • ZF (Zero Flag): Set to 1 if the result of an arithmetic operation is zero.
  • SF (Sign Flag): Set to 1 if the result is negative.
  • CF (Carry Flag): Set to 1 if unsigned math overflowed.
  • OF (Overflow Flag): Set to 1 if signed math overflowed.

3. Intel vs AT&T Syntax

The industry uses two primary syntax formats:

4. Complex Memory Addressing Modes

The x86-64 architecture provides the Base-Index-Scale-Displacement addressing formula, calculating array offsets in hardware within a single instruction cycle:

Effective Address = Base + (Index * Scale) + Displacement

Where Scale ∈ {1, 2, 4, 8} (matching primitive data type byte sizes: byte, word, dword, qword).

; Load the 5th element of an array of 64-bit integers stored at [rbp - 64]
mov rbx, 4                             ; Index = 4 (0-indexed 5th item)
mov rax, [rbp - 64 + rbx * 8]         ; Base: rbp, Disp: -64, Index: rbx, Scale: 8

5. Function Calls & Stack Frames (System V ABI)

When a function executes, it sets up a Stack Frame to manage local variables and register preservation:

+-----------------------------------------------------------------------------------+
|                        FUNCTION CALL STACK FRAME LIFECYCLE                        |
+-----------------------------------------------------------------------------------+
  Higher Memory Address
  ─────────────────────────────────────────────────────────────────────────────────
  [ Caller's Stack Frame ]
  [ Return Address (Pushed automatically by 'call' instruction) ]
  ─────────────────────────────────────────────────────────────────────────────────
  [ Saved RBP (Previous Frame Pointer pushed by 'push rbp') ]   ◄─── RBP Points Here
  [ Local Variable 1: e.g. [rbp - 8]  ]
  [ Local Variable 2: e.g. [rbp - 16] ]
  ─────────────────────────────────────────────────────────────────────────────────
  [ Current Top of Stack ]                                      ◄─── RSP Points Here
  Lower Memory Address
+-----------------------------------------------------------------------------------+

Complete x86-64 NASM Implementation: Recursive Factorial

; factorial.s (x86-64 NASM)
; Signature: uint64_t factorial(uint64_t n);
; Input:  rdi = n
; Output: rax = n!

global factorial

section .text

factorial:
    ; 1. FUNCTION PROLOGUE
    push rbp                ; Save old base pointer
    mov  rbp, rsp           ; Set up new stack frame base
    sub  rsp, 16            ; Allocate 16 bytes for local storage

    ; 2. BASE CASE CHECK (n <= 1)
    cmp  rdi, 1
    jle  .base_case         ; If n <= 1, jump to base case

    ; 3. RECURSIVE STEP
    mov  [rbp - 8], rdi     ; Save current 'n' on stack
    dec  rdi                ; Argument for recursive call: n - 1
    call factorial          ; Returns (n - 1)! in RAX

    mov  rsi, [rbp - 8]     ; Restore original 'n'
    imul rax, rsi           ; rax = (n - 1)! * n
    jmp  .epilogue

.base_case:
    mov  rax, 1             ; Return 1 for 0! and 1!

.epilogue:
    ; 4. FUNCTION EPILOGUE
    mov  rsp, rbp           ; Deallocate stack frame
    pop  rbp                ; Restore caller's base pointer
    ret                     ; Return to caller (pops return address into RIP)

6. SIMD & Vectorization (AVX2 / AVX-512)

Modern CPUs achieve massive parallel throughput using Single Instruction, Multiple Data (SIMD) vector registers:

+-----------------------------------------------------------------------------------+
|                        AVX2 256-BIT VECTOR REGISTER (YMM0)                        |
+-----------------------------------------------------------------------------------+
  Processes EIGHT 32-bit single-precision floats simultaneously in ONE instruction:
  
  [ Float 7 | Float 6 | Float 5 | Float 4 | Float 3 | Float 2 | Float 1 | Float 0 ]
      +         +         +         +         +         +         +         +
  [ Float 7 | Float 6 | Float 5 | Float 4 | Float 3 | Float 2 | Float 1 | Float 0 ]
      =         =         =         =         =         =         =         =
  [ Result 7| Result 6| Result 5| Result 4| Result 3| Result 2| Result 1| Result 0]
+-----------------------------------------------------------------------------------+

; Add 8 floats in parallel using AVX
vmovups ymm0, [rdi]        ; Load 8 floats from array A
vmovups ymm1, [rsi]        ; Load 8 floats from array B
vaddps  ymm2, ymm0, ymm1   ; ymm2 = ymm0 + ymm1 (8 parallel additions!)
vmovups [rdx], ymm2        ; Store results back to memory

7. Summary & Quick Reference

# 🛠️ Assemble, Link, and Debug
nasm -f elf64 factorial.s -o factorial.o    # Assemble NASM into 64-bit object file
gcc -no-pie main.c factorial.o -o program  # Link with C runtime
gdb ./program                              # Step-by-step register & memory disassembly
objdump -d -M intel program                # Disassemble binary in Intel syntax

Assembly reveals the pure, unadorned physics of computing—where every byte, register, and CPU clock cycle is under your direct command.