Chapter 1: Course Introduction and HPC Overview¶

Course: Fundamentals of High-Performance Computing for CFD Simulations
Lecturer: Jun.-Prof. Dr.-Ing. Federica Ferraro (IFAS, Working Group: Reactive Flows in Aero Engines)
Source slides: L01_Introduction_and_Linux.pdf (slides 1 ~ 35), read.pdf slides 1 ~ 22

1. Chapter Overview¶

This first chapter is the conceptual foundation of the entire course. It tells you why an engineer who works on CFD (Computational Fluid Dynamics) must master Linux, scripting, compilers, and parallel computing.

What it is about:

Who teaches the course (IFAS — Institute of Jet Propulsion and Turbomachinery, Leibniz Universität Hannover).
The research goal: low-emission, energy-efficient aircraft propulsion, simulated with HPC.
Why HPC (High-Performance Computing) is essential for CFD.
The role of supercomputers (Top500, El Capitan, Frontier, JUWELS).
The organisation of the course (block course, dates, hands-on sessions).
A first encounter with Unix/Linux and the reason every engineering simulation environment uses Linux.

Why it matters in HPC and CFD:

Real CFD simulations (e.g. of a combustion chamber in a gas turbine) generate terabytes of data and need millions of CPU-core-hours. Without parallel computing on a cluster you simply cannot finish them.
Every HPC cluster on earth runs Linux. There is no Windows supercomputer in the Top500 list.
Mastering the workflow (Linux, SSH, shell scripting, compiling, debugging, plotting, version control) is a prerequisite for doing CFD research at all.

What the examiner may ask:

"Define HPC. Why is HPC necessary for CFD?"
"Name three reasons why Linux dominates supercomputing."
"What is the Top500 list?"
"Describe the difference between experiments and virtual experiments in turbomachinery research."
Short factual / definition questions (3–5 marks).

What you must master for top grade:

A clean two-line definition of HPC that you can write under exam pressure.
One concrete CFD example (gas turbine combustion chamber) you can describe in 4–5 sentences.
The four "supercomputer facts": parallelism, FLOPS unit, Top500, Linux dominance.
The four reasons Linux is preferred for HPC: stability, scriptability, free/open, modular.

2. Basics from Zero¶

Imagine you want to simulate the airflow inside a jet engine. The engine is so complicated that the equations describing the gas (Navier–Stokes) cannot be solved by hand. Instead, you split the engine into millions of tiny cells, write the equations for each cell, and solve all of them at the same time on a computer.

A normal laptop has 4–8 cores. With 8 cores the simulation might take 6 months. A modern supercomputer has millions of cores and can do the same simulation in a few hours. That ability — to run a single huge calculation across many cores at the same time — is what we call High-Performance Computing (HPC).

বাংলায়: HPC-র মূল ধারণা খুবই সহজ: একটা বিশাল হিসাবকে ছোট ছোট টুকরোয় ভেঙে হাজার হাজার core-এ একসাথে চালানো। Laptop-এ যে simulation ৬ মাস লাগত, supercomputer-এ তা কয়েক ঘণ্টায় শেষ হয়, কারণ কাজটা parallel-এ ভাগ হয়ে যায়। পরীক্ষায় দুই লাইনের definition চাওয়া হয় — "many computers working on one calculation" — এটা মুখস্থ রাখুন।

The supercomputer is not magic: it is just a lot of normal computers ("nodes") connected by a fast network. To use it you must connect remotely (SSH), copy your input files (scp/rsync), compile your code (g++/icc/mpicc), run it as a batch job (SLURM), wait, and then post-process the results (gnuplot, ParaView).

Every step of that workflow happens inside a Linux terminal. That is why this entire course teaches Linux first — it is the language of HPC.

বাংলায়: Cluster-এ কাজ করার পুরো রাস্তাটা মনে রাখুন: SSH দিয়ে login, scp দিয়ে file পাঠানো, compile করা, SLURM-এ job জমা দেওয়া, শেষে gnuplot দিয়ে ফলাফল আঁকা — প্রতিটা ধাপই Linux terminal-এ ঘটে। সেজন্যই Linux-কে HPC-র ভাষা বলা হয়; terminal না জানলে supercomputer ব্যবহার করাই অসম্ভব।

Real-life analogy. A normal computer is one chef cooking one meal. A supercomputer is a stadium full of 10 000 chefs all cooking together — but only if there is a head-cook (you, the user) who knows how to give commands so they don't all collide. Linux + the shell is that command system.

What happens if you misunderstand? You write a serial code that runs only on one core, queue it on a 1024-core node, the cluster's accounting bills you for 1024 cores × 24 h = 24 576 core-hours wasted. Real money, real CO₂.

3. Hard English Made Easy¶

Hard Term	Simple English	বাংলা	Example
High-Performance Computing (HPC)	Using many computers together to solve big problems fast	অনেক কম্পিউটার একসাথে কাজ করিয়ে বড় সমস্যা দ্রুত সমাধান করা	Frontier solves climate models in hours
Computational Fluid Dynamics (CFD)	Solving fluid-flow equations on a computer	কম্পিউটারে তরল পদার্থের প্রবাহ সমাধান করা	Simulating air around a wing
Parallel computing	Doing many calculations at the same time	একই সময়ে অনেকগুলি হিসাব করা	1000 cores compute 1000 cells simultaneously
Supercomputer	A very large parallel computer	বিশাল সমান্তরাল কম্পিউটার	El Capitan, Frontier, JUWELS
Top500	A list of the 500 fastest supercomputers	বিশ্বের ৫০০ দ্রুততম সুপারকম্পিউটারের তালিকা	top500.org
FLOPS	Floating-point operations per second (a measure of speed)	প্রতি সেকেন্ডে দশমিক হিসাব সংখ্যা	1 ExaFLOPS = $10^{18}$ ops/s
Cluster	A group of connected computers	একসাথে যুক্ত অনেক কম্পিউটার	A cluster has many nodes
Node	One physical computer inside a cluster	ক্লাস্টারের একটি কম্পিউটার	A node has 64 cores
Core	One processing unit inside a CPU	প্রসেসরের একটি ছোট অংশ	16 cores per CPU
Virtual experiment	A simulation that replaces a physical test	কম্পিউটারে চালানো ভার্চুয়াল পরীক্ষা	Simulating a combustor without building it
SAF	Sustainable Aviation Fuel (drop-in green fuel)	পরিবেশ বান্ধব বিমান জ্বালানি	Used to lower CO₂ emissions
Sustainable aviation	Aviation with low CO₂ impact	পরিবেশসম্মত বিমান চলাচল	Cluster of Excellence SE²A

4. Deep Theory Explanation¶

4.1 What is HPC?¶

Formal definition. HPC is the practice of aggregating computing power so that the resulting performance is much higher than a typical desktop, in order to solve problems that require very large datasets, very many calculations, or very small time-steps.

Intuition. "Many computers, one calculation." Instead of 1 CPU running for 1 year, you let 8 760 CPUs run for 1 hour.

How it works internally. A supercomputer is built from compute nodes (each like a powerful workstation) connected by a high-speed interconnect (e.g. InfiniBand, Slingshot-11). A job scheduler (SLURM) gives the user a slice of the machine. The user's program is split — through MPI (between nodes) and OpenMP (inside one node) — across thousands of cores. Specialised hardware (GPUs, AMD MI300A, NVIDIA H100) accelerates each step.

Why useful. It is the only way to do realistic CFD: detailed turbulence in a gas turbine combustor needs $\approx 10^{9}$ grid cells $\times\, 10^{5}$ time steps. No laptop can finish that.

Assumptions / limitations. HPC requires that the algorithm can be parallelised. Some algorithms cannot (Amdahl's law). Communication between nodes can become the bottleneck.

Common mistakes. Confusing "parallel" with "fast". Adding more cores does not always mean faster — overheads grow.

Written-exam relevance. Definition + 1 example + 1 limitation. Standard 4-mark answer.

বাংলায়: HPC মানে অনেক node-কে দ্রুত interconnect দিয়ে জুড়ে একটাই প্রোগ্রাম MPI (node-গুলোর মাঝে) আর OpenMP (একটা node-এর ভেতরে) দিয়ে হাজারো core-এ ভাগ করে চালানো। সাবধান: parallel মানেই দ্রুত নয় — Amdahl's law অনুযায়ী code-এর serial অংশই speed-up-এর সীমা ঠিক করে দেয়। পরীক্ষায় definition + একটা example + একটা limitation লিখলেই পুরো নম্বর।

4.2 Why CFD needs HPC¶

Navier–Stokes equations have no analytical solution for turbulent flow. Discretisation (Finite-Volume Method) on a fine 3-D mesh produces a sparse linear system with millions of unknowns per time step. A Direct Numerical Simulation (DNS) of a small combustor needs $\approx 10^{15}$ floating-point operations.

At a modern node speed of ≈ 1 TFLOPS that is $10^{15} / 10^{12}$ s $\approx 10^{3}$ s $\approx$ 17 minutes — but only if all cores are fully busy and communication is free. Real life: factor 10–100 slower. Hence: use thousands of nodes.

বাংলায়: Turbulent flow-এর জন্য Navier–Stokes equation-এর কোনো analytical সমাধান নেই, তাই mesh-এর কোটি কোটি cell-এ numerically হিসাব করতে হয়। একটা ছোট combustor-এর DNS-তেই প্রায় $10^{15}$ operation লাগে — কোনো একক workstation-এ এটা অসম্ভব। এই সংখ্যাটাই "Why does CFD need HPC?" প্রশ্নের উত্তরের মেরুদণ্ড।

4.3 The Top500 list¶

A semi-annual ranking of the world's 500 fastest publicly-known computers, measured by the HPL (LINPACK) benchmark in FLOPS. As of mid-2024:

1: El Capitan (LLNL, USA) — HPE Cray EX255a, AMD 4th-Gen EPYC + AMD MI300A, ~1.7 ExaFLOPS Rmax.¶
2: Frontier (Oak Ridge, USA).¶
3: Aurora (Argonne, USA).¶
All ≥ #1 run Linux.

বাংলায়: Top500 হলো বছরে দু'বার (জুন ও নভেম্বর) প্রকাশিত দ্রুততম ৫০০ supercomputer-এর তালিকা, যেখানে গতি মাপা হয় LINPACK (HPL) benchmark-এর $R_{max}$ দিয়ে। দুটো তথ্য short question-এ বারবার আসে: শীর্ষ মেশিন El Capitan প্রায় 1.7 EFLOPS, আর তালিকার ১০০% মেশিনই Linux চালায়।

4.4 Linux dominance in HPC¶

Reason	Why it matters
Open-source	Vendors can patch the kernel for new hardware.
Stable	Cluster nodes run unattended for months.
Modular	You install only what you need on each node.
Scriptable	Job-control / automation via shell scripts.
POSIX-compliant	Portable software across distributions.

বাংলায়: Linux কেন HPC-তে একচেটিয়া — চারটে কারণ মুখস্থ করুন: open-source (vendor কার্নেল বদলাতে পারে), stable (মাসের পর মাস reboot ছাড়া চলে), modular (node-এ শুধু দরকারি অংশ থাকে), scriptable (সব কাজ shell দিয়ে automate করা যায়)। "Name three reasons" ধরনের প্রশ্নে এই তালিকা থেকেই তিনটে লিখবেন।

4.5 The course structure¶

Block course May ~ mid-July, Tue & Thu 12:00–15:00, lecture + hands-on. Topics in order: Linux → SSH/bashrc → Vim → Regex → text tools → shell scripting → gnuplot → Git → C++ → build process → debugging → parallelisation → cluster usage. Exam at end of July.

4.6 Diagram interpretations from L01¶

Slide "Experiments vs Virtual experiments": two photographs side-by-side (a flame in a test rig, a coloured CFD field). Left = costly, ground-truth, limited points. Right = simulation, complementary to experiments, cheap to repeat. In a written answer: describe the trade-off — experiments give truth but few quantities; simulations give all quantities but need validation by experiments.
"Top500 (June 2023)" pie chart: market shares of architectures (AMD, Intel, IBM POWER, ARM). Take-away to write: "Modern supercomputers are predominantly heterogeneous (CPU + GPU/accelerator)."
"Course structure" calendar: lectures and hands-on alternate, exam in last week of July.

4.7 Theoretical peak performance $R_{peak}$ — the formula you must compute with¶

Every HPC machine has a theoretical peak performance: the number of floating-point operations per second it would deliver if every arithmetic unit were busy in every single clock cycle.

\[R_{peak} = N_{\text{nodes}} \times N_{\text{sockets/node}} \times N_{\text{cores/socket}} \times f_{\text{clock}} \times \frac{\text{FLOP}}{\text{cycle}}\]

The last factor is fixed by the vector (SIMD) hardware of one core:

\[\frac{\text{FLOP}}{\text{cycle}} = \underbrace{N_{\text{SIMD lanes}}}_{\text{vector width}} \times \underbrace{2}_{\text{FMA: } a\cdot b + c} \times N_{\text{FMA units}}\]

Vector ISA	Register width	DP lanes	FMA units	DP FLOP/cycle
SSE2	128 bit	2	1	4
AVX2 + FMA	256 bit	4	2	16
AVX-512 (2 FMA units)	512 bit	8	2	32

Fully worked example (write it step-by-step exactly like this in the exam). A node has 2 sockets × 24 cores at $f_{\text{clock}} = 2.4$ GHz, with AVX-512 and two FMA units per core.

FLOP per cycle per core (double precision): $8 \times 2 \times 2 = 32$.
Cores per node: $2 \times 24 = 48$.
Cycles per second of the whole node: $48 \times 2.4 \times 10^{9} = 1.152 \times 10^{11}$.
Peak per node (DP): $1.152 \times 10^{11} \times 32 = 3.686 \times 10^{12} \approx 3.69$ TFLOP/s.
Single precision has twice the lanes (64 FLOP/cycle): $2 \times 3.69 \approx 7.37$ TFLOP/s (SP) per node.
Scale to a 100-node cluster: $100 \times 3.69 = 368.6$ TFLOP/s $\approx 0.37$ PFLOP/s (DP).

Where these cores physically live:

┌─────────────────────────── CLUSTER ────────────────────────────┐
│ ≈ 100 nodes, joined by a fast interconnect (InfiniBand)        │
│                                                                │
│  ┌──────────────────────── RACK ──────────────────────────┐    │
│  │ ≈ 20 nodes per cabinet (5 racks in this example)       │    │
│  │                                                        │    │
│  │   ┌────────────────────── NODE ────────────────────┐   │    │
│  │   │ one "computer": 2 sockets + RAM + network card │   │    │
│  │   │                                                │   │    │
│  │   │   ┌───────────── SOCKET (CPU) ─────────────┐   │   │    │
│  │   │   │ 24 cores sharing one L3 cache          │   │   │    │
│  │   │   │                                        │   │   │    │
│  │   │   │   ┌──────────── CORE ──────────────┐   │   │   │    │
│  │   │   │   │ 1 instruction stream           │   │   │   │    │
│  │   │   │   │ SIMD: 8 doubles / instruction  │   │   │   │    │
│  │   │   │   │ 2 FMA units ──► 32 FLOP/cycle  │   │   │   │    │
│  │   │   │   └────────────────────────────────┘   │   │   │    │
│  │   │   └────────────────────────────────────────┘   │   │    │
│  │   └────────────────────────────────────────────────┘   │    │
│  └────────────────────────────────────────────────────────┘    │
└────────────────────────────────────────────────────────────────┘
Counts here: 100 nodes × 2 sockets × 24 cores = 4 800 cores total

Important caveat: $R_{peak}$ is never reached. HPL/LINPACK sustains 60–80% of it ($R_{max}$); real CFD codes sustain 1–10% (see machine balance below).

বাংলায়: $R_{peak}$ হলো কাগজে-কলমে সর্বোচ্চ গতি — প্রতিটা factor আলাদা লাইনে লিখে তারপর গুণ করুন: node × socket × core × clock × FLOP/cycle। FLOP/cycle আসে vector width (AVX-512-এ ৮টা double) গুণ ২ (FMA-তে multiply আর add একসাথে) গুণ FMA unit সংখ্যা থেকে। পরীক্ষায় ধাপগুলো আলাদা দেখালে আংশিক নম্বরও পাওয়া যায়, তাই কখনো এক লাইনে শুধু চূড়ান্ত উত্তর লিখবেন না।

4.8 FLOPS prefixes — memorise the whole ladder¶

Unit	Value	In words	Typical machine
MFLOPS	$10^{6}$ FLOP/s	million	1980s workstation
GFLOPS	$10^{9}$ FLOP/s	billion	one laptop core
TFLOPS	$10^{12}$ FLOP/s	trillion	one modern node / one GPU
PFLOPS	$10^{15}$ FLOP/s	quadrillion	mid-size cluster
EFLOPS	$10^{18}$ FLOP/s	quintillion	El Capitan, Frontier (exascale)

Each step is a factor of 1000. "Exascale" means $R_{max} \geq 1$ EFLOPS on HPL.

4.9 Machine balance (bytes per FLOP) — why CFD never reaches peak¶

The machine balance $B_m$ compares how fast memory can feed the cores with how fast the cores can compute:

\[B_m = \frac{\text{memory bandwidth [byte/s]}}{R_{peak}\ [\text{FLOP/s}]} \quad \left[\frac{\text{byte}}{\text{FLOP}}\right]\]

The code balance $B_c$ is the same ratio for your loop: bytes that must be moved per FLOP executed. The achievable fraction of peak is approximately $\min(1, B_m / B_c)$.

Fully worked example. Take the node from §4.7: $R_{peak} = 3.6864$ TFLOP/s (DP) and memory bandwidth 200 GB/s.

$B_m = \dfrac{200 \times 10^{9}}{3.6864 \times 10^{12}} \approx 0.054$ byte/FLOP.
A triad loop a(i) = b(i) + s*c(i) does 2 FLOP per iteration and moves 24 bytes (load b, load c, store a, 8 bytes each): $B_c = 24/2 = 12$ byte/FLOP.
Achievable fraction of peak: $0.054 / 12 \approx 0.45\%$ — about 17 GFLOP/s out of 3 686 GFLOP/s.

This is the quantitative reason why CFD (a bandwidth-hungry stencil/sparse workload) sustains only a few percent of $R_{peak}$. Cache reuse improves $B_c$ and pushes the fraction up toward the typical 1–10%.

বাংলায়: Machine balance ($B_m$) বলে memory প্রতি FLOP-এ কত byte জোগান দিতে পারে, আর code balance ($B_c$) বলে আপনার loop-এর প্রতি FLOP-এ কত byte দরকার। আজকের মেশিনে $B_m \approx 0.05$ byte/FLOP, কিন্তু সাধারণ CFD loop-এর দরকার প্রায় 10 byte/FLOP — তাই CFD প্রায় সবসময় memory-bound, পিক-এর মাত্র কয়েক শতাংশ পায়। এটাই ব্যাখ্যা করে কেন $R_{max} \ll R_{peak}$।

4.10 The memory hierarchy — sizes and latencies (order-of-magnitude!)¶

All latency numbers below are order-of-magnitude values for a ~3 GHz CPU; exact values vary by architecture. The ordering is what the exam checks.

Level	Typical size	Latency (cycles)	Latency (≈ ns)
Registers	~ kB	~1	~0.3
L1 cache	32–64 KiB per core	~4	~1.3
L2 cache	0.5–2 MiB per core	~12	~4
L3 cache	32–256 MiB per socket	~40	~13
RAM (DDR/HBM)	0.1–2 TiB per node	~200	~70
Disk / parallel FS	PB per cluster	$10^{5}$–$10^{7}$	µs–ms

                        ▲ smaller, faster, more expensive
                        │
                  ┌───────────┐
                  │ Registers │              ~1 cycle,   ~kB
                ┌─┴───────────┴─┐
                │   L1 cache    │            ~4 cycles,  32–64 KiB/core
              ┌─┴───────────────┴─┐
              │     L2 cache      │          ~12 cycles, 0.5–2 MiB/core
            ┌─┴───────────────────┴─┐
            │       L3 cache        │        ~40 cycles, 32–256 MiB/socket
          ┌─┴───────────────────────┴─┐
          │       RAM (DDR/HBM)       │      ~200 cycles, 0.1–2 TiB/node
        ┌─┴───────────────────────────┴─┐
        │ Disk / parallel FS (Lustre)   │    ~10^5–10^7 cycles, PB/cluster
        └───────────────────────────────┘
                        │
                        ▼ bigger, slower, cheaper

A RAM access costs ~200 cycles during which an AVX-512 core could have done ~6 400 FLOP. That is why cache-friendly data layout is a core HPC skill.

বাংলায়: Memory hierarchy-র নিয়ম: যত ছোট, তত কাছে, তত দ্রুত। Register-এ ~১ cycle, L1-এ ~৪, L2-তে ~১২, L3-তে ~৪০, আর RAM-এ ~২০০ cycle লাগে — মানে RAM থেকে data আনা register-এর চেয়ে প্রায় ২০০ গুণ ধীর। সংখ্যাগুলো order-of-magnitude হিসেবে মুখস্থ রাখুন; পরীক্ষায় ক্রমটা ঠিক রাখাটাই আসল।

4.11 What LINPACK / the Top500 actually measure¶

HPL (High-Performance LINPACK) solves a dense linear system $Ax = b$ by LU factorisation; the operation count is $\approx \frac{2}{3}N^3$ FLOP for an $N \times N$ matrix.
The reported number is $R_{max}$ — the sustained FLOP rate during that solve — always less than $R_{peak}$.
HPL is compute-bound (it does $O(N^3)$ FLOP on $O(N^2)$ data), so it flatters machines: 60–80% of peak is normal.
Real CFD is memory- and communication-bound; the complementary HPCG benchmark (sparse conjugate gradient) reflects that and typically reaches only 1–3% of peak.
Exam phrasing: "The Top500 ranks machines by $R_{max}$ from the HPL benchmark, published twice a year (June/November)."

4.9 New for 2026 — JUPITER and Modular Supercomputing (guest lecture, M. Bode, JSC)¶

The 2026 course adds a guest lecture from the Jülich Supercomputing Centre: "Cutting-edge HPC on the Example of JUPITER." This is now exam-relevant material — learn the numbers and the architecture ideas.

JUPITER — Europe's first exascale system (headline facts).

Fact	Value (memorise the order of magnitude)
Top500 entry (June 2025)	#4 world, #1 Europe
HPL measured	793 PFLOP/s (of 930 PFLOP/s peak at entry, 4650 nodes)
Target	1 ExaFLOP/s FP64 (HPL)
Booster nodes	5884 nodes, each 4× NVIDIA GH200 Grace-Hopper superchips (~24 000 superchips)
Node internals	Grace CPU (Arm Neoverse V2, 72 cores) + Hopper H100 GPU; NVLink C2C 900 GB/s
Memory bandwidths	HBM3 ≈ 4000 GB/s (GPU) vs LPDDR5 ≈ 500 GB/s (CPU) — an order of magnitude apart
Interconnect	NVIDIA InfiniBand NDR200, 4 NICs per node (quad-rail)
Power	~20 MW class; Modular Data Center with 37.5 MW capacity, hot-water cooling
AI performance	~40 ExaFLOP/s FP8 (JUPITER AI Factory, JAIF)

Modular Supercomputing Architecture (MSA). Instead of one homogeneous machine, JUPITER is built from modules sharing one interconnect and file system:

 ┌────────────────────────── JUPITER (MSA) ──────────────────────────┐
 │  BOOSTER module                     CLUSTER module                 │
 │  5884 nodes × 4 GH200               582 nodes AMD Turin (x86)      │
 │  GPU-heavy, 1 EFLOP/s FP64          + 162 nodes SiPearl Rhea1      │
 │  for scalable GPU workloads         (European Arm CPU, 2027)       │
 │        │                                   │                       │
 │        └──────── InfiniBand NDR ───────────┘                       │
 │                        │                                           │
 │  STORAGE tiers: ExaFLASH 29 PB NVMe → ExaSTORE ~300 PB HDD         │
 │                 → ExaTAPE ~370 PB tape                             │
 └───────────────────────────────────────────────────────────────────┘
 Idea: match each application phase to the module that suits it —
 GPU kernels on the Booster, serial/coupling parts on the Cluster.

Energy efficiency is the design driver. The JEDI pilot system (same GH200 blades) was ranked #1 on the Green500 (June 2024) at ≈72.7 GFLOP/s per watt. Compute time on such systems is 3–6× overbooked — efficiency and clean scaling justify your allocation.

Applications shown in the lecture (know one example). Kilometre-scale full-Earth-system climate simulation (SC'25 paper: atmosphere + ocean + carbon cycle at global km-resolution, GH200 CPU handles coupling "for free" next to GPU kernels); nekRS CFD runs; LLM training (DragonLLM) on the same machine via JAIF.

Storage hierarchy (matches Ch. 15's tiers). ExaFLASH (NVMe, fastest, integrated into the InfiniBand fabric) → ExaSTORE (disk, hundreds of PB) → ExaTAPE (tape archive) — the cluster-storage pyramid at exascale.

Trend statement for the exam. "The future is GPU-heavy, increasingly Arm-based, energy-constrained, and modular: CPUs orchestrate, GPUs compute, and modules specialise." Cite JUPITER's HBM3-vs-LPDDR5 bandwidth gap ($\approx 8\times$) as the reason data placement now dominates performance thinking.

বাংলায়: JUPITER-lecture থেকে পরীক্ষায় যা আসতে পারে: (১) সংখ্যাগুলো — 1 ExaFLOP/s লক্ষ্য, ~24,000 GH200 superchip, #1 Europe; (২) MSA-র ধারণা — এক মেশিন নয়, বিশেষায়িত module-এর সমাহার (GPU-Booster + CPU-Cluster), সবাই এক network আর storage শেয়ার করে; (৩) energy efficiency এখন রাজা — Green500, hot-water cooling, GFLOP/W; (৪) HBM3 (4000 GB/s) বনাম LPDDR5 (500 GB/s) — ডেটা কোথায় রাখছ সেটাই এখন আসল খেলা। এই চারটা পয়েন্ট বলতে পারলেই guest-lecture প্রশ্ন কভার।

5. Command / Syntax / Code Breakdown¶

There are no shell commands in Lecture 1 itself, but the course already hints at the workflow. Memorise the first-contact commands you will meet on day 2:

ssh <username>@cluster.luis.uni-hannover.de

Field	Meaning
`ssh`	Secure shell client
`username`	Your cluster account
`cluster...`	Hostname of the gateway node

Common mistake: typing your password where a public key was expected → repeated Permission denied (publickey). Written-exam tip: state that SSH uses asymmetric encryption.

Command — copy input file to the cluster¶

scp input.cfg user@cluster:/home/user/sim01/

Command — submit a job (awareness)¶

sbatch run_simulation.sh

Concept — Linux command structure¶

command [options/flags] [arguments]

Example: ls -lh /home/user. ls = command, -lh = options (long, human-readable), /home/user = argument.

6. Mandatory Practical Example¶

Purpose. Demonstrate "Why HPC" with a tiny computational experiment on your own laptop.

Code — save as cpu_test.sh:

#!/bin/bash
# Compare serial vs parallel CPU work
echo "Serial:"
time seq 1 100000000 | awk '{s+=$1} END{print s}'

echo "Parallel (4 chunks):"
time (
  seq 1 25000000        | awk '{s+=$1} END{print s}' &
  seq 25000001 50000000 | awk '{s+=$1} END{print s}' &
  seq 50000001 75000000 | awk '{s+=$1} END{print s}' &
  seq 75000001 100000000| awk '{s+=$1} END{print s}' &
  wait
) | awk '{s+=$1} END{print s}'

Run: bash cpu_test.sh

Expected output

Serial:
5000000050000000
real    0m12.4s

Parallel (4 chunks):
5000000050000000
real    0m3.6s

Step-by-step

seq 1 N generates the integers 1..N.
awk '{s+=$1} END{print s}' sums them.
time reports wall-clock time.
The parallel block launches four background subshells (&) and wait blocks until all finish — process-level parallelism in its simplest form.

Check the math: $\sum_{k=1}^{10^8} k = \frac{10^8(10^8+1)}{2} = 5000000050000000$ — both versions agree; speed-up $\approx 12.4/3.6 = 3.4$ on 4 processes ($E \approx 86\%$).

বাংলায়: এই ১২-লাইনের script-টাই পুরো কোর্সের দর্শন: কাজ ভাগ করো, একসাথে চালাও, ফল জোড়া দাও। ৪ ভাগে ৩.৪ গুণ গতি (১০০% নয়!) — বাকিটা গেল overhead-এ; এটাই Amdahl-এর প্রথম স্বাদ। পরীক্ষায় "parallelism কীভাবে সময় বাঁচায়" প্রশ্নে এই উদাহরণ + Amdahl উল্লেখ করলেই শক্ত উত্তর।

Real-Life HPC/CFD Meaning. Replace seq with "simulate one cell" and you have a baby version of an OpenMP-parallel CFD loop. On a real cluster the same pattern is mpirun -np 4096 ./solver.

Written Exam Relevance. "Explain how a parallel program saves wall-clock time" — answer with this 4-process model and mention Amdahl's law.

7. Real HPC/CFD Workflow¶

A textbook day in the life of a CFD researcher:

Edit the input file on your laptop with Vim.
Commit the change with Git.
Push to a GitLab remote.
SSH into the cluster.
git pull the new version.
Compile with make or cmake --build.
Submit the simulation: sbatch run.sh.
Monitor with squeue -u $USER.
Copy results back: rsync -avz cluster:results/ ./local/.
Plot with gnuplot.
Debug with gdb / valgrind if it crashed.
Document outcomes in a Markdown report.

Every step is a chapter of this course. Lecture 1 is the map.

8. Exercises and Solutions¶

There is no formal exercise sheet for Chapter 1. The companion practical for L01 is E01_WSL — installing the Windows Subsystem for Linux.

E01 (rewritten in simple English). Install Ubuntu inside Windows 10/11 using WSL2, open a Linux terminal, verify with uname -a.

বাংলায়: Windows-এ WSL2 চালু করে উবুন্টু ইনস্টল করো; টার্মিনাল খুলে uname -a দিয়ে যাচাই করো।

Solution (step-by-step):

# In an elevated PowerShell:
wsl --install -d Ubuntu
# After reboot, open "Ubuntu" from the start menu, set username/password, then:
uname -a

Expected line:

Linux DESKTOP-XXXXX 5.15.X-microsoft-standard-WSL2 ... GNU/Linux

Common mistake. Virtualisation disabled in BIOS → WSL refuses to start.

Harder version. Configure WSL to use 4 CPUs and 8 GB RAM via %UserProfile%\.wslconfig:

[wsl2]
processors = 4
memory     = 8GB

Restart with wsl --shutdown, then check inside WSL: nproc → 4; free -h → ≈ 8G.

Marking-scheme answer (5 marks):

1: name WSL2 = a Linux kernel inside Windows.
1: wsl --install -d Ubuntu.
1: virtualisation in BIOS.
1: verify with uname -a.
1: why a Linux env is needed for HPC/CFD.

9. Written Exam Focus¶

9.1 Short Answers¶

Q. Define HPC. A. HPC aggregates many CPUs/GPUs across networked nodes so one calculation finishes in hours instead of years — essential for problems with billions of unknowns, e.g. CFD of an aero-engine combustor.

Q. What is the Top500? A. A semi-annual list ranking the 500 fastest publicly-known supercomputers by the LINPACK benchmark in FLOPS.

Q. Why is Linux preferred on HPC clusters? A. Open-source, modular, stable, scriptable through shells, POSIX-compliant — vendors adapt the kernel, users automate every step.

Q. Why are virtual experiments needed alongside real experiments in turbomachinery? A. Real engines are mostly inaccessible to optical diagnostics; CFD gives full 3-D fields of every variable but must be validated against experiments.

9.2 Medium Answers¶

Q. (8 marks) Compare experiments and CFD simulations as research tools.

A. Experiments give ground truth and are required for validation; but they are expensive, measure only 2–3 quantities simultaneously, and many regions are optically opaque. CFD is cheap to repeat, yields the full field of all variables, and allows design iteration before hardware exists — but needs validation and becomes trustworthy only at high resolution, which forces HPC. The methods are complementary: experiments validate, simulations interpolate and extrapolate.

9.3 Long Answer (12–15 marks)¶

Q. Discuss the role of HPC in modern CFD-based research on sustainable aviation propulsion.

A. Introduction. HPC is the enabling technology for CFD-driven research on low-emission aero engines.

Main concept. CFD discretises the Navier–Stokes equations on a 3-D mesh; a typical LES of one combustor sector has $10^8$–$10^9$ cells × $10^5$ time-steps × tens of species equations → $10^{15}$–$10^{18}$ FLOP. Far beyond a workstation; exascale machines (e.g. Frontier at ~1.6 EFLOP/s) do it in days.

Step-by-step example. (i) geometry+mesh, (ii) compile a reactive-flow solver, (iii) parallel job on hundreds of nodes via SLURM, (iv) post-process with gnuplot/ParaView, (v) compare against optical test-rig data. Each step uses Linux, SSH, scripting, version control, parallel libraries — the topics of this course.

Conclusion. HPC turns CFD from a qualitative aid into a predictive design tool, making low-CO₂ engines a realistic target.

9.4 Output Prediction¶

Q. Order of magnitude of wall-clock time of a DNS with $10^9$ cells, $10^5$ steps, 100 ops/cell/step, on a 1 PFLOPS machine?

A. Total ops $= 10^9 \times 10^5 \times 10^2 = 10^{16}$. Time $= 10^{16}/10^{15} = 10$ s of pure compute ⇒ in practice 1–10 hours after parallel inefficiencies (memory bandwidth, communication, I/O). Conclusion: HPC required.

9.5 Comparison¶

Item	Workstation	HPC cluster
Cores	4–32	$10^5$–$10^7$
RAM	16–256 GB	TB scale
Job control	manual	SLURM/PBS
Storage	local SSD	parallel FS (Lustre)
OS	Win/macOS/Linux	Linux only
User access	direct	SSH only
Software	GUI installs	Modules / Spack

9.6 Templates¶

Definition template: "[Term] is …; it is used to … In HPC/CFD this matters because … Example: …"

Argument template: "The two methods differ in [criteria]. Method A …; Method B …; therefore A is preferable when …"

9.7 Marking Scheme — "Why HPC for CFD?" (5 marks)¶

1: CFD is computationally expensive.
1: estimate operation count ($10^{15}$+).
1: define HPC.
1: name a real cluster (Frontier / El Capitan / JUWELS).
1: link to an engineering goal (aero-engine, climate, drug design).

10. Very Hard Questions¶

Beginner

What does HPC stand for? → High-Performance Computing.
Name two supercomputers. → El Capitan, Frontier, JUWELS, Aurora.
Which OS runs on HPC clusters? → Linux.
What is a "core"? → One processing unit of a CPU.
What does the Top500 measure? → LINPACK / HPL benchmark performance.

Intermediate

Convert 1 PFLOPS to FLOPS. → $10^{15}$.
Node vs core. → Node = whole computer; core = execution unit inside a CPU.
Why Linux? → open-source + scriptable + POSIX (any 3).
Simulation vs experiment? → cost vs accessibility vs validation.
Role of the interconnect? → fast inter-node communication.

Hard

FLOPs for an LES of $5\times10^8$ cells, $5\times10^4$ steps, 200 ops/cell/step. → $5\times10^{15}$.
Why does adding cores eventually stop helping? → Amdahl's law.
Why are GPUs on the rise? → higher FLOPS/W, high arithmetic density.
Why is communication latency critical? → strong scaling becomes communication-bound.
Why is power a key Top500 figure? → cooling cost ∝ power; targets push GFLOPS/W.

Very Hard

Show algebraically that speed-up plateaus for fixed serial fraction f. → $S(N)=1/(f+(1-f)/N) \to 1/f$.
5% serial code: max speed-up? → $1/0.05 = 20\times$.
Communication ∝ √N, computation ∝ N per node — when does communication dominate? → when problem size per node shrinks enough that the √N message term exceeds compute (strong-scaling limit).

Deep Integration

Trace a CFD job from Vim edit to final plot, naming each chapter. → Vim (5) → Git (10) → SSH/scp (4) → bash/SLURM (8/15) → C++ build (11/12) → debugging (13) → parallelisation (14) → cluster (15) → gnuplot (9).
Why is Linux "scriptability" more important than "free of charge"? → reproducibility & automation.

Coding/Command

Print the core count. → nproc.
Time a program. → time ./a.out.

Debugging

Why might a parallel program give different results each run? → races / non-deterministic FP reduction order.
Why may speed-up degrade beyond N = 16 on a laptop? → hyperthreads share ALUs / cache contention.

Long Written

(250 words) How does institute research tie Linux, HPC, and sustainable aviation? (Use 9.3 as the model.)

11. Debugging and Mistake Analysis¶

Mistake	Why wrong	Correct version	Explanation
"Supercomputer = one giant CPU"	modern HPC is parallel	"many nodes connected by a network"	clock frequency stalled ~2005; only parallelism scales
Mixing core and thread	threads can share a core	core = one instruction stream	SMT/HT shares ALUs
"GPUs replace CPUs"	they complement	heterogeneous nodes have both	CPU control + GPU stencil
"HPC = fast desktop"	misses parallelism	"aggregate parallel performance"	definition matters

12. Mini Project for Mastery¶

Goal: Estimate your laptop's FLOPS and compare to a Top500 entry.

sudo apt install sysbench -y
sysbench cpu --cpu-max-prime=20000 --threads=$(nproc) run

Convert events/s to a rough FLOPS estimate; compare to El Capitan (~1.7 EFLOPS). Expected outcome: your laptop ~$10^9$, ratio ~$10^{-9}$ — a billion times slower. A powerful exam-ready statement.

13. Final Chapter Cheat Sheet¶

Item	Memorise
HPC	High-Performance Computing — many cores together
CFD	Computational Fluid Dynamics
FLOPS	Floating-point Ops per Second
$R_{peak}$	nodes × sockets × cores × clock × FLOP/cycle
Prefixes	M($10^{6}$) G($10^{9}$) T($10^{12}$) P($10^{15}$) E($10^{18}$)
Top500	top500.org — LINPACK/HPL ranking, June+November
El Capitan	#1 (LLNL) ~1.7 EFLOPS
Frontier	Oak Ridge exascale
Linux share in Top500	100%
Memory wall	latency: reg ~1, L1 ~4, L2 ~12, L3 ~40, RAM ~200 cycles
Common trap	adding cores doesn't always help (Amdahl)
Written-exam line	"HPC is essential because CFD problems require $10^{15}$+ ops which exceed any single workstation."

14. Mock Exam — Four Levels¶

Level 1 — Basic (definitions & syntax)¶

Q1. Expand and define FLOPS.

Solution: Floating-point Operations Per Second — the rate of floating-point arithmetic a machine sustains.

Q2. Order these by size: TFLOPS, GFLOPS, EFLOPS, PFLOPS.

Solution: GFLOPS ($10^9$) < TFLOPS ($10^{12}$) < PFLOPS ($10^{15}$) < EFLOPS ($10^{18}$).

Q3. What benchmark ranks the Top500?

Solution: LINPACK (HPL) — dense linear-system solve, reporting $R_{max}$.

Q4. Name the four hardware levels from core to cluster.

Solution: core → socket/CPU → node → (rack →) cluster.

Q5. What is a node?

Solution: One complete computer in the cluster (CPUs, RAM, NIC), connected to others via the interconnect.

Level 2 — Intuitive (predict / explain why)¶

Q1. Why did CPU clock speeds stop rising around 2005, and what replaced frequency scaling?

Solution: Power density (heat) grows superlinearly with clock; cooling became impossible → parallelism (more cores) replaced frequency scaling.

Q2. A "1 TB" disk shows 931 GiB in Linux. Where did the space go?

Solution: Nowhere — units differ: $10^{12}$ B ÷ $2^{30}$ B/GiB $= 931$ GiB. Marketing uses powers of 10, OS uses powers of 2.

Q3. Two machines: A = few cores at 5 GHz, B = many cores at 2 GHz with equal total FLOPS. Which is better for CFD, and why?

Solution: B in general — CFD is mesh-parallel and memory-bandwidth-hungry; many cores with wide memory beat few fast cores. (A wins only for stubbornly serial workloads — Amdahl.)

Q4. Why does doubling cores rarely halve runtime?

Solution: Serial fraction (Amdahl), communication/synchronisation overhead, and memory-bandwidth saturation.

Q5. Why does the Top500 also publish power (MW) per machine?

Solution: Energy is the real operating constraint — GFLOPS/W (Green500) decides feasibility; an exascale machine draws ~20–30 MW.

Level 3 — Hard (exam level)¶

Q1. (8 marks) Compute $R_{peak}$ for one node: 2 sockets × 24 cores × 2.4 GHz, AVX-512 with FMA (32 DP FLOP/cycle/core). Then for 100 nodes, in PFLOPS.

Solution: Per node: $2 \times 24 \times 2.4\times10^9 \times 32 = 3.686\times10^{12}$ → 3.69 TFLOPS. 100 nodes: $0.369$ PFLOPS. বাংলা ইঙ্গিত: সূত্রটা গুণফল মাত্র — সব ইউনিট ঠিক রেখে (GHz → $10^{9}$) পরপর গুণ করো; FLOP/cycle = vector width × 2 (FMA)।

Q2. (8 marks) A solver does $10^{14}$ FLOP per timestep and sustains 5% of the 0.369 PFLOPS machine above. Time per step? For $10^4$ steps?

Solution: Sustained $= 0.05 \times 3.69\times10^{14} = 1.84\times10^{13}$ FLOP/s. Per step: $10^{14}/1.84\times10^{13} \approx 5.4$ s. Total: $5.4\times10^4$ s ≈ 15 h. বাংলা ইঙ্গিত: "sustained %" না দিলে peak ধরে নিও না — বাস্তবে 1–10%; পরীক্ষাও ঠিক এই ফাঁদটাই পাতে।

Q3. (8 marks) Machine balance: a node has 3.69 TFLOPS peak and 400 GB/s memory bandwidth. Compute bytes/FLOP, and decide whether a stencil code needing 1 byte/FLOP is compute- or memory-bound there.

Solution: Balance $= 400\times10^9 / 3.69\times10^{12} \approx 0.11$ bytes/FLOP. The code needs 1 byte/FLOP $\gg$ 0.11 → memory-bound: the CPU starves; effective performance ≈ bandwidth-limited $400\times10^9 \times 1$ FLOP/byte... i.e. ~0.4 TFLOPS (11% of peak). বাংলা ইঙ্গিত: machine balance < code demand ⇒ memory-bound — CFD-র প্রায় সব stencil-ই এই দলে; "১০% peak-ই স্বাভাবিক" — এই বাক্যটা লিখো।

Q4. (10 marks) Using the memory-hierarchy latencies (L1 ~4, RAM ~200 cycles), explain quantitatively why blocking/tiling a matrix kernel can give ~10× speed-up.

Solution: Unblocked: most accesses miss cache → ~200-cycle stalls dominate; effective ~1 op per ~200 cycles of waiting in the worst case. Blocked: working set fits in L1/L2 → ~4–12 cycle access; ratio 200/12 ≈ 16× fewer stall cycles, realised as ~an order of magnitude — locality, not arithmetic, sets the speed. বাংলা ইঙ্গিত: সংখ্যা দুটো (4 বনাম 200) উত্তরে রাখলেই "quantitatively" শর্ত পূরণ — অনুপাতই গল্পের নায়ক।

Q5. (10 marks) Estimate (order of magnitude) how long the serial cpu_test.sh sum of $10^8$ numbers SHOULD take if the machine ran at 1 GFLOPS effective, and explain why the measured 12 s is far larger.

Solution: $10^8$ additions / $10^9$ FLOP/s $= 0.1$ s ideal. Measured 12 s ⇒ ~120× overhead: the pipeline spawns processes, formats/parses ASCII decimal text through awk (dozens of cycles per char), and crosses a pipe — I/O and parsing dominate, not arithmetic. Lesson: real performance is set by data movement, the recurring HPC theme. বাংলা ইঙ্গিত: "কেন এত ধীর" প্রশ্নের উত্তর কখনোই "কম্পিউটার ধীর" নয় — text parsing/process overhead-এর মতো নির্দিষ্ট খরচ ধরিয়ে দাও।

Level 4 — Beyond the lecture (transfer + coding)¶

Q1. Write a portable bash snippet printing a one-line summary of this machine: hostname, #cores, RAM (GiB), kernel version.

Solution:

printf "%s: %s cores, %s GiB RAM, kernel %s\n" \
  "$(hostname)" "$(nproc)" \
  "$(awk '/MemTotal/ {printf "%.1f", $2/1048576}' /proc/meminfo)" \
  "$(uname -r)"

/proc/meminfo reports KiB → divide by $1024^2$ for GiB; everything is standard on any Linux node. বাংলা ইঙ্গিত: /proc-ফাইলগুলোই Linux-এর "নিজের সম্পর্কে সত্য" — awk দিয়ে পড়াটা দুই chapter-এর মেলবন্ধন।

Q2. You must choose between 1 node with 128 cores or 8 nodes with 16 cores each (same total). For (a) an embarrassingly parallel parameter sweep and (b) a tightly-coupled CFD halo-exchange solver, which do you pick and why?

Solution: (a) Either works — no communication; pick by availability/queue time (8 small nodes often schedule faster). (b) The single 128-core node: halo exchange stays in shared memory/NUMA instead of crossing the network — lower latency, no interconnect bottleneck (Ch 14 surface-to-volume logic). বাংলা ইঙ্গিত: প্রশ্নের ধরন চেনো: communication-হীন কাজ topology-উদাসীন; coupled কাজ network এড়াতে চায়।

Q3. A vendor claims "our 2 EFLOPS machine will run your code 2000× faster than your 1 TFLOPS workstation." Give two quantitative reasons this is misleading.

Solution: (1) Amdahl: with even 1% serial fraction, $S_{max} = 100$ — never 2000×, however many cores. (2) The claim compares peak values: your code's sustained fraction (often 1–10% of peak, memory-bound) differs per machine; LINPACK ratios don't transfer to stencil codes. (Bonus: strong-scaling communication overheads grow with node count.) বাংলা ইঙ্গিত: মার্কেটিং-দাবি ভাঙার দুই হাতুড়ি সবসময়: Amdahl আর peak-বনাম-sustained।

Q4. Estimate the energy (kWh) of a 24 h job on 50 nodes at 500 W/node, and the CO₂ (at 400 g/kWh). Should the course's bash/awk skills factor into "green computing"? One concrete example.

Solution: $50 \times 0.5\,\text{kW} \times 24\,\text{h} = 600$ kWh → $600 \times 0.4 = 240$ kg CO₂. Yes: profiling and scripting prevent waste — e.g. an awk check that stops a diverged run after 10 min (residual = nan in the log) saves the remaining 23.8 h × 50 nodes ≈ 595 kWh of garbage compute. বাংলা ইঙ্গিত: গণনাটা সহজ (kW × h), কিন্তু "স্ক্রিপ্টিং = সবুজ HPC" সংযোগটা দেখাতে পারাই out-of-topic প্রশ্নের আসল উত্তর।

15. MCQ Drill — Exam Section (32 questions)¶

Difficulty ramps from basic (1) to very hard (32). Questions marked (select all that apply) have more than one correct option.

1. HPC stands for… A. High-Precision Calculation B. High-Performance Computing C. Hybrid Parallel Computing D. High-Power Cluster

Answer: B. High-Performance Computing — aggregating many processors so one calculation finishes in reasonable time.

2. 1 PFLOPS equals… A. $10^{9}$ FLOPS B. $10^{12}$ FLOPS C. $10^{15}$ FLOPS D. $10^{18}$ FLOPS

Answer: C. Prefix ladder: G=$10^9$, T=$10^{12}$, P=$10^{15}$, E=$10^{18}$. Exa is D.

3. The Top500 list ranks machines by… A. core count B. memory size C. LINPACK (HPL) performance D. power consumption

Answer: C. $R_{max}$ from the HPL dense linear-algebra benchmark; power is tracked separately (Green500).

4. Which OS family runs on essentially 100% of Top500 systems? A. Windows Server B. Linux C. BSD D. macOS

Answer: B. Linux — open-source, scriptable, POSIX, vendor-adaptable.

5. Order the hardware hierarchy from smallest to largest: A. node → core → cluster → socket B. core → socket → node → cluster C. socket → core → node → cluster D. core → node → socket → cluster

Answer: B. Cores sit inside a CPU socket; sockets populate a node; nodes form the cluster.

6. A "node" in a cluster is… A. one CPU core B. one complete computer with CPUs, RAM and network C. one rack D. the login server only

Answer: B. The unit that runs an OS instance and connects to the interconnect.

7. Which are genuine reasons Linux dominates HPC? (select all that apply) A. it is scriptable through shells B. it is closed and certified C. vendors can modify the kernel D. POSIX portability

Answer: A, C, D. Open-source (not closed) + automation + portability; B is the opposite of the truth.

8. CFD needs HPC primarily because… A. graphics cards render flow nicely B. operation counts reach $10^{15}$+ per simulation C. Linux requires clusters D. licenses demand it

Answer: B. Cells × timesteps × ops per cell explodes; a workstation cannot finish in research time.

9. JUPITER's headline FP64 target performance is… A. 1 PFLOP/s B. 100 PFLOP/s C. 1 EFLOP/s D. 40 EFLOP/s

Answer: C. 1 ExaFLOP/s HPL — Europe's first exascale system. (40 EF is the FP8 AI figure — a favourite trap.)

10. The JUPITER Booster node contains… A. 4× NVIDIA GH200 Grace-Hopper superchips B. 2× AMD Turin CPUs C. 4× Intel Xeon D. 1× Rhea1

Answer: A. Four Grace-Hopper superchips per node; AMD Turin belongs to the Cluster module.

11. Why did CPU clock frequencies stop rising around 2005? A. transistor supply ended B. power density / heat became unmanageable C. software could not use GHz D. marketing reasons

Answer: B. Dennard scaling ended: power (and heat) grows superlinearly with frequency; the industry pivoted to more cores.

12. A "1 TB" disk shows ≈931 GiB in Linux because… A. the filesystem eats 7% B. $10^{12}$ B ÷ $2^{30}$ B/GiB ≈ 931 C. bad sectors D. compression overhead

Answer: B. Decimal (vendor) vs binary (OS) units; nothing is lost.

13. Amdahl's law with serial fraction f gives maximum speed-up… A. N B. 1/f C. f·N D. N/f

Answer: B. $S(N)=1/(f+(1-f)/N) \to 1/f$ as $N\to\infty$.

14. With f = 0.05, the maximum possible speed-up is… A. 5× B. 20× C. 50× D. unlimited

Answer: B. $1/0.05 = 20$ — regardless of how many cores you buy.

15. $R_{peak}$ of one node = 2 sockets × 24 cores × 2.4 GHz × 32 FLOP/cycle ≈ … A. 3.7 GFLOPS B. 370 GFLOPS C. 3.7 TFLOPS D. 37 TFLOPS

Answer: C. $2\cdot24\cdot2.4\times10^9\cdot32 = 3.69\times10^{12}$ FLOP/s.

16. Which memory level has ~200-cycle latency? A. register B. L1 cache C. L3 cache D. main RAM

Answer: D. Order of magnitude: register 1, L1 ~4, L2 ~12, L3 ~40, RAM ~200 cycles — the memory wall.

17. "Machine balance" is the ratio… A. FLOPS per watt B. bytes/s of memory bandwidth per FLOP/s of peak C. cores per node D. price per TFLOP

Answer: B. If a code needs more bytes/FLOP than the machine supplies, it is memory-bound.

18. A stencil code needing 1 byte/FLOP on a machine with balance 0.11 byte/FLOP will run at roughly… A. 100% of peak B. ~50% of peak C. ~11% of peak D. 1% of peak

Answer: C. Bandwidth-limited: 0.11/1 ≈ 11% of peak is the ceiling — typical for CFD kernels.

19. Which statements about experiments vs CFD are correct? (select all that apply) A. experiments provide validation ground truth B. CFD yields all field variables everywhere C. CFD needs no validation at high resolution D. many real geometries are optically inaccessible

Answer: A, B, D. C is false — resolution never removes the need for validation.

20. The Green500 ranks systems by… A. total FLOPS B. FLOPS per watt C. renewable energy share D. CO₂ emitted

Answer: B. Energy efficiency; the GH200-based JEDI pilot took #1 with ≈72.7 GFLOPS/W.

21. In the Modular Supercomputing Architecture, modules… A. are physically separate clusters with separate storage B. specialise (GPU Booster, CPU Cluster) but share interconnect and file system C. only differ in rack colour D. must run identical node types

Answer: B. Specialised modules, one fabric, one storage — match each application phase to the right module.

22. HBM3 vs LPDDR5 bandwidth on a GH200 node is roughly… A. equal B. 2× apart C. ~8× apart (4000 vs 500 GB/s) D. 100× apart

Answer: C. That gap is why data placement (GPU vs CPU memory) dominates performance thinking.

23. The serial cpu_test.sh sums $10^8$ integers in 12 s, the 4-process version in 3.6 s. Efficiency ≈ … A. 100% B. 83% C. 60% D. 25%

Answer: B. $S = 12/3.6 = 3.33$; $E = S/4 = 0.83$.

24. A DNS with $10^{16}$ total operations on a 1 PFLOPS machine needs at least… A. 10 s of pure compute B. 17 min C. 3 h D. 4 days

Answer: A. $10^{16}/10^{15} = 10$ s ideal — real runs take hours because sustained $\ll$ peak.

25. Which factors make sustained performance far below peak? (select all that apply) A. memory bandwidth limits B. communication/synchronisation C. I/O D. the compiler license

Answer: A, B, C. The classic trio; D is nonsense.

26. "Strong scaling" means… A. growing the problem with core count B. fixed problem, more cores C. overclocking D. using stronger nodes

Answer: B. Fixed total work; weak scaling fixes work per processor.

27. Why do 1000-core jobs often queue longer than 8-core jobs? A. SLURM dislikes big jobs B. contiguous large allocations are rarer; backfill favours small/short jobs C. big jobs cost more per core D. network limits

Answer: B. The scheduler must find a big free slot; small jobs slip into gaps (backfill).

28. The JUPITER storage hierarchy, fastest to slowest: A. ExaTAPE → ExaSTORE → ExaFLASH B. ExaFLASH → ExaSTORE → ExaTAPE C. ExaSTORE → ExaFLASH → ExaTAPE D. all equal

Answer: B. NVMe flash → disk → tape; capacity grows as speed drops — the standard pyramid.

29. A vendor promises "2000× faster" comparing your 1 TFLOPS workstation with a 2 EFLOPS machine. Which objections are valid? (select all that apply) A. Amdahl caps speed-up at 1/f B. peak ratios ignore sustained fractions C. FLOPS are not defined for CFD D. strong-scaling communication overheads grow with node count

Answer: A, B, D. C is false — FLOP counting applies fine; the ratio argument is what misleads.

30. Energy: 50 nodes × 500 W × 24 h ≈ … and at 400 g CO₂/kWh ≈ … A. 600 kWh, 240 kg B. 60 kWh, 24 kg C. 6000 kWh, 2400 kg D. 600 kWh, 24 kg

Answer: A. $50\times0.5\,\mathrm{kW}\times24\,\mathrm{h}=600$ kWh; $600\times0.4=240$ kg.

31. Which statements about GH200 are correct? (select all that apply) A. Grace is an Arm CPU B. Hopper is the GPU C. NVLink C2C links them at ~900 GB/s D. it is an x86 design

Answer: A, B, C. Grace = Arm Neoverse V2 (72 cores); D is wrong — the Arm shift is exactly the exam-worthy trend.

32. Frontier-class machines draw ~20 MW. The deepest reason power is THE design constraint: A. electricity contracts are annual B. cooling and supply costs scale with power while budgets are fixed — FLOPS/W decides feasible size C. fuses limit current D. EU regulation caps MW

Answer: B. Machines are sized by the power/cooling envelope; that is why Green500, hot-water cooling and GPU efficiency dominate design talks.

End of Chapter 1.

Unit	Value	In words	Typical machine
MFLOPS	\(10^{6}\) FLOP/s	million	1980s workstation
GFLOPS	\(10^{9}\) FLOP/s	billion	one laptop core
TFLOPS	\(10^{12}\) FLOP/s	trillion	one modern node / one GPU
PFLOPS	\(10^{15}\) FLOP/s	quadrillion	mid-size cluster
EFLOPS	\(10^{18}\) FLOP/s	quintillion	El Capitan, Frontier (exascale)

Item	Workstation	HPC cluster
Cores	4–32	\(10^5\)–\(10^7\)
RAM	16–256 GB	TB scale
Job control	manual	SLURM/PBS
Storage	local SSD	parallel FS (Lustre)
OS	Win/macOS/Linux	Linux only
User access	direct	SSH only
Software	GUI installs	Modules / Spack