03. Pre-system boot
- Power on. What now? Run a program? What program? All we have is CPU and pristine clean RAM
- — Use ROM dude. Copy it to the pre-defined address of RAM and start there
ROM:
Primary loader:
- Very small. Like, less than one sector on disk
- Very dull. What you can program into 446 bytes?
- Specific by the device it resides:
- Knows how to read blocks from device
Knows what to read ad run from this device (e. g. by hardcoding a list of blocks to read just into primary loader code)
Can load almost and run any amount of code from the fixed place of the device and run it. Call it secondary loader
Secondary loader:
- Very smart (must be):
- Must recognize a umber filesystems (e. g. ext2, ext3, ext4, jfs, xfs, btrfs, zfs, HPFS, ...) to find loadable file on
Must recognize a number of formats (e. g. linux, BSD, windows, OS/X etc) to load any OS kernel
Must have an ability to load and link modules (drivers) with the kernel
Why? Because no modern kernel is so monolith to include all the dirivers in it, and disk driver is sufficient to the kernel, that must mount that versy disk as root
IRL: No such thing as universal boot loader
- Knows about disk partition and filesystems
Can select and boot specific OS kernel or something alike from the given FS
Can boot other secondary loader to boot another OS (e. g. windows)
- Very smart (must be):
OS kernel:
- Checks and tunes hardware
- mounts root filesystem
- initial OS startup
So, theoretically:
- BootROM
- Select a device to boot from
- Load a Boot block from it
- Boot block (primary loader)
(almost never) select a secondary loader
load and boot a secondary loader from somewhere at the current device
- Boot program (secondary loader)
- Select and boot variant (one of kernels available and parameters) and boot it
- or select and chainload another secondary loader (capable for boot other type kernels)
- Kernel
- check and set-up devices
- find filesystem to mount as root, mount it and initiate OS startup
The locked key problem:
- Kernel is to mount / directory from the particular filesystem of the particular partition on the particular hard disk drive connected to the particular disk controller
There are so many filesystem types, disks and disk controllers, that no kernel has them compiled in. Those are loadable modules, that collected as files on the particular filesystem (often in the /lib/modules directory). The kernel shall load some of them first
- See (1.)
Linux: lazy and simple
Suppose we already loaded the kernel, and it has an access to the small filesystem contains minimal set of modules and executables, enough to mount «/».
- Then no problem to re-mount from this tiny storage to actual root and run a full OS
- The solution:
Create an archive this just-need-to-mount-root set of modules and binaries. Call it Initial virtual disk (initrd, initramfs, name it).
Teach secondary loader to load two files instead of just kernel (kernel and initrd). No other capabilities is needed (no individual module loading, no kernel linking, etc.)
Teach kernel to mount initrd from memory as filesystem and boot from it.
- We get a tiny all-in-memory linux-based OS that can load all the modules, unomunt and purge memory of initrd, mount actual root and boot from it
- PROFIT
Example (VirualBox VM)
Boot sequence
See also: Simply Linux 9 installation on VirtualBox When starting a (virtual) computer, first we get this message:
It is displayed by firmware, and do what boot ROM is to do: allows us to select bootable media. Press F12.
This is rather simple VirtualBox firmware boot menu. Select 1 anyway.
We are using GRUB bootloader. So «stage 1» (primary) loader serves only to boot secondary loader, and works invisible until something bad happens (e. g. secondary loader data is corrupted on disk). Secondary GRUB loader is really powerful software. It allows to select some predefined bootable configurations from config file, to navigate though submenus and much more.
More than that, it allows to edit selected bootable configuration! Press 'E' here:
Now we can observe boot parameters.
Note linux16 command that specifies a kernel image
and initrd16 command that specifies an initial virtual disk image
Lets add some gibberish kernel boot parameter:
And press F10 to keep it and boot.
After the OS is loaded
We need root password (or rights to perform a sudo command) to continue. Let's gather super-user rights (sudo su - goes as well).
[user@comp-core-i7-8750h-8b78f7 ~]$ su - Password: [root@comp-core-i7-8750h-8b78f7 ~]# id uid=0(root) gid=0(root) группы=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),19(proc)
Note # at the end of the shell prompt? That means we have proper rights. What filesystems are mounted now?
[root@comp-core-i7-8750h-8b78f7 ~]# df Filesystem Size Used Avail Use% Mounted on udevfs 984M 0 984M 0% /dev runfs 993M 1.1M 992M 1% /run /dev/sda2 30G 6.2G 22G 22% / tmpfs 993M 4.0K 993M 1% /dev/shm tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 993M 0 993M 0% /sys/fs/cgroup tmpfs 993M 8.0K 993M 1% /tmp tmpfs 199M 16K 199M 1% /run/user/500
These are «true files» filesystems only, not including «more virtual» ones like /proc, /sys and others. What block devices are available?
[root@comp-core-i7-8750h-8b78f7 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 32G 0 disk ├─sda1 8:1 0 2G 0 part [SWAP] └─sda2 8:2 0 30.1G 0 part / sr0 11:0 1 1.7G 0 rom
There are two partitions on /dev/sda device — one for swap and one for root filesystem What unique ID's the devices partitions have?
[root@comp-core-i7-8750h-8b78f7 ~]# blkid /dev/sda1: UUID="f5636c63-6d23-4b03-872b-db787212b7a3" TYPE="swap" /dev/sda2: UUID="311e5f8a-0a5d-45b4-961f-466b538d3e57" TYPE="ext4" /dev/sr0: UUID="2020-04-02-21-43-51-00" LABEL="Simply Linux live 9.0 x86_64" TYPE="iso9660" PTUUID="333d32d7" PTTYPE="dos" /dev/pktcdvd/pktcdvd0: UUID="2020-04-02-21-43-51-00" LABEL="Simply Linux live 9.0 x86_64" TYPE="iso9660" PTUUID="333d32d7" PTTYPE="dos"
Note swap and ext4 filessytems all have and UUID, but CD doesn't These UUIDs are used to mount filesystems and uitilize the swap:
[root@comp-core-i7-8750h-8b78f7 ~]# cat /etc/fstab proc /proc proc nosuid,noexec,gid=proc 0 0 devpts /dev/pts devpts nosuid,noexec,gid=tty,mode=620 0 0 tmpfs /tmp tmpfs nosuid 0 0 UUID=311e5f8a-0a5d-45b4-961f-466b538d3e57 / ext4 relatime 11 UUID=f5636c63-6d23-4b03-872b-db787212b7a3 swap swap defaults 00
Previously, it was common to use device partition names (like /dev/sda3), but device names are generated in runtime, and there's a chance that /dev/sda will be named as /dev/sdb when you reboot you computer with the external hard drive plugged in.
- UUIDs are never changed Another way to list partitions on certain device:
[root@comp-core-i7-8750h-8b78f7 ~]# parted /dev/sda GNU Parted 3.2.46-e4ae Using /dev/sda Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) p Model: ATA VBOX HARDDISK (scsi) Disk /dev/sda: 34.4GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 1049kB 2080MB 2079MB primary linux-swap(v1) 2 2080MB 34.4GB 32.3GB primary ext4 boot (parted) quit
parted (and gparted, if you're interested), is mainly used to change device partitions, but we skip that
Where primary loader is located? It's the initial block of your boot device:
[root@comp-core-i7-8750h-8b78f7 ~]# dd if=/dev/sda count=1 | hexdump -C 1+0 records in 1+0 records out 512 bytes copied, 2.4268e-05 s, 21.1 MB/s 00000000 eb 63 90 00 8e d0 bc 00 01 fb fc 33 c0 8e d8 8e |.c.........3....| 00000010 c0 be 00 7c bf 00 7e b9 00 02 f3 a5 68 20 7e c3 |...|..~.....h ~.| 00000020 be fa 7e bb be 7f 80 7f 04 0a 74 41 83 c3 10 81 |..~.......tA....| 00000030 fb fe 7f 7c f1 33 c0 cd 13 b4 08 b2 81 cd 13 72 |...|.3.........r| 00000040 2c b9 b4 7f b2 81 e8 ee 00 0a e4 75 20 81 3e fe |,..........u .>.| 00000050 7d 55 aa 75 18 bb be 7d 80 7f 00 80 01 00 00 00 |}U.u...}........| 00000060 00 00 00 00 ff fa 90 90 f6 c2 80 74 05 f6 c2 70 |...........t...p| 00000070 74 02 b2 80 ea 79 7c 00 00 31 c0 8e d8 8e d0 bc |t....y|..1......| 00000080 00 20 fb a0 64 7c 3c ff 74 02 88 c2 52 be 80 7d |. ..d|<.t...R..}| 00000090 e8 17 01 be 05 7c b4 41 bb aa 55 cd 13 5a 52 72 |.....|.A..U..ZRr| 000000a0 3d 81 fb 55 aa 75 37 83 e1 01 74 32 31 c0 89 44 |=..U.u7...t21..D| 000000b0 04 40 88 44 ff 89 44 02 c7 04 10 00 66 8b 1e 5c |.@.D..D.....f..\| 000000c0 7c 66 89 5c 08 66 8b 1e 60 7c 66 89 5c 0c c7 44 ||f.\.f..`|f.\..D| 000000d0 06 00 70 b4 42 cd 13 72 05 bb 00 70 eb 76 b4 08 |..p.B..r...p.v..| 000000e0 cd 13 73 0d 5a 84 d2 0f 83 d8 00 be 8b 7d e9 82 |..s.Z........}..| 000000f0 00 66 0f b6 c6 88 64 ff 40 66 89 44 04 0f b6 d1 |.f....d.@f.D....| 00000100 c1 e2 02 88 e8 88 f4 40 89 44 08 0f b6 c2 c0 e8 |.......@.D......| 00000110 02 66 89 04 66 a1 60 7c 66 09 c0 75 4e 66 a1 5c |.f..f.`|f..uNf.\| 00000120 7c 66 31 d2 66 f7 34 88 d1 31 d2 66 f7 74 04 3b ||f1.f.4..1.f.t.;| 00000130 44 08 7d 37 fe c1 88 c5 30 c0 c1 e8 02 08 c1 88 |D.}7....0.......| 00000140 d0 5a 88 c6 bb 00 70 8e c3 31 db b8 01 02 cd 13 |.Z....p..1......| 00000150 72 1e 8c c3 60 1e b9 00 01 8e db 31 f6 bf 00 80 |r...`......1....| 00000160 8e c6 fc f3 a5 1f 61 ff 26 5a 7c be 86 7d eb 03 |......a.&Z|..}..| 00000170 be 95 7d e8 34 00 be 9a 7d e8 2e 00 cd 18 eb fe |..}.4...}.......| 00000180 47 52 55 42 20 00 47 65 6f 6d 00 48 61 72 64 20 |GRUB .Geom.Hard | 00000190 44 69 73 6b 00 52 65 61 64 00 20 45 72 72 6f 72 |Disk.Read. Error| 000001a0 0d 0a 00 bb 01 00 b4 0e cd 10 ac 3c 00 75 f4 c3 |...........<.u..| 000001b0 00 00 00 00 00 00 00 00 00 00 00 00 33 cc 00 20 |............3.. | 000001c0 21 00 82 eb 2f fc 00 08 00 00 00 f8 3d 00 80 eb |!.../.......=...| 000001d0 30 fc 83 fe ff ff 00 00 3e 00 00 e8 c1 03 00 00 |0.......>.......| 000001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa |..............U.| 00000200
Not much of the information, but it is Master_boot_record, so we can see:
- Some messages at the end of the primary boot loader program (first 0x1BE=446 bytes)
- Some data describes two of four possible partitions (starting from 0x1be and 0x1ce)
Boot block signature 55 aa at theend of the block
All the fiels needed for per-system boot are groped in /boot directory:
[root@comp-core-i7-8750h-8b78f7 ~]# ls /boot/ config-5.4.28-std-def-alt1 memtest-5.01.bin elf-memtest-5.01 splash grub System.map-5.4.28-std-def-alt1 initrd-5.4.28-std-def-alt1.img vmlinuz initrd.img vmlinuz-5.4.28-std-def-alt1 initrd-std-def.img vmlinuz-std-def [root@comp-core-i7-8750h-8b78f7 ~]# ls /boot/grub/ fonts grub.cfg grubenv i386-pc locale themes unifont.pf2 [root@comp-core-i7-8750h-8b78f7 ~]# ls /boot/grub/i386-pc/ acpi.mod gcry_tiger.mod password.mod … gcry_rsa.mod part_msdos.mod xnu_uuid.mod … gcry_sha512.mod parttool.mod zfs.mod
- GRUB is great! There are almost 100 pluggable extensions, that GRUB can load en use on bare hardware, without any OS support.
Also note some variations of vmlinuz and initrd; they're, of course, kernels and initial virtual disks The configuration file of GRUB is complex and large.
[root@comp-core-i7-8750h-8b78f7 ~]# cat /boot/grub/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/sysconfig/grub2
…
insmod part_msdos
insmod ext2
…
menuentry 'Simply Linux 9.0' --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-simple-311e5f8a-0a5d-45b4-961f-466b538d3e57' {
savedefault
load_video
insmod gzio
insmod part_msdos
insmod ext2
set root='hd0,msdos2'
…
echo 'Loading Linux vmlinuz ...'
linux16 /boot/vmlinuz root=UUID=311e5f8a-0a5d-45b4-961f-466b538d3e57 ro resume=/dev/disk/by-uuid/f5636c63-6d23-4b03-872b-db787212b7a3 panic=30 quiet splash
echo 'Loading initial ramdisk ...'
initrd16 /boot/initrd.img
}As you see, the file is autogenerated from various sources — automatic OS prober, current OS configs, autmatic hardware detector and so on
Note insmod commands that loaded specific partitioning and filesystem GRUB modules
- Note root filesystem designation
Note linux16 and initrd16 commands that point just to the appropriate files
By the way! What happened to out edited boot configuration? Where is that «gibberish» stuff?
[root@comp-core-i7-8750h-8b78f7 ~]# cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz root=UUID=311e5f8a-0a5d-45b4-961f-466b538d3e57 ro resume=/dev/disk/by-uuid/f5636c63-6d23-4b03-872b-db787212b7a3 panic=30 quiet splash gibberish=1
- Here it is!
What lies st /boot:
[root@comp-core-i7-8750h-8b78f7 ~]# ls -l /boot total 19844 -rw-r--r-- 1 root root 230253 Mar 25 20:49 config-5.4.28-std-def-alt1 -rw-r--r-- 1 root root 184380 Jan 9 2017 elf-memtest-5.01 drwxr-xr-x 6 root root 4096 Apr 23 11:42 grub -rw------- 1 root root 10563947 Apr 22 21:26 initrd-5.4.28-std-def-alt1.img lrwxrwxrwx 1 root root 30 Apr 22 21:26 initrd.img -> initrd-5.4.28-std-def-alt1.img lrwxrwxrwx 1 root root 30 Apr 22 21:26 initrd-std-def.img -> initrd-5.4.28-std-def-alt1.img -rw-r--r-- 1 root root 182704 Jan 9 2017 memtest-5.01.bin drwxr-xr-x 3 root root 4096 Apr 3 00:37 splash -rw-r--r-- 1 root root 3579967 Mar 25 20:49 System.map-5.4.28-std-def-alt1 lrwxrwxrwx 1 root root 27 Apr 22 21:26 vmlinuz -> vmlinuz-5.4.28-std-def-alt1 -r--r--r-- 1 root root 5551160 Apr 22 21:26 vmlinuz-5.4.28-std-def-alt1 lrwxrwxrwx 1 root root 27 Apr 22 21:26 vmlinuz-std-def -> vmlinuz-5.4.28-std-def-alt1
All vmlinuz and initrd variations are happen to be symbolic links
⇒ We have just one kernel/initrd pair installed. Can be more
One of boot menu says Memtest. memtest-5.01.bin is kernel-style binary, which you can load instead the kernel to test memory without OS
config-… file is kernel configuretion. Serves only informational purpose.
What's initrd.img?
[root@comp-core-i7-8750h-8b78f7 ~]# file /boot/initrd.img /boot/initrd.img: symbolic link to `initrd-5.4.28-std-def-alt1.img' [root@comp-core-i7-8750h-8b78f7 ~]# file /boot/initrd-5.4.28-std-def-alt1.img /boot/initrd-5.4.28-std-def-alt1.img: gzip compressed data, last modified: Wed Apr 22 21:26:49 2020, max compression, from Unix [root@comp-core-i7-8750h-8b78f7 ~]# zcat /boot/initrd-5.4.28-std-def-alt1.img > initrd.ungzipped.img [root@comp-core-i7-8750h-8b78f7 ~]# file initrd.ungzipped.img initrd.ungzipped.img: ASCII cpio archive (SVR4 with no CRC)
- it's symbolic link to a certain initrd image
which is, in fact, gzip-ped cpio archive
So that is filesystem that can reside in memory and kernel can mount it
What's archived in initrd?
[root@comp-core-i7-8750h-8b78f7 ~]# cpio -i -t -v -I initrd.ungzipped.img … -rwxr-xr-x 1 root root 979 Mar 6 13:08 ./init drwxr-xr-x 2 root root 0 Apr 22 21:26 ./lib64 -rw-r--r-- 1 root root 100752 Nov 5 06:52 ./lib64/libgcc_s.so.1 … drwxr-xr-x 2 root root 0 Apr 22 21:26 ./bin -rwxr-xr-x 1 root root 18640 Feb 12 2019 ./bin/loadkeys … lrwxrwxrwx 1 root root 8 Apr 22 21:26 ./bin/rm -> busybox … drwxr-xr-x 2 root root 0 Apr 22 21:26 ./lib/modules/5.4.28-std-def-alt1 … -rw------- 1 root root 7297 Mar 25 20:49 ./lib/modules/5.4.28-std-def-alt1/kernel/lib/crc-ccitt.ko -rw------- 1 root root 5985 Mar 25 20:49 ./lib/modules/5.4.28-std-def-alt1/kernel/lib/crc16.ko … -rw------- 1 root root 1498505 Mar 25 20:49 ./lib/modules/5.4.28-std-def-alt1/kernel/fs/ext4/ext4.ko … -rw------- 1 root root 101353 Mar 25 20:49 ./lib/modules/5.4.28-std-def-alt1/kernel/drivers/scsi/sd_mod.ko -rw------- 1 root root 492329 Mar 25 20:49 ./lib/modules/5.4.28-std-def-alt1/kernel/drivers/scsi/scsi_mod.ko … drwxr-xr-x 2 root root 0 Apr 22 21:26 ./etc -rw-r--r-- 1 root root 26 Apr 22 21:26 ./etc/host.conf … brw-r--r-- 1 root root 1, 1 Apr 22 21:26 ./dev/ram crw-rw-rw- 1 root root 1, 3 Apr 22 21:26 ./dev/null crw-rw-rw- 1 root root 1, 5 Apr 22 21:26 ./dev/zero crw-rw-rw- 1 root root 1, 7 Apr 22 21:26 ./dev/full crw-rw-rw- 1 root root 1, 8 Apr 22 21:26 ./dev/random crw-rw-rw- 1 root root 4, 0 Apr 22 21:26 ./dev/systty crw-rw-rw- 1 root root 4, 0 Apr 22 21:26 ./dev/tty0 crw-rw-rw- 1 root root 4, 1 Apr 22 21:26 ./dev/tty1 crw-rw-rw- 1 root root 5, 0 Apr 22 21:26 ./dev/tty crw------- 1 root root 5, 1 Apr 22 21:26 ./dev/console crw-rw-rw- 1 root root 5, 2 Apr 22 21:26 ./dev/ptmx 48627 blocks
- It's rather complete linux OS, extremely minimal and capable only to find the appropriate device to mount root directory on, and start the OS
- It has binary utilities
Some of them are BusyBox, some are not
- It has libraries neede for binaries
It has /dev/ with some common devices (like /dev/null) already there
It has /etc /usr/share and other informational stuff
It has some kernel modules, which put there solely to recognize bootable device and filesystem
Let's see, if any of these modules, say, scsi_mod is used
[root@comp-core-i7-8750h-8b78f7 ~]# lsmod Module Size Used by usbhid 53248 0 hid 139264 2 usbhid,hid_generic … i2c_piix4 24576 0 … vboxsf 86016 0 vboxguest 356352 6 vboxsf … snd 90112 11 snd_seq,snd_seq_device,snd_intel8x0,snd_timer,snd_ac97_codec,snd_pcm,snd_rawmidi … scsi_mod 249856 3 sd_mod,libata,sr_mod …
- Linux monolith kernel is not so monolith after all!
And here is our scsi_mod.ko:
[root@comp-core-i7-8750h-8b78f7 ~]# cpio -i -t -v -I initrd.ungzipped.img | grep scsi_mod -rw------- 1 root root 492329 Mar 25 20:49 ./lib/modules/5.4.28-std-def-alt1/kernel/drivers/scsi/scsi_mod.ko 48627 blocks
IRL
Legacy free, more complex GUID_Partition_Table scheme
Unique UUID for all partition types
- Unlimited partitions
EFI instead of BIOS (firmware!)
- Can check bootable image signature (secure boot)
- Needs a special partition on disk (strangely, ancient FAT filesystem is needed)
- Run special EFI applications from that partition
So we do not need to put secondary loader somewhere into hidden spaces of the disk
- Can be other type of application, even primitive games
Provide OS-independent driver API
Varies from vendor to vendor dramatically
