Getting Started with NUX

This is a short guide that will help you compile nux, run the demo, and understand its log.

Building NUX

Before we start building NUX, we need to make sure we are running on the same GNU toolchain.

Compile the toolchain

The first step is to compile the toolchain. NUX uses embedded elf GNU toolchain, but to avoid conflicts between GCC version and what NUX supports, it's highly reccomended to use gcc_toolchain_build (available on GitHub).

The following command will fetch gcc_toolchain_build, and build all the toolchain used by the architectures supported by NUX.

git clone https://github.com/glguida/gcc_toolchain_build
cd gcc_toolchain_build
make populate
make amd64-unknown-elf-gcc
make i686-unknown-elf-gcc
make riscv64-unknown-elf-gcc
export PATH=$PWD/install/bin
cd ..

Note: compiling this takes a while.

Compile NUX and example kernel

Once to toolchain is compiled and loaded in the PATH, compiling NUX is as simple as running configure and make.

git clone https://github.com/glguida/nux
cd nux
git submodule update --init --recursive
mkdir build
cd build
../configure ARCH=i386
make -j

Note ARCH=i386, this can be changed to ARCH=amd64 or ARCH=riscv64 to generate NUX for these architectures.

Run the demo.

Once the compilation has finished, you can run the demo. example is an example NUX kernel and userspace. To run it, use the following commands:

cd example
make qemu

You should see something similar to this (the demo will change from time to time):

The first thing to boot will be APXH (the NUX bootloader). Currently supports EFI and multiboot v1 on Intel Architectures, and EFI and SBI in RISCV64.

This will load the kernel and userspace binaries and set things up for kernel execution:

Booting from ROM..Multiboot memory map:
0000000000000000:000000000009fc00:1
000000000009fc00:0000000000000400:2
00000000000f0000:0000000000010000:2
0000000000100000:0000000007ee0000:1
0000000007fe0000:0000000000020000:2
00000000fffc0000:0000000000040000:2
000000fd00000000:0000000300000000:2

APXH started.

multiboot initialised: brk at 001c1000
Kernel payload AMD64 ELF at addr 0x1133c4 (629456 bytes)
PAT TABLE: WB Entry at 0
PAT Table: UC Entry at 3
PAT Table: WC Entry at 7
Using PAE64 paging (CR3: 001c1000, NX: 1).
Setting ffff800000024280 <- 0 (u:0, w:1, x: 0, 10352 bytes)
Boot Information area at ffff800000040000 (size: 4096d).
TOP PT Alloc VA area at ffff800005147000 (size: 549755813888).
TOP PT Alloc VA area at ffff808005147000 (size: 549755813888).
PT Alloc VA area at ffff810005147000 (size: 268435456).
Physmap VA area at ffff818000000000 (size: 549755813888).
S-Tree at ffff800000141000 (size: 16777216).
Populating size 4184 (order: 0)
Maximum reached.
Maximum reached.
Maximum reached.
Framebuffer Map at ffff800001141000 (size: 67108864).
Region Map at ffff800000041000 (size: 1048576).
Size of area: 112 = 7 * 16
Linear Map at ffff820000000000 (size: 549755813888).
Wrote 1c1003 at 0x1c1820
Kernel entry: ffff800000004fb0
User payload AMD64 ELF at addr 0x1acea8 (78200 bytes)
Setting 00407008 <- 0 (u:1, w:1, x: 0, 1368 bytes)
User entry: 400293
Searching for RSDP.
Scanning EBDA at addr 9fc00 (0x9fc00)
Scanning 0xE0000 -> 0xFFFFF
Checking Checksum
Revision: 0
RSDP found at 0xf52a0
tramp is 49c000 (e7ff)
Mapping at va ffff800000004fb0 PA 1c9fb0 (p:0, w:0, x:1)
mapping in 49b000 ffff800000004fb0 at 1c9fb0
CR4: 00000000 -> 00000020.
CR3: 00000000 -> 0049b000.
EFER: 0000000000000800 -> 0000000000000900.
CR0: 00000011 -> 80010011.

Once the kernel is loaded, APXH will start it:

AMD64 HAL booting from APXH.
NUX library (nux) 0.0
Copyright (C) 2019 Gianluca Guida

Initializing PFN boot cache.
Lowest physical page free:  00000001.
Highest physical page free: 00007fdf.
Memory available: 127632 Kb.
KVA Area from ffff808005147000 to ffff810005147000
PFN Cache from 0xffff810005147000 to 0xffff810015147000 (65536 entries)
RSDP: f52a0
TABLE: 'RSD PTR ' [BOCHS ] rev: 0
SDT found at addr 7fe1c40
loaded table 'RSDT' [BOCHS  BXPC     rev1]
loaded table 'FACP' [BOCHS  BXPC     rev1]
TABLE 'FACP' [BOCHS  BXPC     rev1]
loaded table 'APIC' [BOCHS  BXPC     rev1]
TABLE 'APIC' [BOCHS  BXPC     rev1]
loaded table 'HPET' [BOCHS  BXPC     rev1]
TABLE 'HPET' [BOCHS  BXPC     rev1]
loaded table 'WAET' [BOCHS  BXPC     rev1]
TABLE 'WAET' [BOCHS  BXPC     rev1]
RDST table at pa 7fe1c40
APIC table at pa 7fe1b68
HPET table at pa 7fe1be0
loaded table 'APIC' [BOCHS  BXPC     rev1]
ACPI MADT LAPIC 00 00 00000001
ACPI MADT IOAPIC 00 fec00000 00
LAPIC PA: fee00000 VA: 0xffff808005158000
IOAPIC: 00 PA: fec00000 VA: 0xffff808005159000 IRQ:00 PINS: 24
ACPI MADT INTOVR BUS 00 IRQ: 00 GSI: 02 FL: 0000
ACPI MADT INTOVR BUS 00 IRQ: 05 GSI: 05 FL: 0000
ACPI MADT INTOVR BUS 00 IRQ: 09 GSI: 09 FL: 0000
ACPI MADT INTOVR BUS 00 IRQ: 10 GSI: 10 FL: 0000
ACPI MADT INTOVR BUS 00 IRQ: 11 GSI: 11 FL: 0000
ACPI MADT LAPICNMI LINT1 FL:0000 PROC:255
loaded table 'HPET' [BOCHS  BXPC     rev1]
HPET Found at fed00000, mapped at 0xffff80800515a000
HPET period: 989680x, counters: 3
HPET counter frequency: 100000000 Hz
Using Legacy Routing.

CPUs found: 0[0] 
Waiting for APs to boot..done.

After NUX is fully initialised and all the secondary CPUs have started, kernel's main function will be called:

Hello, {NULL} (237a08)!RAX: 0000000000000000 RBX: 0000000000000000
RCX: 0000000000000000 RDX: 0000000000000000
RDI: 0000000000000000 RSI: 0000000000000000
RBP: 0000000000000000 RSP: 0000000000000000
R8 : 0000000000000000 R9 : 0000000000000000
R10: 0000000000000000 R11: 0000000000000000
R12: 0000000000000000 R13: 0000000000000000
R14: 0000000000000000 R15: 0000000000000000
GS:  0000000000000000 FS:  0000000000000000
 CS: 002b     RIP: 0000000000400293 RFL: 0000000000000202
 SS: 0023     RSP: 0000000000000000
CR2: 0000000000000000 err: 00000000
400000 - 492001(493005)
402000 - 492011(495005)
404000 - 492021(8000000000497007)
406000 - 492031(8000000000499007)
IPI!

In this case, the kernel issues an IPI that will start the user space. And the userspace will execute.

Hello from userspace, NUX!
SYSC0 test passed.
SYSC1 test passed.
SYSC2 test passed.
SYSC3 test passed.
SYSC4 test passed.
SYSC5 test passed.
SYSC6 test passed.
User exited with error code: 42

Here the user space exited, so that all CPUs are idle. What follows is a timer interrupt, set up by the example kernel, that gathers performance counters and prints them.

TMR: 1008179000 us
INVALID/IDLE FRAME
ctr: pnux_entry_ipi      	               1
ctr: pnux_entry_irq      	               0
ctr: pnux_entry_timer    	               1
ctr: pnux_entry_nmi      	               0
ctr: pnux_entry_exception	               0
ctr: pnux_entry_pagefault	               0
ctr: pnux_entry_syscall  	              35
msr: syscalls_nsecs      	              34
    min/avg/max [ 3000 / 80491 / 356000 ]
msr: syscalls_cycles     	              34
    min/avg/max [ 4000 / 83698 / 360000 ]
TMR: 2009613000 us
INVALID/IDLE FRAME
ctr: pnux_entry_ipi      	               1
ctr: pnux_entry_irq      	               0
ctr: pnux_entry_timer    	               2
ctr: pnux_entry_nmi      	               0
nctr: pnux_entry_exception	               0
ctr: pnux_entry_pagefault	               0
ctr: pnux_entry_syscall  	              35
msr: syscalls_nsecs      	              34
    min/avg/max [ 3000 / 80491 / 356000 ]
msr: syscalls_cycles     	              34
    min/avg/max [ 4000 / 83698 / 360000 ]

[...]

That's it! Now you can experiment with example/kern/main.c and example/user/main.c, which are the sources for this demo.