Build and Run your own kernel in 60 mins

I will teach you how to make a lot of mon..er.. No, in this we will have our own running kernel within the next 60 mins (likely lesser). If you do it right, most of these 60mins will be spent like this:

Recently, I was asked by a friend about how to compile Linux Kernel. Shortly after, I realized that he assumed this needed a really complex system with a very sophisticated configuration. Although he was running a fairly new system he felt it needed a lot more “setup time” to get his kernel compiled and running. I found that most people are under the impression that you had to have this complex setup to be able to start “hacking the kernel”. I challenged him that we could find a way to be able to boot and run a kernel in under 60 mins! For the impatient, I have kept this entry short enough but if you are feeling extra-impatient follow the instructions inside

this block

Steps:

Install QEMU and KVM

# Ex: Debian systems
sudo apt-get install qemu-kvm

Get Linux kernel:

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
$ export KERNEL_ROOT=$PWD/linux

Build kernel config

I have to mention that in my experience as a kernel n00b, I used to get stuck in this step more often than I should have. I’d end up double guessing myself about whether I’d need a config option that enables a cool new feature. This was simply based on the short description in config menu. Many times, I’d drown in the kernel documentation rabbit-hole, although kernel is one of the better documented software projects it is suited well to lookup something dictionary-style, that’s for a different post. The kernel ships with a default config that is good enough to compile all the necessary features for booting a kernel while leaving the extra-luggage out by the door. Simply:

cd $KERNEL_ROOT/arch/<YOUR_MACHINE_ARCH_TYPE>/configs/

You should make sure you know what your machine’s “YOUR_MACHINE_ARCH_TYPE” is. Not sure?

uname -m

Even if you are running x86_64 the config files are placed under

ls $KERNEL_ROOT/arch/x86/configs/
# no suffix '_64' in dirname

In my case, I’m running a x86_64 machine for desktop. So,

# I will choose x86_64_defconfig
cd $KERNEL_ROOT
make x86_64_defconfig

Typically, I add a list of my favorite options to the kernel that I leave turned on by default in my kernels. Ex: CONFIG_FUNCTION_GRAPH_TRACER, CONFIG_PREEMPT_TRACER, CONFIG_IRQSOFF_TRACER

Compile

This should only take a few seconds. It’s already time to actually compile our shiny new kernel:

make -jNN
# NN = number of cores. Ex: make -j4 on a 4-core machine

Compile Success? Verify.

Ensure that the kernel compilation succeeded by looking for bzImage under KERNEL_ROOT/arch/x86/boot/. I enourage you to have this in your shell to save some typing xlater.

export MY_BZIMG=$KERNEL_ROOT/arch/x86/boot/bzImage

This will take several minutes for the first time, say about 6-7 mins in a normal case for an idle 4-core machine (for assumptions within reason)

Buildroot

Now we have the kernel already compiled. However, remember kernel doesn’t run all by itself. We will need a shell running in user space for which you will need a sandbox userspace. One of the quickest options is to build an initial ramdisk, which basically provides a temporary root fs in memory. The best option here is buildroot, you don’t have to click and get lost in the ocean of details (yet!). Here’s an intro from it’s official documentation: “Buildroot is a tool that simplifies and automates the process of building a complete Linux system for an embedded system, using cross-compilation.”. OK! so you know enough to follow up on the details later. For now we have got to start compiling.

git clone git://git.buildroot.net/buildroot
cd buildroot
make menuconfig

# Select these two options

Target options -> Target Architecture -> ex: x86_64

Filesystem Images ->  cpio the root filesystem (for use as an initial RAM filesystem)

In the menu config for buildroot, there are a ton of config options that might seem interesting and you would want to explore them one by one, because it’s magic! But we still are quite a way away from being able to run our kernel, yet we are close!

make

To ensure buildroot compiled successfully, look for rootfs.cpio under $BUIDLROOT_ROOT/output/images/. I encourage you to do this

export BUIDLROOT_PKG=$BUILDROOT_ROOT/output/images/rootfs.cpio

If you’re missing any dependencies you should be able to quickly figure out by searching stackoverflow or by installing packages directly. Once buildroot has started compiling then you can take a break for several minutes. This is, easily, the most time-consuming step of the 60 mins we allocated for this activiity. If everything went well, you should be very close to having your first kernel compiled on your machine. Now we have a compressed image of the kernel binaries and a filesystem package for running a basic ramdisk image with a minimal userspace binaries to talk to your kernel! The following part is borrowed from LKP test script and Laura’s blog (You should check it out, it’s awesome!): Copy this script into, say, start-qemu.sh

#!/bin/bash

kernel=$1
initrd=$2

if [ -z $kernel ]; then
    echo "pass the kernel argument"
    exit 1
fi

if [ -z $initrd ]; then
    echo "pass the initrd argument"
    exit 1
fi

kvm=(
    qemu-system-x86_64
    -enable-kvm
    -cpu kvm64,+rdtscp
    -kernel $kernel
    -m 300
    -device e1000,netdev=net0
    -netdev user,id=net0
    -boot order=nc
    -no-reboot
    -watchdog i6300esb
    -rtc base=localtime
    -serial stdio
    -vga qxl
    -initrd $initrd
    -spice port=5930,disable-ticketing
    -s
)

append=(
    hung_task_panic=1
    earlyprintk=ttyS0,115200
    systemd.log_level=err
    debug
    apic=debug
    sysrq_always_enabled
    rcupdate.rcu_cpu_stall_timeout=100
    panic=-1
    softlockup_panic=1
    nmi_watchdog=panic
    oops=panic
    load_ramdisk=2
    prompt_ramdisk=0
    console=tty0
    console=ttyS0,115200
    vga=normal
    root=/dev/ram0
    rw
    drbd.minor_count=8
)

"${kvm[@]}" --append "${append[*]}"

Finally, It’s time to run!

After our 60mins, I encourage you to play with these options. But first let’s try to get our system running:

./start-qemu.sh $MY_BZIMG $BUILDROOT_PKG

This should boot your kernel with a print out of intial kernel logs on your screen and end at a login prompt (user: root, no passwd)

That’s it! you have successfully downloaded the absolute latest and greatest kernel from the source tree of Linus Torvalds and compiled it. This is really the basic building block of kernel hacking. Save this setup, play around with it until you have exactly what you are looking for, tweak kernel config options, add your own prints in the kernel for fun!

Caveat: Are you screaming “Woohoo! I never felt this powerful before”? or “Nothing works! This entry sucks”? Either way, this is just the start of something that can repay rich dividends for a programmer. Play around with everything mentioned above until it works for you. There’s no shorter route. There’s no substitute for your own work. Above steps assume that you are using a reasonably modern machine running Linux to compile. Also, if you don’t make it in 60 mins don’t sue me please.

Time to start growing a beard on your necks! Wait, what?!