Understanding UBSAN

Undefined Behavior Sanitizer (UBSAN) is a powerful runtime checker that helps detect undefined behavior in C and C++ code. The Linux kernel integrates UBSAN to catch subtle memory and arithmetic errors that could otherwise lead to security vulnerabilities or unpredictable behavior.

What is Undefined Behavior?

Undefined behavior (UB) occurs when a program executes an operation that is not defined by the C standard. Some common examples include:

Signed integer overflow
Use of uninitialized memory
Out-of-bounds array access
Dereferencing NULL pointers
Misaligned memory access

Since undefined behavior can lead to inconsistent program execution, UBSAN is used to catch these issues at runtime.

Enabling UBSAN in the Linux Kernel

To enable UBSAN in your kernel build, use the following configuration options:

CONFIG_UBSAN=y
CONFIG_UBSAN_TRAP=y # Optional: Stops execution on UB detection

If you want to include only specific UBSAN checks, you can select:

CONFIG_UBSAN_BOUNDS=y  # Detects out-of-bounds array access
CONFIG_UBSAN_SHIFT=y   # Detects invalid shift operations
CONFIG_UBSAN_DIV_ZERO=y # Detects division by zero

Rebuild and install the kernel, then boot into it.

Live Example: Detecting Integer Overflow

Let’s write a simple kernel module that triggers an integer overflow:

#include <linux/module.h>
#include <linux/kernel.h>
 
static int __init ubsan_test_init(void) {
    int a = INT_MAX;
    int b = 1;
    int c = a + b;  // This causes signed integer overflow
    pr_info("UBSAN test: a=%d, b=%d, c=%d\n", a, b, c);
    return 0;
}
 
static void __exit ubsan_test_exit(void) {
    pr_info("UBSAN test module exiting\n");
}
 
module_init(ubsan_test_init);
module_exit(ubsan_test_exit);
 
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Security Researcher");
MODULE_DESCRIPTION("UBSAN Test Module");

Compiling and Running

Compile the module and insert it into the kernel:

make -C /lib/modules/$(uname -r)/build M=$(pwd) modules
sudo insmod ubsan_test.ko

Expected UBSAN Output

If UBSAN is enabled, you’ll see an error message similar to:

[  105.123456] UBSAN: Undefined behavior in ubsan_test.c:7:9
[  105.123467] signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
[  105.123479] CPU: 0 PID: 1234 Comm: insmod Not tainted 5.10.0 #1
[  105.123490] Call Trace:
[  105.123501]  dump_stack+0x6d/0x88
[  105.123512]  ubsan_test_init+0x2f/0x1000 [ubsan_test]
[  105.123523]  do_one_initcall+0x48/0x1a0
[  105.123534]  ? do_init_module+0x21/0x220
[  105.123545]  do_init_module+0x50/0x220
[  105.123556]  load_module+0x1e92/0x21a0
[  105.123567]  __do_sys_finit_module+0xad/0x110
[  105.123578]  do_syscall_64+0x33/0x40
[  105.123589]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  105.123600] RIP: 0033:0x7f2b3c9e515d

If CONFIG_UBSAN_TRAP=y is set, the system will panic instead of continuing execution.

Cleaning Up

To remove the module:

sudo rmmod ubsan_test

Expected output:

[  120.456789] UBSAN test module exiting

Why UBSAN Matters for Security

UBSAN helps find critical memory safety issues and logic errors that attackers might exploit. It plays a vital role in hardening the kernel against:

Integer overflows leading to privilege escalation
Out-of-bounds reads/writes causing memory corruption
Type mismatches that can lead to unexpected execution paths

By enabling UBSAN in testing environments, developers can catch and fix undefined behavior before it becomes a security risk.

Conclusion

UBSAN is a valuable tool for detecting undefined behavior in the Linux kernel. By enabling it in kernel builds and leveraging it during fuzz testing (e.g., with syzkaller), developers can proactively identify and fix critical security bugs.

Table of Contents