Slackware kernel (re)compilation

I needed to rebuild my kernel with NUMA support ( for VPP installation )

Stock Slackware kernel does not have NUMA built-in, let’s fix this !

No matter the option, the (re)built method is the same 😉

Kernel sources are installed in /usr/src/

:~ # ls /usr/src/ -lastrh
total 97M
 97M -rw-r--r--  1 root root  97M Jan  5 15:51 linux-4.14.12.tar.xz
4.0K drwxr-xr-x 22 root root 4.0K Jan 30 00:59 linux-4.14.12/
4.0K drwxr-xr-x 17 root root 4.0K Feb  4 00:12 ../
4.0K drwxr-xr-x 25 root root 4.0K Feb  4 02:26 linux-4.14.17/
   0 lrwxrwxrwx  1 root root   13 Feb  5 19:33 linux -> linux-4.14.17/
4.0K drwxr-xr-x  4 root root 4.0K Feb  5 19:33 ./
23:42 root@ws1:~ # 

Let’s save the stock kernel config file, not that I use the `linux` dir directly, as it’s a symlink to the latest linux-4.14.17 dir.

:~ # cp /usr/src/linux/.config /usr/src/linux/.config.stock

Get the running kernel config, (though I could have used the previous .config file ) . As a good habit, I always run `make oldconfig` to show if there is newer option . I doubt to find some new option here as I’m recompiling the running slackware kernel… but when compiling a new kernel version, new option can be added. You night want to check them out and set them as built-in/module/not set.

:~ # zcat /proc/config.gz > /usr/src/linux/.config
:~ # cd /usr/src/linux
:/usr/src/linux # 
:/usr/src/linux # 
:/usr/src/linux # make oldconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/basic/bin2c
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/
  HOSTLD  scripts/kconfig/conf
scripts/kconfig/conf  --oldconfig Kconfig
# configuration written to .config
:/usr/src/linux #

Then modify options  with `make menuconfig` ( it’s a ncurses interface ).

  • General setup -> add as built-in Memory placement aware NUMA scheduler
  • Processor type and features -> add as built-in Numa Memory Allocation and Scheduler Support
  • Processor type and features -> add as built-in NUMA emulation
  • Also as explained in the official doc, if you rebuilt an already installed  kernel, in order to avoid messing up with kernel modules for that kernel, you need to change the LOCAL_VERSION under: General setup -> Local version ( I used a suffix -vpp )
:/usr/src/linux # make menuconfig
  HOSTCC  scripts/kconfig/mconf.o
  HOSTCC  scripts/kconfig/lxdialog/checklist.o
  HOSTCC  scripts/kconfig/lxdialog/util.o
  HOSTCC  scripts/kconfig/lxdialog/inputbox.o
  HOSTCC  scripts/kconfig/lxdialog/textbox.o
  HOSTCC  scripts/kconfig/lxdialog/yesno.o
  HOSTCC  scripts/kconfig/lxdialog/menubox.o
  HOSTLD  scripts/kconfig/mconf
scripts/kconfig/mconf  Kconfig
configuration written to .config

*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.

:/usr/src/linux #

Let’s make the kernel and modules

:/usr/src/linux # make all
 HOSTCC scripts/basic/fixdep
 CHK include/config/kernel.release
 CHK include/generated/uapi/linux/version.h
 HOSTCC scripts/basic/bin2c
 DESCEND objtool
 HOSTCC /usr/src/linux-4.14.17/tools/objtool/fixdep.o
<--- output suppressed--->
 LD [M] sound/usb/usx2y/snd-usb-us122l.ko
 LD [M] sound/usb/usx2y/snd-usb-usx2y.ko
 LD [M] sound/x86/snd-hdmi-lpe-audio.ko
 LD [M] virt/lib/irqbypass.ko
:/usr/src/linux #

then install the modules in /lib/modules

:/usr/src/linux # make modules_install
  INSTALL arch/x86/crypto/blowfish-x86_64.ko
  INSTALL arch/x86/crypto/camellia-aesni-avx-x86_64.ko
<--- output suppressed --->
 INSTALL sound/x86/snd-hdmi-lpe-audio.ko
  INSTALL virt/lib/irqbypass.ko
  DEPMOD  4.14.17-vpp
:/usr/src/linux #

Note that a 4.14.17-vpp directory has been created 😉 ( that means that is you have forgotten to modify LOCAL_VERSION, you are likely to overwrite actual kernel modules.

:~ # ls /lib/modules/
4.14.15/  4.14.17/  4.14.17-vpp/
:~ #

Let’s put the kernel in place, copying kernel/config/ file in /boot

:/usr/src/linux # cp arch/x86/boot/bzImage /boot/vmlinuz-vpp-4.14.17
:/usr/src/linux # cp .config /boot/.config-vpp-4.14.17
:/usr/src/linux # cp /boot/
:/usr/src/linux #
:/usr/src/linux # ls -lastr /boot/
0 lrwxrwxrwx 1 root root 23 Feb  5 19:27 /boot/ ->
:/usr/src/linux # 
:/usr/src/linux # rm /boot/
:/usr/src/linux # ln -s /boot/ /boot/
:/usr/src/linux #
:/usr/src/linux # ls -lastr /boot/
0 lrwxrwxrwx 1 root root 28 Feb  6 20:03 /boot/ -> /boot/
:/usr/src/linux #

Let’s create the initrd

:/usr/src/linux # /usr/share/mkinitrd/ -k 4.14.17-vpp
# revision 1.45
# This script will now make a recommendation about the command to use
# in case you require an initrd image to boot a kernel that does not
# have support for your storage or root filesystem built in
# (such as the Slackware 'generic' kernels').
# A suitable 'mkinitrd' command will be:

mkinitrd -c -k 4.14.17-vpp -f ext4 -r /dev/sda1 -m xhci-pci:ohci-pci:ehci-pci:xhci-hcd:uhci-hcd:ehci-hcd:hid:usbhid:i2c-hid:hid_generic:hid-cherry:hid-logitech:hid-logitech-dj:hid-logitech-hidpp:hid-lenovo:hid-microsoft:hid_multitouch:jbd2:mbcache:ext4 -u -o /boot/initrd.gz

Please note that I’ve specified the -k option with kernel version set to 4.14.17-vpp instead of just 4.14.17. It’s because mkinitrd will look in the corresponding directory for kernel modules, ie in /lib/modules/$kernelVersion … here we want mkinitrd to look in /lib/modules/4.14.17-vpp/

:/usr/src/linux # mkinitrd -c -k 4.14.17-vpp -f ext4 -r /dev/sda1 -m xhci-pci:ohci-pci:ehci-pci:xhci-hcd:uhci-hcd:ehci-hcd:hid:usbhid:i2c-hid:hid_generic:hid-cherry:hid-logitech:hid-logitech-dj:hid-logitech-hidpp:hid-lenovo:hid-microsoft:hid_multitouch:jbd2:mbcache:ext4 -u -o /boot/initrd-4.14.17-vpp.gz
OK: /lib/modules/4.14.17-vpp/kernel/drivers/usb/host/xhci-hcd.ko added.
OK: /lib/modules/4.14.17-vpp/kernel/drivers/usb/host/xhci-pci.ko added.
<--- output suppressed --->
OK: /lib/modules/4.14.17-vpp/kernel/fs/mbcache.ko added.
OK: /lib/modules/4.14.17-vpp/kernel/fs/ext4/ext4.ko added.
49081 blocks
/boot/initrd-4.14.17-vpp.gz created.
Be sure to run lilo again if you use it.
:/usr/src/linux #

Let’s add the corresponding section in lilo.conf

image = /boot/vmlinuz-vpp-4.14.17                                                                                                         
  initrd = /boot/initrd-4.14.17-vpp.gz                                                                                                    
  root = /dev/sda1                                                                                                                        
  label = vpp-4.14.17                                                                                                                     

And run LILO

:~ # lilo -v
LILO version 24.2 (released 22-November-2015)
<--- output suppressed --->

Boot image: /boot/vmlinuz-vpp-4.14.17
Mapping RAM disk /boot/initrd-4.14.17-vpp.gz
The initial RAM disk will be loaded in the high memory above 16M.
Added vpp-4.14.17  +

<--- output suppressed --->
Writing boot sector.
/boot/boot.0800 exists - no boot sector backup copy made.
:~ #

Then reload 🙂


Once it’s back running on our new kernel, let’s check NUMA support.

:~ # uname -a
Linux 4.14.17-vpp #2 SMP Tue Feb 6 09:15:33 CET 2018 x86_64 Intel(R) Xeon(R) CPU D-1540 @ 2.00GHz GenuineIntel GNU/Linux
:~ # zcat /proc/config.gz | grep -i numa
# CONFIG_NUMA_EMU is not set
:~ # 


kernel install/compilation: :

Slackware-current FDIO VPP

Ongoing work to get VPP working on slackware-current …. draft draft draft


VPP won’t compile on Slackware ( and probably other distrib ) running on the latest GLIBC 2.27.

I’ve been able to build VPP on GLIBC 2.26.

I did not check on 14.2 which still runs 2.23.

Test Setups:

  • Slackware-current: latest kernel available is 4.14.17
  • Intel Xeon D -1540 ( 8cores/16threads @ 2.00gHz )
  • 2* 10G baseT ports based on intel X552/X557-AT ( integrated to motherboard )
  • 2* 1G baseT ports based on intel I350 ( integrated to motherboard )

Another test config:

  • Slackware-current: latest kernel available is 4.14.17
  • Intel C2750 ( 8cores/8threads @ 2.40gHz )
  • 4*1G baseT ports based on intel I354 ( integrated to motherboard )


Clone VPP repo

git clone

I took the latest stable to build VPP, v18.01 and tried to test the compilation ( try to `make` alone will print you all possible option ).

Got error about numa.h not found

:~/dev/vpp (master) # git checkout v18.01
:~/dev/vpp ((v18.01)) #
:~/dev/vpp ((v18.01)) # make test
ake -C /root/dev/vpp/build-root PLATFORM=vpp TAG=vpp vpp-install
make[1]: Entering directory '/root/dev/vpp/build-root'
@@@@ Arch for platform 'vpp' is native @@@@
@@@@ Finding source for dpdk @@@@
<---- output suppressed ---->
== Build lib/librte_compat
== Build lib/librte_eal
== Build lib/librte_eal/common
== Build lib/librte_eal/linuxapp
== Build lib/librte_eal/linuxapp/eal
  CC eal_memory.o
/root/dev/vpp/build-root/build-vpp-native/dpdk/dpdk-17.11/lib/librte_eal/linuxapp/eal/eal_memory.c:56:10: fatal error: numa.h: No such file or directory
 #include <numa.h>
compilation terminated.
make[11]: *** [/root/dev/vpp/build-root/build-vpp-native/dpdk/dpdk-17.11/mk/internal/ eal_memory.o] Error 1

Okay, I’m missing libnuma

Let’s built it from where you can find numactl – which contains libnuma. ( need probably a way to build libnuma alone ? )

Next error after libnuma install, java compiler is missing

checking for submit_job_sse in -lIPSec_MB... yes
checking for submit_job_sse in -lIPSec_MB... (cached) yes
checking for LIBRTE_MLX4_PMD in rte_config.h... no
checking for LIBRTE_MLX5_PMD in rte_config.h... no
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking /usr/lib64/jvm/bin for Java 8 compiler... no
checking /usr/lib64/jvm/include for Java 8 compiler... no
checking /usr/lib64/jvm/jre for Java 8 compiler... no
checking /usr/lib64/jvm/jvm-exports for Java 8 compiler... no
checking /usr/lib64/jvm/lib for Java 8 compiler... no
configure: error: Could not find Java 8 compiler
make[1]: *** [Makefile:639: vpp-configure] Error 1
make[1]: Leaving directory '/root/dev/vpp/build-root'
make: *** [Makefile:368: test] Error 2
:~/dev/vpp ((v18.01.1)) # 

Let’s go for JDK install:

Once installed, got the exact same error euh ? `which java` returned nothing…

I’ve seen in other java slackbuilds that you might need to reboot the box to have /etc/profile.d filled. I did check that much and decide to reboot …. and java was found ( yeah).

After 2nd though, I decided to take a look in /etc/profile.d/ and there are new files like and jdk.csh, what it bassicaly do is exporting JAVA paths -_-

I guess a source of /etc/profile.d/ would have been enough. Maybe a logout/login also as profile.d would have been called… anyway…next time 🙂

Then after restart compilation,  got another error, about `./ylwrap` file is missing, and ask you to exec `automake –add-missing` to fix. Well got some time to figure out that you need to exec it from the `src/` directory.

 lb                      yes
  memif                   yes
  pppoe                   yes
  sixrd                   yes
  nat                     yes
  stn                     yes
  l2e                     yes
@@@@ Building vpp in /root/dev/vpp/build-root/build-vpp_debug-native/vpp @@@@
make[2]: Entering directory '/root/dev/vpp/build-root/build-vpp_debug-native/vpp'
 cd /root/dev/vpp/build-data/../src && /bin/sh /root/dev/vpp/src/missing automake-1.15 --foreign error: required file './ylwrap' not found   'automake --add-missing' can install 'ylwrap'
make[2]: *** [Makefile:4048: /root/dev/vpp/build-data/../src/] Error 1
make[2]: Leaving directory '/root/dev/vpp/build-root/build-vpp_debug-native/vpp'
make[1]: *** [Makefile:686: vpp-build] Error 2
make[1]: Leaving directory '/root/dev/vpp/build-root'
make: *** [Makefile:328: build] Error 2
:~/dev/vpp ((v18.01.1)) # 
:~/dev/vpp ((v18.01.1)) # 
:~/dev/vpp ((v18.01.1)) # cd /root/dev/vpp/build-data/../src/ && /bin/sh /root/dev/vpp/src/missing automake-1.15 --add-missing installing './ylwrap'
:~/dev/vpp/src ((v18.01.1)) # 

Once done, I’ve restarted the test, and another error

/root/dev/vpp/src/vpp-api/vapi/ --prefix /root/dev/vpp/build-root/vapi_test/ fake.api.json
cc -o /root/dev/vpp/build-root/vapi_test/vapi_c_test -std=gnu99 -g -Wall -pthread -I/root/dev/vpp/src -I/root/dev/vpp/build-root/install-vpp-native//vpp/include -I/root/dev/vpp/build-root/vapi_test/ vapi_c_test.c -L/root/dev/vpp/build-root/build-vpp-native/vpp/.libs/ -L/root/dev/vpp/build-root/build-vpp-native/vpp/vpp-api/vapi/.libs/ -lvppinfra -lvlibmemoryclient -lsvm -lpthread -lcheck -lrt -lm -lvapiclient -lsubunit
vapi_c_test.c:24:10: fatal error: check.h: No such file or directory
 #include <check.h>
compilation terminated.
make[2]: *** [Makefile:32: /root/dev/vpp/build-root/vapi_test/vapi_c_test] Error 1
make[2]: Leaving directory '/root/dev/vpp/test/ext'
make[1]: *** [Makefile:121: ext] Error 2
make[1]: Leaving directory '/root/dev/vpp/test'
make: *** [Makefile:368: test] Error 2

I need to find the ‘libcheck’ avail also from at

New error about not finding libsubunit

Finished processing dependencies for vpp-papi==1.4
make -C ext
make[2]: Entering directory '/root/dev/vpp/test/ext'
mkdir -p /root/dev/vpp/build-root/vapi_test/
/root/dev/vpp/src/vpp-api/vapi/ --prefix /root/dev/vpp/build-root/vapi_test/ fake.api.json
cc -o /root/dev/vpp/build-root/vapi_test/vapi_c_test -std=gnu99 -g -Wall -pthread -I/root/dev/vpp/src -I/root/dev/vpp/build-root/install-vpp-native//vpp/include -I/root/dev/vpp/build-root/vapi_test/ vapi_c_test.c -L/root/dev/vpp/build-root/build-vpp-native/vpp/.libs/ -L/root/dev/vpp/build-root/build-vpp-native/vpp/vpp-api/vapi/.libs/ -lvppinfra -lvlibmemoryclient -lsvm -lpthread -lcheck -lrt -lm -lvapiclient -lsubunit
/usr/lib64/gcc/x86_64-slackware-linux/7.3.0/../../../../x86_64-slackware-linux/bin/ld: cannot find -lsubunit
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:32: /root/dev/vpp/build-root/vapi_test/vapi_c_test] Error 1
make[2]: Leaving directory '/root/dev/vpp/test/ext'
make[1]: *** [Makefile:121: ext] Error 2
make[1]: Leaving directory '/root/dev/vpp/test'
make: *** [Makefile:368: test] Error 2

From the VPP-DEV mailling list, it seems that issue comes from the install of  `check` used to require lsubunit as of < 0.10 version

monkey patching the test/ext/Makefile to avoid using libsubunit for slackware on top on opensuse.

ifeq ($(OS_ID), $(filter slackware,$(OS_ID)) $(filter opensuse,$(OS_ID)),$(OS_ID))

/!\ It seems too `fall in service` somehow so need rework. /!\

then test are running but got lot of VPP crash during tests, and upon restart, not alway on the same test…  which means that stability is not fully there for VPP on my system…..

typically stuff like:

VPP subprocess died unexpectedly with returncode -6 [unknown]
Basic test for IPSEC using AH transport and Tunnel mode
ipsec ah v4 transport basic test                                         OK
ipsec ah v4 transport burst test                                         OK
ipsec ah 4o4 tunnel basic test                                           OK
ipsec ah 4o4 tunnel burst test                                           OK
UDP Encap Test Case 
01:13:06,235 VPP subprocess died unexpectedly with returncode -6 [unknown]
01:13:06,235 Core file present, debug with: gdb /root/dev/vpp/build-root/install-vpp-native/vpp/bin/vpp /tmp/vpp-unittest-TestUdpEncap-BV72B1/core
01:13:06,237 [Errno 17] File exists
UDP Encap test                                                           ERROR [ temp dir used by test case: /tmp/vpp-unittest-TestUdpEncap-BV72B1 ]
CRUD Loopback
test_crud (test_interface_crud.TestLoopbackInterfaceCRUD)                ERROR [ temp dir used by test case: /tmp/vpp-unittest-TestLoopbackInterfaceCRUD-j83o7s ]
01:13:13,586 [Errno 17] File exists
test_down (test_interface_crud.TestLoopbackInterfaceCRUD)                ERROR [ temp dir used by test case: /tmp/vpp-unittest-TestLoopbackInterfaceCRUD-j83o7s ]

I decided to build it anyway following FDIO doc:,_install,_and_test_images#Build.2C_Install.2C_and_Test_VPP_without_using_Packages

:~/dev/vpp (stable/1801) # make build-release

Seems to build fine then…..couple of warning but not critical to fail the build.

Following the doc after a succesfull build

:~ # mkdir dev/sandbox/vpp -p
:~ # cat dev/sandbox/vpp/ # script adapted from FDIO doc.



echo 'removing old stuff'
sudo /bin/rm -f /dev/shm/db /dev/shm/global_vm /dev/shm/vpe-api || return

echo 'loading module uio_pci_generic'
sudo /sbin/modprobe uio_pci_generic || return

echo 'starting VPP with its config file'
sudo $T/bin/vpp -c /etc/vpp/startup.conf

:~/dev/vpp (stable/1801) # mkdir /etc/vpp
:~/dev/vpp (stable/1801) # cp src/vpp/conf/startup.conf /etc/vpp/startup.conf

you might have thgis kind of warning:

:~ # source dev/sandbox/vpp/ 
vlib_plugin_early_init:356: plugin path /usr/lib/vpp_plugins
api_segment_config: group vpp does not exist

that can be easily fixed by creating the group, and adding your user to it. I work as root here, but need obviously to make it usable for a non-root user as well. As it’s POC/test at the moment, I focus on the making VPP work.

:~ # groupadd vpp
:~ # usermod -aG  vpp root

Then VPP seems to start fine 🙂

:~ # source /root/dev/sandbox/vpp/
removing old stuff
loading module uio_pci_generic
starting VPP with its config file
vlib_plugin_early_init:356: plugin path /root/dev/vpp/build-root/build-vpp-native/vpp/plugins/

in another terminal: use `vppctl` to get to the VPP prompt, you probably want to add the path ( or symlink ) to the correct executable path.

:~/dev/vpp/build-root/build-vpp-native/vpp/bin (stable/1801) # ls
summary_stats_client*  vpp*  vpp_get_metrics*  vppctl*
:~/dev/vpp/build-root/build-vpp-native/vpp/bin (stable/1801) # ./vppctl
    _______    _        _   _____  ___ 
 __/ __/ _ \  (_)__    | | / / _ \/ _ \
 _/ _// // / / / _ \   | |/ / ___/ ___/
 /_/ /____(_)_/\___/   |___/_/  /_/    

vpp# quit
:~/dev/vpp/build-root/build-vpp-native/vpp/bin (stable/1801) #