linux 4.18: gem5 didn't print to terminal and silently crashes

yunchih commented 4 years ago

Running Environment (please complete the following information):

Host OS and Kernel version: Debian 10, Linux 4.19
Compiler version used to build SimpleSSD-FullSystem: GCC 8.3
SimpleSSD-FullSystem version: v2.0.9
SimpleSSD version: v2.0.13
Guest Kernel version: 4.18 aarch64
Compiler version used to build Guest Kernel binary: gcc-linaro-7.5.0-2019.12

Question

linux 4.9 from simplessd

linux 4.9 kernel and the linaro image provided by simplessd

In nvme mode, ./nvme list shows the emulated nvme device; however, in ocssd mode, ./nvme list shows none.

export M5_PATH=upstream-images/
./build/ARM/gem5.opt --debug-flag=M5Print --debug-file=debug.txt \
    ./configs/example/fs.py \
    --kernel=upstream-images/binaries/aarch64-vmlinux-4.9.92 \
    --dtb-file=armv8_gem5_v1_4cpu.dtb --machine-type=VExpress_GEM5_V1 \
    --disk-image=upstream-images/disks/linaro-aarch64-linux.img \
    --num-cpu=4 --cpu-clock=2GHz --caches --l2cache --cpu-type=AtomicSimpleCPU \
    --mem-size=2GB --mem-type=DDR4_2400_8x8 \
    --command-line "earlyprintk=pl011,0x1c090000 console=ttyAMA0 lpj=19988480 norandmaps ro \
    loglevel=8 mem=2GB root=/dev/sda1 kpti=0 nospectre_v2 ssbd=force-off init=/bin/sh" \
    --ssd-interface=ocssd --ssd-config=./src/dev/storage/simplessd/config/sample.cfg

linux 4.18 I built

I've tried compiling 4.18 kernel using the kernel config from simplessd and runs as follow:

export M5_PATH=upstream-images/
./build/ARM/gem5.opt --debug-flag=M5Print --debug-file=debug.txt \
    ./configs/example/fs.py \
    --kernel=/home/yunchih/research/simplessd/linux-4.18.12-arm/vmlinux \
    --dtb-file=armv8_gem5_v1_4cpu.dtb --machine-type=VExpress_GEM5_V1 \
    --disk-image=upstream-images/disks/linaro-aarch64-linux.img \
    --num-cpu=4 --cpu-clock=2GHz --caches --l2cache --cpu-type=AtomicSimpleCPU \
    --mem-size=2GB --mem-type=DDR4_2400_8x8 \
    --command-line "earlyprintk=pl011,0x1c090000 console=ttyAMA0 lpj=19988480 norandmaps rw \
    loglevel=8 mem=2GB root=/dev/sda1 kpti=0 nospectre_v2 ssbd=force-off init=/bin/sh" \
    --ssd-interface=ocssd --ssd-config=./src/dev/storage/simplessd/config/sample.cfg

GEM5 full output Nothing is shown in m5term, and the kernel crashes. The system.dmesg does not run far enough to inspect the crash reason.

Because I plan to use OCSSD 2.0, I need a working kernel at least > 4.18. Maybe @kukdh1 could provide a more recent working kernel, as you've mentioned in #3 Thanks for your hardwork!

kukdh1 commented 4 years ago

Hi, yunchih.

For the m5term, you should double check that kernel configuration file config_arm is applied properly.

# Assume your working directory is root of Linux kernel source.
# Download config_arm file here.
# Step 1. Create default ARM64 Linux kernel configuration
ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make defconfig
# Step 2. Overwrite generated .config file with config_arm
mv config_arm .config
# Step 3. Modify overwritten config file -- enable OpenChannel SSD or other features.
ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make menuconfig
# Step 4. Compile vmliunx
ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make vmlinux -j $(nproc)

To use OpenChannel SSD v2.0, you need to use --ssd-interface=ocssd2 instead of ocssd.

Please note: When I implement OCSSD protocol (2017 ~ 2018), the specification was quite ambiguous. I found the spec is updated little bit but such changes are not applied to SimpleSSD.

P.S. Please close this issue if solved. If not, I can provide my kernel configuration file for 4.19.114 - it worked very well.

# Linux kernel bootlog
[    0.208456] nvme 0000:00:02.0: assign IRQ: got 13
[    0.208512] nvme nvme0: pci function 0000:00:02.0
[    0.208521] nvme 0000:00:02.0: enabling device (0000 -> 0002)
[    0.208528] nvme 0000:00:02.0: enabling bus mastering
[    0.208608] ata_piix 0000:00:01.0: assign IRQ: got 15
[    0.208614] ata_piix 0000:00:01.0: version 2.13
[    0.208621] ata_piix 0000:00:01.0: enabling device (0000 -> 0001)
[    0.208643] ata_piix 0000:00:01.0: enabling bus mastering
[    0.209533] nvm: registered nvme0n1 [8/8/512/64/8]

# SimpleSSD output
897733495500: HIL::NVMe: SQ 3    | Submission Queue Tail Doorbell | Item count in queue 0 -> 1 | head 34 | tail 34 -> 35
897739449272: HIL::NVMe: OCSSD   | READ  | 7FFFFC0 + 14
897743791772: HIL::NVMe: OCSSD   | READ  | 7FFFFC0 + 14 | NAND 897739449272 - 897743791772 (4342500)
897755934500: HIL::NVMe: SQ 3    | Submission Queue Tail Doorbell | Item count in queue 0 -> 1 | head 35 | tail 35 -> 36
897761553475: HIL::NVMe: OCSSD   | READ  | 7FFFFCF + 16
897765895975: HIL::NVMe: OCSSD   | READ  | 7FFFFCF + 16 | NAND 897761553475 - 897765895975 (4342500)
897769538052: HIL::NVMe: OCSSD   | READ  | 7FFFFC0 + 14 | DMA 897743791772 - 897769538052 (25746280)
897777494500: HIL::NVMe: SQ 3    | Submission Queue Tail Doorbell | Item count in queue 0 -> 1 | head 36 | tail 36 -> 37
897783815178: HIL::NVMe: OCSSD   | READ  | 7FFFFE1 + 15
897788157678: HIL::NVMe: OCSSD   | READ  | 7FFFFE1 + 15 | NAND 897783815178 - 897788157678 (4342500)

root@genericarmv8:~# ./nvme list
Node             SN                   Model                                    Namespace Usage                      Format           FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     00000000000000000000 SimpleSSD OCSSD Controller by CAMELab    1         549.76  GB / 549.76  GB      4 KiB +  0 B   02.01.03
root@genericarmv8:~# ./nvme lnvm list
Number of devices: 1
Device          Block manager   Version
nvme0n1         gennvm          (1,0,0)
root@genericarmv8:~# ./nvme lnvm create -d nvme0n1 -n lnvm0 -t pblk -b 0 -e 7
[    1.401681] pblk lnvm0: luns:8, lines:512, secs:16777216, buf entries:128
root@genericarmv8:~# cat /proc/partitions
major minor  #blocks  name

 259        0  536870912 nvme0n1
   8        0    1048576 sda
   8        1    1047552 sda1
 259        1   59555840 lnvm0

P.S. 2. If you want to use most-recent kernel, I think you can use Linux 5.5 without any problem. I checked Linux 5.2 worked with SimpleSSD.

P.S. 3. pblk maintainers highly work on their kernel drivers and some version may have bug.

Thanks.

yunchih commented 4 years ago

@kukdh1 Can't thank you enough. I successfully run linux 5.4 following your detailed instructions. For future reference, the disk image can be obtained by the following means:

Download from linaro
Build from Buildroot's aarch64 little endian architecture

Build from debootstrap:

dd if=/dev/zero of=debian.img bs=1M count=1000
sudo mount -o loop debian.img /mnt
sudo debootstrap --arch=arm64 \
--variant=minbase --foreign \
testing /mnt \
http://ftp.debian.org/debian/

Just use the one provided by simplessd

I tested them all, they all worked.

SimpleSSD / SimpleSSD-FullSystem