RISC-V emulator

[AntonKozlov]

An emulator of RISC-V with Linux. Should be able to execute simple binaries and debug them (gdb, strace are required at least)

Be ready to integrate along #1, i.e. in single Docker image

[tsarn]

There are instructions for running Fedora in qemu at https://wiki.qemu.org/Documentation/Platforms/RISCV

[AntonKozlov]

Have you tried it?

You may kick-off a Dockerfile (based on ubuntu for example) that would have the qemu and fedora risc-v image installed

[tsarn]

I added Dockerfile and instructions to /dev-riscv/emulator. The image doesn't provide gdb and strace by default and changes will probably be lost when the container stops, so this probably needs to be addressed somehow.

[AntonKozlov]

Great, I've set Docker Hub automation to build the image. It have significant lag, I'll update the README with a link to https://hub.docker.com/r/azulresearch/riscv-emu. In final, it could be just

docker run -p 2222:22 --rm azulresearch/riscv-emu

without necessity to docker build

The image doesn't provide gdb and strace and changes will probably be lost when the container stops

It seems that the hard drive image (stage4-disk.img) is provided with write permissions. How about running some necessary steps after downloading?

Like

RUN qemu-system-riscv ... &
RUN ssh localhost -p 2222 "apt install ... ; shutdown"

[AntonKozlov]

Have you considered user-space emulation? qemu-riscv or something, that emulates only CPU and translates syscalls to host OS

[tsarn]

I've built RISC-V port of Debian in a chroot locally. Apparently qemu doesn't support ptrace(2) syscall in userspace, so strace won't work this way. Other than that, it seems to work fine, albeit a bit slow (but that is to be expected I suppose). I will probably go and dockerize it now.

[AntonKozlov]

Apparently qemu doesn't support ptrace(2) syscall in userspace, so strace won't work this way

Hmm, then GDB will not work too. Is there any open bug report for qemu for this?

I mean, it may make the userspace emulation unusable for us :(

[tsarn]

Hmm, then GDB will not work too. Is there any open bug report for qemu for this?

strace ./command can be replaced with QEMU_STRACE= ./command. As for gdb, it is possible to start a gdb server using qemu-riscv64 -g <PORT> and connect to it from anywhere (including from the inside of the chroot).

As for dockerizing this whole thing, there is a possibility that it's impossible to modify binfmt settings from an unprivileged Docker container (update-binfmts: warning: Couldn't mount the binfmt_misc filesystem on /proc/sys/fs/binfmt_misc).

[AntonKozlov]

strace ./command can be replaced with QEMU_STRACE= ./command. As for gdb, it is possible to start a gdb server using qemu-riscv64 -g <PORT> and connect to it from anywhere (including from the inside of the chroot).

Oh, that's different. If debugging is possible, then we want that :)

As for dockerizing this whole thing, there is a possibility that it's impossible to modify binfmt settings from an unprivileged Docker container (update-binfmts: warning: Couldn't mount the binfmt_misc filesystem on /proc/sys/fs/binfmt_misc).

If running qemu-riscv /bin/bash is possible, then it would be fine without adjusting binfmt.

You may try to chroot and simultaneously start emulating. Something like

chroot sysroot opt/qemu-riscv /bin/bash

(qemu binary should be accessible in chroot)

[tsarn]

I've dockerized the emulator and written instructions on how to use it. I've also commited qemu binary to the repository for easier deployment (since I had to spend 30 minutes building it and its dependencies manually, I thought others might not want to), it's not that big, but if this is considered bad style, it can be removed.

[AntonKozlov]

Thanks!

I've also commited qemu binary to the repository for easier deployment

Usually it's not a good practice. But I think we'll be able to live with that:

• we don't need to rebuild it
• it's small compared with the rest of repo

I don't think we totally need to fix that.

But I think binfmt approach (being able to run foreign binaries as like they are native) is not very useful. Doing chroot adds transparency and allows building things in modular way. I left a note in commit.

[tsarn]

But I think binfmt approach (being able to run foreign binaries as like they are native) is not very useful. Doing chroot adds transparency and allows building things in modular way. I left a note in commit.

We probably want to modify binfmt, otherwise running any command from within the chroot would require prefixing it with the qemu binary (this means shell scripts will definitely be broken, among other things). Modifying binfmt can't be done from inside the Docker container and binfmt can't be made to run a binary that doesn't exist (it fails with ENOFILE). But fun fact: if binfmt is already configured, the qemu binary on the host system can be safely removed and if it exists inside of the container, it will continue to work.

[AntonKozlov]

Responded in https://github.com/azul-research/jdk-riscv/commit/217446adc4962395afba51247988c7caac78a18f#commitcomment-35446546

Let's have the discussion here. Building in Docker Hub automation leads to

Step 3/8 : RUN /bootstrap.sh
---> Running in 3be03e4bf0ab
Get:1 http://cdn-fastly.deb.debian.org/debian unstable InRelease [139 kB]
Get:2 http://cdn-fastly.deb.debian.org/debian unstable/main amd64 Packages [8234 kB]
Fetched 8373 kB in 7s (1263 kB/s)
...
Setting up libpsl5:amd64 (0.20.2-2) ...
Setting up wget (1.20.3-1+b1) ...
Setting up qemu-user-static (1:4.1-1+b2) ...
mount: /proc/sys/fs/binfmt_misc: permission denied.
update-binfmts: warning: Couldn't mount the binfmt_misc filesystem on /proc/sys/fs/binfmt_misc.
mount: /proc/sys/fs/binfmt_misc: permission denied.
update-binfmts: warning: Couldn't mount the binfmt_misc filesystem on /proc/sys/fs/binfmt_misc.
mount: /proc/sys/fs/binfmt_misc: permission denied.
update-binfmts: warning: Couldn't mount the binfmt_misc filesystem on /proc/sys/fs/binfmt_misc.
mount: /proc/sys/fs/binfmt_misc: permission denied.
...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
E: Unable to execute target architecture
Removing intermediate container 3be03e4bf0ab
The command '/bin/sh -c /bootstrap.sh' returned a non-zero code: 1

[tsarn]

Building in Docker Hub automation leads to

Yeah, makes sense. Unfortunately I am using debootstrap to bootstrap the chroot environment. It fails spectacularly when binfmt is not configured. Maybe this can be addressed somehow, but I currently don't know how.

Shell scripts themselves may be started with qemu bash ....

Unless they happen to use stuff like [, grep, sed, ls, cat and many other tools which require running external command and need to be prefixed with qemu.

[AntonKozlov]

Unless they happen to use stuff like [, grep, sed, ls, cat and many other tools which require running external command and need to be prefixed with qemu.

I was totally sure qemu handles this, and I was wrong. OK, binfmt will be here for a while.

I was able to build riscv-emu-user. Thanks! Pushed as: https://hub.docker.com/r/azulresearch/riscv-emu-user. Could you add the link to the page, so it would be clear that it's either docker run azulresearch/riscv-emu-user or docker build; docker run

I think the issue will be Done after that

I can provide push access to docker hub to anyone requested, I just don't know user names.

[tsarn]

Done.

[AntonKozlov]

Thanks! Done :)

[tsarn]

See #10 for progress on eliminating dependency on binfmts_misc