stm32 is using a qemu version that is too old and zephyr is crashing

[fariouche]

Hello,

This is not really a bug, but a feature request maybe: The qemu stm32 is too old, and projects using zephyr are crashing the emulator. So I've created a branch and ported the stm32 machines to the latest qemu. You can give it a try in my fork, branch is picsimlab :) i don't know when the maintainer of the stm32 port of qemu will do the merge.

I would like to add esp32 support too. but this is an other story.

[lcgamboa]

Hi @fariouche,

congratulations for the great job!

I compiled your fork with the configure below and it worked 99% (apparently only the ADC converter is not working)

./configure --target-list="arm-softmmu" --disable-werror --disable-sdl --disable-vnc --disable-docs  \
--disable-blobs --static  --disable-virtfs --disable-libusb --disable-libnfs --disable-vhost-net  \
--disable-vde  --disable-curses --disable-gtk  --disable-xkbcommon --disable-libudev --disable-alsa \
--disable-opengl --disable-pa

My PR took more than years to be accepted on qemu stm32, apparently the project is stalled.

I'm not very familiar with the inner workings of qemu, currently I don't find the integration with PICSimLab satisfactory. The performance is pretty bad and it usually crashes a lot. If you have this kind of knowledge and are available, we can try to improve it. As qemu stm32 support in PICSimLab didn't work entirely as I wanted, qemu stm32 is not under active development and my time is short, I prioritized other parts of the simulator.

I've also thought about integrating qemu-esp32 into PICSimLab, but as it still doesn't support WIFI, which is the main peripheral, and with the low performance already mentioned, I haven't been interested yet. But it shouldn't be more complex than the integration of qemu-stm32.

[fariouche]

Hum, strange. I have the adc working as it was one of the peripheral I wanted. What platform did you test? in my case I test it with the blue pill one. If you tell me what adc example you used, I can have a look.

About esp32, I have found a qemu fork with wifi and bluetooth support. However, it is using a stub static library to replace the esp32's one. Didn't had time to look at it yet.

[lcgamboa]

I tested using an example that reads the adc and writes to the serial using the arduino stm32 support. It didn't work because I hadn't committed the change to github, so your code doesn't have the changes. Without them the analogRead function always returns 0. (Apparently there is some ADC mode with incomplete implementation in qemu-stm32 ) https://github.com/lcgamboa/qemu_stm32/commit/6aa558cb1a898d8e9f84533f3bdf1ebd6edad0e0 I would be grateful if you apply the changes to your fork, as I intend to use it instead of mine.

The stm32 arduino uses the peripheral DWT (Data watchpoint trigger) to implement the micros() function, as it is not implemented in qemu-stm32 several libraries do not work without changing the code.

PICSimLab's integration with qemu stm32 works well with synchronous signals. Asynchronous signals or signals that need exact timing don't work (like the stepper motor for example), this is a problem to be solved.

How has your experience been using zephyr with PICSimLab + qemu-stm32?

As for the implementation of ESP32 support, I believe we will have the same problem with the synchronism of qemu-stm32 signals. It would be interesting to solve this problem, but on the part of qemu I don't have enough knowledge to implement it and I don't have time to study it for a while.

[fariouche]

I will have a look at the modifications. I do not use arduino env for stm32. If you have a file to share or point me to a documentation to build an example and If I have time, I can have a look too try to implement DWT. Should not be too hard.

About zephyr, For now I only test blinky test app. So task scheduling is now working (was crashing before), GPIO and uart are also working. Didn't test much more than that yet as I would like to focus on ESP32 first.

[lcgamboa]

In this link there is an example of code that uses the DWT to generate delays: STM32 Delay Microsecond Millisecond Utility | DWT Delay & Timer Delay

What is your goal in focusing on the ESP32 simulation? Most PICSimLab users are students who use simulation to carry out practical activities applied by teachers. Soon I try to focus the development for didactic applications.

[fariouche]

I've found a number of issues using the arduino stm32 adc example. The first one is because of the emulator not ignoring writes bit to 1 to SR register. The other is that the arduino never init the adc peripheral. I don't see in the code anything yet that relates to this, seems like only a warning. But I see that the adc timer that is responsible for setting the convert ready bit to 1 never fires. It seems also that the example never actually wait for this bit and blindly tries to read the register. So you patch that bypass this convertion ready bit check is not correct. For now I'm trying to check why the timer never fires.

For the esp32, to be honest, I didn't find any better program than picsimlab. Easy to use, user friendly, with a scope and ability to check all pin and manipulate them! A lot of applications exists with the esp32, and they are easy to program and work with. However, I miss a real diagnostic tool with gdb debugging and I believe that picsimlab will help a lot!

[lcgamboa]

I agree that my patch is not an efficient solution. I'm not very familiar with low-level stm32 hardware, but the arduino example works on real hardware, so there's something missing in the qemu-stm32 ADC implementation.

The key to using qemu with PICSimLab would be the sync between instructions and hardware. PICSimLab executes an instruction and updates the hardware. I couldn't find a way to run qemu-stm32 instruction by instruction (and compile it as a library). Do you know if this is possible? Another project called unicorn implements just that, but without the hardware part (and without support for esp32 and gdb).

[fariouche]

So, I fixed the adc with what I believe is a correct fix regarding the HW datasheet. I'm still concerned by the stm32 arduino library not initializing the ADC peripheral correctly, but this does not prevent the emulator from working as expect. it just emit a warning every time (this will slow down the execution)

As for your question, I'm not sure I understand. I've looked at unicorn, and it seems like they have rewritten all the qemu makefiles. I don't know of a way to compile qemu as a library, but I'm no expert :) For stepping instruction by instruction, I usually use gdb + the qemu "-s -S" options to start and pause the cpu waiting for gdb to remote connect. Then it's as with any external gdb remote connection.

[lcgamboa]

I tested it here and it worked without problems. I will update the PICSimLab scripts to use your fork which is more up to date.

Do you intend to continue making improvements to qemu-stm32? Or do you intend to focus on the integration of qemu-esp32?

The PICSimLab uses several simulators libraries that have functions that execute one instruction at a time: PICsim has the pic_step(), simavr the avr_run() and ucSim the ucsim_step() . In the case of qemu_stm32 it runs in parallel with PICSimLab, not allowing PICSimLab to correctly time the IO operations.
Connecting using sockets (Unix or TCP/IP) adds a lot of delay to the simulation, which could be solved by using qemu as a library and creating a qemu_step() function. Implementing the qemu_step() function using gdb's step-by-step mode would solve the timing problem, but sending gdb commands via socket would make the simulation very slow. To give you an idea, the well-optimized simavr can emulate a maximum of 20MHz in real time, the qemu_stm32 in the current implementation can't even update the IO part at 2MHz in real time, and it goes a long way to get close to running at 72MHz!

[fariouche]

Ok, I'm starting to understand. But why do picsimlab need exact timings? Is it because you can connect to real hardware? The main problem I see is that qemu is not time exact. As far as I understand, it seems like all timers in qemu are not accurate. But for emulation it's not important, except of you want real time simulations...

One possible optimizations I see is to avoid the sockets and use shared memories instead. You will then have close to a library performance. But there is no portable implementation (maybe by using a third-party helper library?)

[lcgamboa]

For simulation of devices like stepper motors, ws2812b LEDs, software USART to work are need exact timing. For example the speed of the stepper motor depends on the pulses, see the simulation result of simavr: Signal:

Qemu-stm32 simulation: Signal: As in qemu-stm32 it is not possible to count the exact number of instructions executed and sincronize the IO on PICSimLab, the pulses vary according to the communication delay and the operational system load.

The real-time simulation issue is for the user experience. A simple 1 second LED blink programmed for a 72MHz CPU emulating at 4MHz will take 18 seconds, which is a very different result from the real hardware and a bad experience.

[fariouche]

it's so nice to see everything moving with good looking animations :heart_eyes:

Now I see where come from the real time constraints. They are from the simulated parts. At least now I see how to reproduce the issue so that I can try to have a look at it. I hope that qemu timers will not be the limiting factor.

[lcgamboa]

Hi @fariouche ,

I found this gist Build qemu as a library and with some modifications I was able to create a library with your fork of qemu-stm32. It will make possible to remove socket communication with PICSimLab. When I have time I will implement and test it. Now only the part of synchronism of the execution of the instructions with the update of the IO is missing.

Modified unicorn_but_worse.sh script:

#!/bin/sh

set -e

target="arm-softmmu"

case "$(uname)" in
  Linux)
    ncpu="$(nproc)"
    ;;
  Darwin)
    ncpu="$(sysctl -n hw.physicalcpu)"
    ;;
esac

flags="$(./configure --help | perl -ne 'print if s/^  ([a-z][\w-]*) .*/\1/' | tail -n +2 | awk '{print "--disable-"$1}' ORS=' ')"

${2:-.}/configure --target-list=$target --extra-cflags=-fPIC --disable-slirp $flags --enable-tcg --enable-system --disable-werror --disable-alsa 

make clean >/dev/null

# Build everything as usual
make "-j$ncpu" 

# Build a shared library, without softmmu/main.o and otherwise *exactly* the same
# flags.
cd build
rm -f qemu-system-arm
ninja -d keeprsp
sed -i 's/qemu-system-arm.p\/softmmu_main.c.o//g' qemu-system-arm.rsp
sed -i 's/-o\ qemu-system-arm/-shared\ -o\ libqemu.so/g' qemu-system-arm.rsp
c++ -m64 -mcx16 @qemu-system-arm.rsp

[fariouche]

Interesting! Meanwhile, I've looked at DWT... and it seems that this relates to the other problem (instruction timing): qemu is not accurate by design. So having synchronism may be very difficult, but I'm not a qemu expert. I'm still discovering it.

[lcgamboa]

I modified PICSimLab to work with the qemu-stm32 library. It was much faster because it doesn't use sockets (and doesn't crash like the previous one), but it doesn't work with the official version of PICSimLab yet. The official version uses wxwidgets, which on Linux uses gtk/glib which conflicts with qemu main_loop (glib). The X11 and SDL2 version works without problems, I haven't tested it on the windows version that uses wxwidgets, but it should work because it doesn't use glib. I'm not very familiar with glib, I'll try to solve the problem. If anyone is familiar with using glib and wants to help feel free to get in touch.

[fariouche]

Love it! I used long time ago wxwidgets and glib main loop... but it was very long ago. Do you have you current work in a branch somewhere?

[lcgamboa]

I solved the incompatibility problem with wxwidgets. The problem now is compiling the qemu_stm32 library statically. To use the new version compile the library from the repository https://github.com/lcgamboa/qemu_stm32/tree/picsimlab using the script build_libqemu-stm32.sh. Copy the build/libqemu-stm32.so file to /usr/local/lib/ And use the PICSimLab branch https://github.com/lcgamboa/picsimlab/tree/libqemu

[lcgamboa]

Hi @fariouche I have updated PICSimLab to use your new version of qemu-stm32. The upcoming stable release 0.8.10 will use your version of qemu-stm32 still with sockets. The version using the library without sockets will need a lot of improvements and will be left for another future version. I'll close this issue, feel free to make new suggestions. Thanks for the qemu-stm32 update.