search

zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.88k stars 6.63k forks source link

evaluate Emul8 as a replacement for QEMU in sanitycheck #3499

Closed zephyrbot closed 5 years ago

zephyrbot commented 7 years ago

Reported by Andrew Boie:

QEMU tests fail all the time due to timing issues - it tries to sync with the host system clock, causing all kinds of problems with test cases if the host system is under heavy load, which is almost always the case with Jenkins.

Emul8 doesn't do this. Investigate what it would take to replace QEMU with Emul8 on those architectures that it supports.

(Imported from Jira ZEP-2058)

zephyrbot commented 7 years ago

by Andrew Boie:

Hi,

First, I'll answer your questions:

1) to show uart output run ``showAnalyzer sysbus.uart_b`` - it will open a new window
2) you can run Emul8 with socket instead of a window. Use -P switch with a port number. It will prevent opening any windows. You may also pass the script name as the last parameter, it will be included automatically - it won't be started though, so you need to end the script with "start" command (I can see you already have it in your example).
3) redirecting uart to a socket is also possible. You need the following commands in the script:
emulation CreateServerSocketTerminal 9999 "uart_socket" (where 9999 is a target port number and "uart_socket" will be a name of the socket object)
connector Connect sysbus.uart_b uart_socket

This way, combining -P option and ServerSocketTerminal, you may get a headless setup. You should not run "showAnalyzer" command in that case, as it will fail, being unable to open a window.

Please keep in mind, that running Emul8 in headless mode will still require a running X server on the current session. If you would like to run it, for example, via ssh, please use "xvfb-run" command (it's available on most Linux distros).

Now, regarding your script - it is a good start, but it will need some more lines to go past the bootloader. (TL;DR - I attach a proper script to this mail)
The bootloader waits for the GPIO pin 15 to be high (related to JTAG, I think) - you have to set that manually in the script:

sysbus.gpio OnGPIO 15 true

Since we don't implement the full peripheral set for Quark, we have to mock some not-that-relevant registers, that are often read just to verify that, for example, a clock is running or that we're past some initialization.
This bootloader, as far as I can tell right now, requires this line:

sysbus Tag <0xB0800004 4> "OSC0_STAT1" 0xFFFFFFFF

It will create a 4-byte region that will always respond with 0xffffffff. Every access will be logged with "OSC0_STAT1" tag.

The last change - it is strongly advised to set the deterministic timer mode:

machine SetClockSource cpu

Without this setting the execution is strongly tied to the performance of the host machine and you may observe strange results.
We will probably make it a default, although sometimes it may slightly reduce the performance.

This is the minimal set of commands that allows me to run the philosophers demo. I attach the script you made, with my changes.

For a complete script running the shell demo, you may take a look at scripts/demos/standalone/quark_c1000-shell.

To run it with your own elf, just overwrite the "bin" variable:

$bin="phil.elf"
i @scripts/demos/standalone/quark_c1000-shell

Keep in mind that this script does open a new window for uart, so you will probably have to modify it to meet your requirements.

If you have any more questions, I'll be glad to help.

Best regards

Piotr Zierhoffer
mobile: +48 696 419 606
Antmicro Ltd | www.antmicro.com
Zwierzyniecka 3, 60-813 Poznan, Poland

2017-04-07 21:14 GMT+02:00 Michael Gielda <mgielda@antmicro.com>:
---------- Forwarded message ----------
From: "Boie, Andrew P" <andrew.p.boie@intel.com>
Date: 7 Apr 2017 21:11
Subject: RE: Emul8 and Zephyr: Demonstration and milestone acceptance
To: "Laperie, Andrei" <andrei.laperie@intel.com>, "Michael Gielda" <mgielda@antmicro.com>, "Perez-Gonzalez, Inaky" <inaky.perez-gonzalez@intel.com>, "Ross, Andrew J" <andrew.j.ross@intel.com>, "Pereira, Leandro" <leandro.pereira@intel.com>, "Rissanen, Jukka" <jukka.rissanen@intel.com>, "Bursztyka, Tomasz" <tomasz.bursztyka@intel.com>, "Veeramally, Ravikumar" <ravikumar.veeramally@intel.com>
Cc: "Nashif, Anas" <anas.nashif@intel.com>, "Hedberg, Johan" <johan.hedberg@intel.com>

Hi,

I’m experimenting with Emul8 as a potential replacement for QEMU when running sanitycheck.
I was able to build the whole thing without problems using Mono libraries, at one point someone said you had to download stuff from Microsoft but that was not my experience.

I am now trying to run some of our Zephyr samples under the emulator but have not had much success.
I built ‘samples/philosophers’ for arduino_101 and saved the resulting .elf binary as “phil.elf”.
I then attempted to create a script to build and run this binary but didn’t have much success. This is what I came up with:

$name="bare-machine"
$elf="phil.elf"

mach create $name

i @platforms/cpus/quark_c1000
i @platforms/boards/arduino_101-shield

sysbus LoadELF $CWD/phil.elf

$boot ?= @https://antmicro.com/emul8_files/binaries/emul8/quark_se_rom.bin-s_8192-b688c8b9380014d7deb2bd42dc218fc5ee8d1abf
sysbus LoadBinary $boot 0xffffe000

sysbus.cpu PC 0xfffffff0

start

Upon running it I’m not sure if it’s actually working or not, I get the following:

12:08:01.6155 [INFO] Terminal shown
12:08:04.4101 [INFO] Including script: /projects/emul8-quark-se-second-delivery/bare-machine
12:08:04.4707 [INFO] System bus created.
12:08:05.1389 [DEBUG] Segment size automatically calculated to value 64KiB (5)
12:08:05.8256 [DEBUG] [no-name]: Registered memory at 0xFFFFE000, size 0x2000.
12:08:05.8480 [DEBUG] [no-name]: Registered memory at 0xE000, size 0x2000.
12:08:05.8481 [DEBUG] [no-name]: Registered memory at 0x40000000, size 0x10000.
12:08:05.8481 [DEBUG] [no-name]: Registered memory at 0x40010000, size 0x10000.
12:08:05.8482 [DEBUG] [no-name]: Registered memory at 0x40020000, size 0xF000.
12:08:05.8482 [DEBUG] [no-name]: Registered memory at 0x40030000, size 0x10000.
12:08:05.8483 [DEBUG] [no-name]: Registered memory at 0x40040000, size 0x10000.
12:08:05.8483 [DEBUG] [no-name]: Registered memory at 0x40050000, size 0xF000.
12:08:05.8484 [DEBUG] [no-name]: Registered memory at 0x4002F000, size 0x1000.
12:08:05.8484 [DEBUG] [no-name]: Registered memory at 0xA8000000, size 0x10000.
12:08:05.8485 [DEBUG] [no-name]: Registered memory at 0xA8010000, size 0x4000.
12:08:07.0896 [INFO] sysbus: Loading segment of 14824 bytes length at 0x40010000.
12:08:07.1014 [DEBUG] sysbus: Segment loaded.
12:08:07.1016 [INFO] sysbus: Loading segment of 164 bytes length at 0x400139E8.
12:08:07.1016 [DEBUG] sysbus: Segment loaded.
12:08:07.1017 [INFO] sysbus: Loading segment of 8780 bytes length at 0xA80064C0.
12:08:07.1018 [DEBUG] sysbus: Segment loaded.
12:08:07.1939 [INFO] cpu: Setting PC value to 0x40010030.
12:08:07.5122 [INFO] Including script: /projects/emul8-quark-se-second-delivery/bare-machine
12:08:07.5702 [DEBUG] sysbus: Loading binary /tmp/emul8-14181/8c7e4296-11ae-46b1-a4cc-2a6e19314bf0.tmp at 0xFFFFE000.
12:08:07.5718 [DEBUG] sysbus: Binary loaded.
12:08:07.8728 [INFO] bare-machine: Machine started.
12:08:07.9801 [WARNING] sysbus: [CPU0: 0xFFFFE242] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800100, returning 0x0.
12:08:07.9836 [WARNING] sysbus: [CPU0: 0xFFFFE242] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800100, value 0x0
12:08:07.9867 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800910, value 0xFFFFFFFF
12:08:07.9871 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800914, value 0xFFFFFFFF
12:08:07.9874 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800918, value 0xFFFFFFFF
12:08:07.9877 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB080091C, value 0xFFFFFFFF
12:08:07.9880 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800008, returning 0x0.
12:08:07.9883 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800008, value 0x0
12:08:07.9886 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800008, returning 0x0.
12:08:07.9889 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800008, value 0x40000
12:08:07.9892 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800000, returning 0x0.
12:08:07.9895 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800000, value 0x0
12:08:07.9898 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800000, returning 0x0.
12:08:07.9901 [WARNING] sysbus: [CPU0: 0xFFFFE2C2] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800000, value 0x400000
12:08:07.9941 [WARNING] sysbus: [CPU0: 0xFFFFE514] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800038, returning 0x0.
12:08:07.9945 [WARNING] sysbus: [CPU0: 0xFFFFE514] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800008, returning 0x0.
12:08:07.9948 [WARNING] sysbus: [CPU0: 0xFFFFE514] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800008, value 0x0
12:08:07.9951 [WARNING] sysbus: [CPU0: 0xFFFFE514] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800008, returning 0x0.
12:08:07.9954 [WARNING] sysbus: [CPU0: 0xFFFFE514] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800008, value 0x0
12:08:07.9957 [WARNING] sysbus: [CPU0: 0xFFFFE514] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800008, returning 0x0.
12:08:07.9960 [WARNING] sysbus: [CPU0: 0xFFFFE514] (tag: 'SCSS') WriteDoubleWord to non existing peripheral at 0xB0800008, value 0x2
12:08:07.9963 [WARNING] sysbus: [CPU0: 0xFFFFE514] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800004, returning 0x0.
12:08:11.1203 [WARNING] sysbus: [CPU0: 0xFFFFE54D] (tag: 'SCSS') ReadDoubleWord from non existing per12:08:11.1888 [WARNING] sysbus: [CPU0: 0xFFFFE54D] (tag: 'SCSS') ReadDoubleWord from non existing per12:08:11.6382 [WARNING] sysbus: [CPU0: 0xFFFFE54D] (tag: 'SCSS') ReadDoubleWord from non existing peripheral at 0xB0800004, returning 0x0. (7419

I suppose I have some questions:

•   Where can I get the UART console output, i.e. the output of printk()?
•   Is there a way to get this thing to stop spawning a GUI terminal window? I had planned on integrating this into sanitycheck and I really need this to work purely as a terminal application
•   Assuming we can get the console output, is there a way to redirect it to a file or something? emul8 –help-cli has very few options

Andrew
zephyrbot commented 7 years ago

by Andrew Boie:

attachment configuration to boot quark SE

#the next line allows you to drop "sysbus." when refering to peripherals
using sysbus

$name="bare-machine"
$elf="phil.elf"

mach create $name

i @platforms/cpus/quark_c1000
i @platforms/boards/arduino_101-shield

#since you have a variable, I used it here
sysbus LoadELF $elf

#the notation you used, ?=, means "set variable if not yet set". It's not important here.
$boot = @https://antmicro.com/emul8_files/binaries/emul8/quark_se_rom.bin-s_8192-b688c8b9380014d7deb2bd42dc218fc5ee8d1abf
sysbus LoadBinary $boot 0xffffe000

cpu PC 0xfffffff0

gpio OnGPIO 15 true
sysbus Tag <0xB0800004 4> "OSC0_STAT1" 0xFFFFFFFF
machine SetClockSource cpu
# the line below - to show uart console
showAnalyzer uart_b

#uncomment these to open a socket. Please comment out "showAnalyzer" above if you want to run with -P switch
#emulation CreateServerSocketTerminal 9876 "uart_socket"
#connector Connect uart_b uart_socket

#we don't usually add this command to a script.
#In interactive mode, you may load a script with "i @script.name" or explicitly start it with "s @script.name"
#On the other hand, it will be useful to keep it here if you plan to run the script in the headless mode.
start
zephyrbot commented 7 years ago

by Andrew Boie:

Hi Andrew,

The current approach is to run a fake X server called "xvfb" - it has a relatively small footprint. The X client is not required.

At the moment we are investigating the possibility of running a full-headless mode, which will require the user to be a little bit more careful, since some of the commands will not be legal (like "open a terminal window").

We will keep you posted on the development in that field, but you can already use the xvfb approach right now and just expect the xvfb-less solution to appear later on.

Your use case definitely sounds like a sound application of Emul8 and of course it's been one of the envisioned uses all along. We'd be very happy to discuss this further and work with you to achieve this.

Best regards

Piotr Zierhoffer mobile: +48 696 419 606 Antmicro Ltd | www.antmicro.com Zwierzyniecka 3, 60-813 Poznan, Poland

nashif commented 5 years ago

we have renode now