QEMU

[flypie]

Can Circle be used with QEMU?

[rsta2]

I have never tried this. I'm using rpi_stub by @turboscrew to debug Circle. You may have a look at doc/debug.txt in the Circle tree for details.

[flypie]

Sounds like it might do the job. I'll still give the QEMU a test as I think it will support the full set of interrupts. I had some debug working using Eclipse or Java Beans as the PC end but I cannot remember how I did it now. The reason I'd like the QEMU is that it just means one less thing to cart around when travelling, sitting on a train with a laptop no one turns a head but when you start plugging weird looking boxes in it get a bit untidy and a suspicious to some peoples eye's.

[rsta2]

Perhaps the simple samples may work. I don't think that the USB host controller driver will work. I did read somewhere, that QEMU does not support the 8 bit per pixel framebuffer depth with palette. The depth can be changed in include/circle/screen.h to 16 or 32.

[flypie]

I will let you know but fir what I'm doing 1 bit monochrome with a character ROM and memory-mapped screen, like CGA in text mode would do fine. So everything above is overkill. Disappoint the PI doesn't have such a mode. If using standard PI Linux in QEMU USB devices can be accessed then there must be some level of interaction. If I was the foundation I would be offering a small prize to someone who can produce a QEMU module the provides a high degree of compatibility. I not in a position to do it but if someone could set up the tests for what needed to run, then some sort of Kickstarter or other fund raiser might get the prize money.

[rsta2]

According to some sources on the web they are using the versatilepb machine emulation and a special kernel, which has support for this machine, to boot Raspbian in QEMU. I tried this with the Circle sample/02-screenpixel without success, probably because Circle lacks the required drivers for versatilepb. I used a relatively old QEMU version (2.1.3) which came with my OS distribution. Maybe it will be different with a more recent version, but I wouldn't expect this.

[flypie]

I cannot get the latest Rasparian stuff to boot on QEMU for Windows. One of the problems with creating a PI machine would be how closely the SoC details of the PI are guarded. The QEMU stuff all being open source, I did take a look at the source but like all Open Source I look at it takes me a couple of days to get over the shock before I go back.

[rsta2]

Recently I found the this QEMU fork which emulates the RPi 2 and got it running with these Circle samples on a 64-bit Linux host (may also work on Windows):

• 02-screenpixel
• 03-screentext
• 04-timer
• 05-usbsimple
• 08-usbkeyboard
• 17-fractal
• 19-tasks
• 26-cpustress

QEMU successfully emulates the MMU, the USB host controller and an USB keyboard. The USB mouse emulation may also work, but the property mailbox function which shows the mouse cursor does not work. Further the SMP emulation (-smp 4) does not work, but is not required for most samples. Circle reports the SpeedFactor of 0.35 on a relatively slow E350 netbook. A native RPi 2 reports 1.72 with 600 MHz ARM clock.

Before building the Circle samples, you have to change the screen depth in include/circle/screen.h as follows:

#define DEPTH 16

That is because QEMU does not support the 8-bit depth which is used by default. Don't forget to set RASPPI = 2 in Rules.mk or Config.mk before building the samples!

Building and using QEMU from this fork works as follows:

git clone --recurse-submodules https://github.com/0xabu/qemu.git
cd qemu
mkdir build
cd build
../configure
make

cd arm-softmmu/
qemu-system-arm -machine raspi2 -smp 1 -bios /path_to_sample/kernel7.img -usbdevice keyboard

Have a look at the project wiki and the QEMU docs for further information!

[flypie]

I have a look, I've just spent a week getting the Windows Compile to work(see http://repo.or.cz/qemu/ar7.git/bundles).

Then had a look and found a Raspi & Raspi2 mention in the above src this time by "Written by Andrew Baumann"

I found I think a different folk to you. https://github.com/jncronin/rpi-boot

I went and had a look at the latest main release of Qemu and that has Baumann Code http://www.qemu-project.org/download/#source

The link you gave is to Baumans orginal folk I haven't had a chance to see if the main release is up to this yet. I need tea and painkillers. One way or the other though it looks like a descent Raspi Emu is on the way.

raspi.zip

[rsta2]

I have tried the official QEMU 2.8.0 and the most recent development version from the official repo. Both are not working with Circle. The reason is, that the emulation for the system timer (at 0x3F003000) is not included there (in hw/arm/bcm2835_peripherals.c). It is included in the repo of Andrew Baumann. There may be more problems, but this is the first I was facing using GDB.

I know this rpi-boot project by John Cronin. I ported the SD card driver (emmc.c) from there to Circle. Looking at this repo now again, I have found an update to the driver, which allows using it with QEMU. So the SD card will be working too in Circle with Andrew Baumann's version.

It seems the latest version of raspi.c from the attached archive is a clean-up. The RPi 1 emulation has been removed there.

[flypie]

Hopefully it just a matter of time till he get unified.

[rsta2]

I have added some QEMU support to Circle. This includes network and SD card support. It is currently at the develop branch. Have a look into the file doc/qemu.txt in the Circle tree for details.

I also forked the QEMU repository of Andrew Baumann and applied a patch to hw/usb/dev_network.c. Without this patch QEMU dropped incoming guest packets which where over 63 bytes long.

Because Andrew Baumann does not work on QEMU at the moment, how it seems, and the official QEMU source is currently not able to run Circle, this new fork should be the best base of running Circle inside QEMU for now. Hopefully we can move over to the official QEMU version later.

[flypie]

Good one. I'll give it a look. Sorry for delay but my SPam folder had decided to eat messages.

[rsta2]

Thanks. No problem!

[flypie]

I currently having a go at merging the QEMU branch you have put up with the Main Branch, the main branch has better PC support. Could you opload the QMEU command line you are using to run the circle demos on QEMU, just to reduce the number of unknowns I'm dealing with.

[flypie]

Sorry found it in earlier message

[rsta2]

I have tried additional options to the command line in the comment above. They are all listed in doc/qemu.txt. The "-smp 1" option can be omitted.

[flypie]

Is this what you would expect for Sample 3? This is running on Windows. It is your QEMU merged with the QEMU Master. I put the Raspi 1 support back in but it doesn't work yet.

[rsta2]

Basically that's it. Cool to have this on Windows! QEMU doesn't write the ATAGS at 0x100, but that's normally not a problem, because it is not used by Circle elsewhere. The assert() is only for demonstration.

[flypie]

If I could work out how to make it a pubic branch I would.

qemu-changes.zip

[rsta2]

A public branch of which repo? If you want to contribute to the official QEMU you should read this. There is an official mirror which can be easily forked. I could also add you as collaborator for my QEMU fork, if you want this. But this would be only another QEMU fork for the RPi. I think it would be better to try to get it into the official repo.

[flypie]

It the offical branch, I'll try and work out how to get it in. The version yours is based on doesn't work on WIndows.

Thanks for you help.

[rsta2]

OK, that's the best option. You are welcome.

[flypie]

Managed to get this repository up. Which is a branch of the Qemu Master. I have included the changes in you QEMU repository. I'm going to try and keep this up to date with the Master and get the Basic Pi working. https://github.com/flypie/flypie-pi-qemu

[Piping]

@rsta2 @flypie Hi, I try to run the QEMU that build from your repo in the docker.

I run a few example, but there is no screen output.

More, there is no -bios option for qemu-system-arm. Should it be able to use VNC to display graph?

I built with script

git clone https://github.com/rsta2/qemu.git
cd qemu 
git submodule update --init dtc 
./configure --target-list=arm-softmmu 
make -j$(nproc) && make install && rm -fr qemu

Do you have any suggestion?

[rsta2]

@flypie Great! I will have a look at your repo.

@Piping The QEMU display options are here. I am using gtk which is default on my system. If a VNC server is started at your system, GTK is probably not available or your QEMU build environment is different. Perhaps you can start a VNC viewer to see the screen output. The -bios option must be there, otherwise you cannot load a Circle image.

[rsta2]

@flypie I tried QEMU from your repo on Linux and it is running well with Circle. I think, because your QEMU version is based on the official QEMU version, this is a great progress. I will mention your repo in my QEMU hints for Circle (doc/qemu.txt).

For the networking samples I had to apply this patch to your repo. Otherwise the packets sent by Circle will not arrive at QEMU and Circle will not get a DHCP address. Unfortunately I don't know if this patch has side effects and if this USB CDC Ethernet support is available on Windows too.

This is sample/21-webserver running in QEMU on a Linux host:

[flypie]

I thought I had included that, but obviously not. I have added it now and committed it.

[rsta2]

OK. Thanks.

[flypie]

I got a version built with an external IO Panel. Suggest this might be a better simple gpio demo, which uses 1/4 for in 1/4 for out.

TShutdownMode CKernel::Run (void) { CGPIOPin AudioLeft (GPIOPinAudioLeft, GPIOModeOutput); CGPIOPin AudioRight (GPIOPinAudioRight, GPIOModeOutput);

    CGPIOPin    *AllPins[GPIO_PINS];
    bool        OldPins[GPIO_PINS]={0};
    unsigned    i;

    for (i=0;i<GPIO_PINS/4;i++)
    {
        AllPins[i]=new CGPIOPin(i,GPIOModeOutput); 
    }

    for (i =GPIO_PINS/4;i<GPIO_PINS/2;i++)
    {
        AllPins[i]=new CGPIOPin(i,GPIOModeInput); 
    }

// flash the Act LED 10 times and click on audio (3.5mm headphone jack)
for (i = 1; i <= 10; i++)
{
    m_ActLED.On ();
    AudioLeft.Invert ();
    AudioRight.Invert ();
    CTimer::SimpleMsDelay (200);

    m_ActLED.Off ();
    CTimer::SimpleMsDelay (500);
    if (i % 8 == 0)
    {
                m_Screen.Write ("\n", 1);
    }

            for (unsigned j=0;j<GPIO_PINS/4;j++)
            {
                if(i&1)
                {
                    AllPins[j]->Write(LOW); 
                }
                else
                {
                    AllPins[j]->Write(HIGH); 
                }
            }

            for (unsigned j=GPIO_PINS/4;j<GPIO_PINS/2;j++)
            {
                    bool In=AllPins[j]->Read()==HIGH;
                    if(In!=OldPins[j])
                    {
                        CString Message;
                    Message.Format ("Pin %d Changed state", (unsigned) j);
                        m_Screen.Write ((const char *) Message, Message.GetLength ());  
                        OldPins[j]=In;
                    }
            }

            CString Message;
            Message.Format ("%06d ", (unsigned) i);
            m_Screen.Write ((const char *) Message, Message.GetLength ());                
}

return ShutdownReboot;

} See https://youtu.be/eu8H8R85Zik

[rsta2]

Nice! But unfortunately the first sample should be as simple as possible and should not use the frame buffer. Also it may be dangerous to switch all these pins (0-12) on output, because a user may have connected something there on a real RPi. I think, I will add the directory addon/qemu/ and place QEMU specific samples (including this one) there. Thanks!

[flypie]

I based it on an earlier version of the example.

Though the system does show your current demo does work on the emulator.

[smuehlst]

@flypie: If I understand @rsta2's documentation on QEMU with Circle correctly, your QEMU fork is the preferred one to get a Windows build of QEMU that works with Circle. I tried to follow the documentation http://wiki.qemu.org/Hosts/W32 to do a cross build under Debian, but the instructions are fairly outdated and links are broken. Could you please briefly document how QEMU for Windows can be built from your repository? Thanks!

[rsta2]

@flypie: Sorry, I think my comment was not clear. I have seen your video and it is really nice what QEMU is doing together with your panel application.

Further I meant, I cannot take your example above as replacement for sample/01-gpiosimple, because of the reasons I wrote. Instead I want to add it in a different directory. That does not mean, that I do not appreciate your work.

Generally I have to say that QEMU is a great help for Circle development. Thanks, that you have initiated this and for your work on it!

[flypie]

I wasn't criticising in any way you were quite clear, I didn't take any offence. I'm grateful for you help Sorry for any confusion.

[flypie]

@smuehlst you need to install cygwin first. The install exe is in the zip. Make sure you pick all the mingw option s or if you can just in stall every thing.

The from within the cygwin shell run the script files in the zip run the debugconfigurearm then the debugmake. That should give you a 64 but executable, the bat files can be run from the desktop to actually run qemu.

For a 32 bit build change all the x86_64-w64-mingw32 to i686-w64-mingw32, it a full blown debug version but removing a few of the options should get you an optimised exe. To debug the applications running on qemu you don't need have a debug build of qemu.

I keep the qemu sources in C:\cygwin64\home\John-Bradley\qemu-main. What ever you do when you set up cygwin do not put a space in the username.

I use netbeans to build the circle progs, the run the bat files and attach the debugger.

Qemubuildtools.zip /

[smuehlst]

@flypie: Thanks for the instructions and the build scripts. I have successfully built qemu, so I'm almost there, but I'm getting error messages when I try to launch a Circle kernel image inside qemu. As this is getting too off-topic for Circle itself I will create an issue in your https://github.com/flypie/flypie-pi-qemu repository.

[smuehlst]

I just tried the latest official QEMU build for Windows from https://qemu.weilnetz.de/w64/, and it seems to work at least somewhat.

The weird thing is that it seems to work better when Circle is compiled with -O2 compared to when it is compiled with -O0. I can run various sample programs successfully, e.g. 02-screenpixel, 03-screentext and 08-usbkeyboard. I also can attach gdb and set breakpoints. But as some local variables were optimized out, I recompiled everything with -O0, and in that case the kernel won't run. In this case QEMU bails out with the following message:

PS E:\Users\stm\Documents> C:\QEMU\qemu-system-arm.exe -machine  raspi2 -smp 1 -bios E:\Users\stm\Documents\GitHub\circle\sample\02-screenpixel\kernel7.img -usbdevice keyboard
C:\QEMU\qemu-system-arm.exe: Trying to execute code outside RAM or ROM at 0x44000000`

When I use the option the QEMU option -d guest_errors, I get this:

qemu: fatal: Trying to execute code outside RAM or ROM at 0x44000000
R00=00227fd4 R01=0000000c R02=00016be5 R03=44000000
R04=00000000 R05=00000000 R06=00000000 R07=00000000
R08=00000000 R09=00000000 R10=00000000 R11=00227ffc
R12=00000000 R13=00227fd0 R14=0000a780 R15=44000000
PSR=8000019f N--- A S sys32
s00=00000000 s01=00000000 d00=0000000000000000
s02=00000000 s03=00000000 d01=0000000000000000
s04=00000000 s05=00000000 d02=0000000000000000
s06=00000000 s07=00000000 d03=0000000000000000
s08=00000000 s09=00000000 d04=0000000000000000
s10=00000000 s11=00000000 d05=0000000000000000
s12=00000000 s13=00000000 d06=0000000000000000
s14=00000000 s15=00000000 d07=0000000000000000
s16=00000000 s17=00000000 d08=0000000000000000
s18=00000000 s19=00000000 d09=0000000000000000
s20=00000000 s21=00000000 d10=0000000000000000
s22=00000000 s23=00000000 d11=0000000000000000
s24=00000000 s25=00000000 d12=0000000000000000
s26=00000000 s27=00000000 d13=0000000000000000
s28=00000000 s29=00000000 d14=0000000000000000
s30=00000000 s31=00000000 d15=0000000000000000
s32=00000000 s33=00000000 d16=0000000000000000
s34=00000000 s35=00000000 d17=0000000000000000
s36=00000000 s37=00000000 d18=0000000000000000
s38=00000000 s39=00000000 d19=0000000000000000
s40=00000000 s41=00000000 d20=0000000000000000
s42=00000000 s43=00000000 d21=0000000000000000
s44=00000000 s45=00000000 d22=0000000000000000
s46=00000000 s47=00000000 d23=0000000000000000
s48=00000000 s49=00000000 d24=0000000000000000
s50=00000000 s51=00000000 d25=0000000000000000
s52=00000000 s53=00000000 d26=0000000000000000
s54=00000000 s55=00000000 d27=0000000000000000
s56=00000000 s57=00000000 d28=0000000000000000
s58=00000000 s59=00000000 d29=0000000000000000
s60=00000000 s61=00000000 d30=0000000000000000
s62=00000000 s63=00000000 d31=0000000000000000
FPSCR: 00000000

[rsta2]

@smuehlst: I debugged this. It is a problem with section alignment, which did not arise with -O2. Because of this the Circle initialization code called the wrong address of static constructors. I will prepare a fix.

[rsta2]

@smuehlst: The hotfix is available on all branches.

[smuehlst]

@rsta2: I can confirm that the fix works and I can now debug Circle which is compiled with -O0, thanks!

Just out of curiosity: I saw that the fix only modified the linker script. How can the linker script be affected by the compiler optimization level?

[rsta2]

@smuehlst: The variable __init_start, which points to the beginning of the static initializer table, was previously defined outside of the .init_array section. The sections are always at least 4-byte aligned. So if the .rodata section has a length which is not a multiple of 4 then __init_start was not an aligned address and did not point right to the first initializer.

I have never watched this with -O, -O2 and -O3, but with -O0 it happened now. Now __init_start is defined inside the section and is aligned in any case. So the problem was the length of .rodata. I cannot tell, why this section has been always a multiple of 4 bytes long with -O[123] but it was.

[flypie]

Works on Linux to https://t.co/ZvYa3p2U9o

[flypie]

There was a question somewhere about VNC and QEMU turns out that is the default behaviour on Linux. It is different from windows. I tries but I could not get it to work the same way.

[rsta2]

There are some new features for QEMU in Circle 41. This issue has been around for some time now, without receiving new comments. I think, it should be closed and a new issue can be created for new specific QEMU-related problems, if there are any.