Cannot run statically linked program?

[ghost]

I statically compiled a simple HelloWorld program on x86 linux, but it cannot run on box86.

[ghost]

Can you tell me why?

[ghost]

It got stuck when calling 0xF3 syscall.

[ghost]

Debug level is 2 Dynarec for ARM, with extension: HALF FAST_MULT EDSP NEON VFPv4 IDIVA PageSize:4096 Setting up canary (for Stack protector) at GS:0x14, value:CC120300 Using default BOX86_LD_LIBRARY_PATH: ./:lib/:lib32/:x86/ Using default BOX86_PATH: ./:bin/ Counted 16 Env var Looking for /a.out Read 31 Section header Read 6 Program header Loading Sections Table String (idx = 30) Loading SymTab Strings (idx = 29) Loading SymTab (idx = 28) Loading Dynamic (idx = 0) The GOT.PLT Table is at address 0x80d9000 The GOT Table is at address 0x80d8fd4 The .init is at address 0x80481a8 The .init_array is at address 0x80d76f0, and have 2 elements The .fini is at address 0x80abef8 The .fini_array is at address 0x80d76f8, and have 2 elements The .text is at address 0x8048240, and is 405457 big Loading DynSym Strings (idx = 0) Loading DynSym (idx = 0) Adding "/a.out" as #0 in elf collection Elf Addr(v/p)=0x8048000/0x8048000 Memsize=0x93010 (align=0x1000) Elf Stack Memsize=1048576 (align=16) Elf TLS Memsize=48 (align=4) Allocating 0x94000 memory @0x8048000 for Elf "/a.out" Loading block #0 @0x8048000 (0x8e394/0x8e394) Loading block #1 @0x80d76e0 (0x2c3c/0x3930) Loading TLS block #3 @0x2c7e880 (0x10/0x30) Calc stack size, based on 1 elf(s) Stack is @0xf35b9020 size=0x800000 align=0x10 Allocate a new X86 Emu, with EIP=(nil) and Stack=0xf35b9020/0x800000 Setup X86 Emu And now export symbols / relocation for /a.out... PLT Resolver injected in got at 0x80d8fdc Entry Point is 0x8048730 Start x86emu on Main Run X86 (0x2c6d0a0), EIP=0x8048730, Stack=0xf35b9020 0x80934f7: Calling syscall 0x2D (45) (nil) 0x3f 0x80d9000 0x80d9000 0xd40 => 0x2cab000 0x80934f7: Calling syscall 0x2D (45) 0x2cabd40 0x3f 0x80d9000 0x80d9000 0x2cabd40 => 0x2cabd40 0x8049505: Calling syscall 0xF3 (243) 0xf3db8720 (nil) 0x80d9000 0x80d9000 0x4set_thread_area(0xf3db8720[-1/base=0x2cab840/limit=1/32bits:1/0/0...]) => 0x0

[ghost]

This is the debug of box86.

[ptitSeb]

Box86 works by bypassing ld-linux and libc mecanism, among other thing. Using a static build embbed all libc and friend inside the program, preventing box86 to do it's magic... So there is only the syscall that the regular libc / libm / pthread are doing, that will also conflict with the arm version of libc and phtread that box86 is used.

TL;DR: don't use static build, most will not work.

[ghost]

You seem to define my_read and my_open in wrappedlibc.c, why not let syscall read,open handle in my_read,my_open?

[ptitSeb]

because syscall read is interuptible (by signals), while libc read handle that.

[ghost]

What about my_open?

[ptitSeb]

If I want to use libc read, I need to use libc open... Why are you asking that? Is there an issue here?

[ghost]

Cannot use x86 libc. Is this a problem with box86 syscall 0xf3?

[ptitSeb]

No, x86 libc canot be used because it will conflict with libc used by box86. syscall 0xf3 is a tricky one. It works with wine. The return 0 indicate a success in the function. I don't know why it would not go further, but I know it will later confilct with pthread library anyway.

static linked program are not supported for now.

[ghost]

Will you add support for static linked programs in the future?

[ptitSeb]

Maybe, but I have to find some trick to make both "libc" live together. But it's not on my immediate todo.

[ghost]

I want to test the speed of box86 fully emulating x86.

[ghost]

Can you tell me how to do it?

[ptitSeb]

You mean, without using wrapped function?

[ghost]

Yes.

[ptitSeb]

I don't see how. you can force a wrapped lib to use emulated lib instead, with BOX86_EMULATED_LIBS=libXXX.so;libYYY.so but libc is quite special, and using emulated one will not work. If you have something that is CPU only and doesn't do many printf or memcpy, it shouldn't change much.

[ghost]

How fast is box86 dynarec?

[ptitSeb]

It's pretty fast, but I haven't done any competitve benchmark, so I'm not sure how fast, and how it compare to other x86 emulators

[ghost]

How much speed is improved when using wrapper functions?

[ptitSeb]

That really depend on the program usage of function. It really depends!

[ptitSeb]

Here, I have done a quick 7z benchmark (using 7z b command). I used my Pandora for that, and a native 7z build by me as a reference and x86 7z comming from Debian (using sudo apt install p7zip-full:i386).

Here the result of the i386 version using box86

Dynarec for ARM, with extension: HALF FAST_MULT EDSP NEON VFPv3 SWP PageSize:4096
Using default BOX86_LD_LIBRARY_PATH: ./:lib/:lib32/:x86/
Using default BOX86_PATH: ./:bin/
Counted 61 Env var
Looking for ./7z
argv[1]="b"
Using native(wrapped) libpthread.so.0
Using native(wrapped) libdl.so.2
Using emulated /mnt/utmp/codeblocks/usr/lib/i386-linux-gnu/libstdc++.so.6
Using native(wrapped) libm.so.6
Using native(wrapped) libc.so.6
Using native(wrapped) ld-linux.so.2
Using native(wrapped) librt.so.1
Using emulated /mnt/utmp/codeblocks/usr/lib/i386-linux-gnu/libgcc_s.so.1

7-Zip [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,32 bits,1 CPU Intel(R) Pentium(R) 4 CPU 1000MHz (601),ASM)

Using emulated ./7z.so
              Intel(R) Pentium(R) 4 CPU 1000MHz (601)
CPU Freq:   823   823   827   825   793   787   827   824

RAM size:     501 MB,  # CPU hardware threads:   1
RAM usage:    435 MB,  # Benchmark threads:      1

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:        174    99    171    170  |       2895    99    249    247
23:        171    99    176    175  |       2854    99    249    247
24:        168    99    183    181  |       2828    99    250    248
25:        165    99    191    188  |       2744    98    249    244
----------------------------------  | ------------------------------
Avr:              99    180    179  |               99    249    247
Tot:              99    215    213

And here using a native build:

7-Zip [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,32 bits,1 CPU LE)

LE
CPU Freq:   992   912   993   994   988   995   996   992

RAM size:     501 MB,  # CPU hardware threads:   1
RAM usage:    435 MB,  # Benchmark threads:      1

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:        458    99    449    446  |       9942    98    870    849
23:        448   100    459    457  |       9950    99    867    861
24:        432    99    468    466  |       9676    99    857    850
25:        418    99    481    478  |       9465    99    849    843
----------------------------------  | ------------------------------
Avr:              99    464    462  |               99    861    851
Tot:              99    663    656

So, if you look at the MIPS, for compression it's 464 vs 180 where for decompression it's 861 vs 249. So, for this bench (that should be mainly CPU), it's between 2.5 to 3.5 slower than native.

[ptitSeb]

Native PI4, I get:

7-Zip [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE)

LE
CPU Freq:  1013  1484  1481  1498  1478  1497  1496  1497  1498

RAM size:    3776 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       3443   346    969   3350  |      90596   397   1949   7729
23:       3345   349    977   3409  |      88290   398   1921   7639
24:       3277   364    969   3524  |      85284   397   1885   7487
25:       3207   370    989   3662  |      82052   398   1836   7303
----------------------------------  | ------------------------------
Avr:             357    976   3486  |              397   1898   7539
Tot:             377   1437   5513

Current box86 version on the PI4:

Dynarec for ARM, with extension: HALF FAST_MULT EDSP NEON VFPv4 IDIVA PageSize:4096
Using default BOX86_LD_LIBRARY_PATH: ./:lib/:lib32/:x86/
Using default BOX86_PATH: ./:bin/
Counted 49 Env var
Looking for ./7z
argv[1]="b"
Using native(wrapped) libpthread.so.0
Using native(wrapped) libdl.so.2
Using emulated /usr/lib/i386-linux-gnu/libstdc++.so.6
Using native(wrapped) libm.so.6
Using native(wrapped) libc.so.6
Using native(wrapped) ld-linux.so.2
Using native(wrapped) librt.so.1
Using emulated /usr/lib/i386-linux-gnu/libgcc_s.so.1

7-Zip [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs Intel(R) Pentium(R) 4 CPU 1500MHz (601),ASM)

Using emulated ./7z.so
              Intel(R) Pentium(R) 4 CPU 1500MHz (601)
CPU Freq:  1421  1438  1442  1439  1442  1439  1442  1441  1441

RAM size:    3776 MB,  # CPU hardware threads:   4
RAM usage:    882 MB,  # Benchmark threads:      4

                       Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
         KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       1277   269    461   1243  |      27485   397    591   2345
23:       1286   275    477   1311  |      27054   396    591   2341
24:       1277   279    493   1374  |      26308   394    586   2310
25:       1260   286    502   1439  |      25241   396    568   2246
----------------------------------  | ------------------------------
Avr:             277    484   1342  |              396    584   2310
Tot:             336    534   1826

[ghost]

Can you tell me how to output the assembly instructions executed by the running program?

[ptitSeb]

There is 2 way: 1/ is to build a box86 with trace enabled. Read the COMPILE.md, it can be tricky. That will build a box86t where you can use BOX86_TRACE=1 to trace each x86 instruction run. It's a HUGE trace that can be generated, and I don't recommend using that trace. 2/ you can get all x86->arm block generated. Trace is shorter that way. to use that, use BOX86_DYNAREC_DUMP=2 to generate colored dump. I suggest you also use BOX86_TRACE_FILE=trace.txt to generate the trace in a specific file. Note that box86 (without "t") does not decode x86 instruction in detail. If you want precise x86 dump, you need box86t and also you need to enable trace without generating it, using BOX86_TRACE=0x0-0x1

[ptitSeb]

What will use all those information for?

[ghost]

But I need to trace arm elf, Do not use box86.

[ghost]

Can you tell me?

[ptitSeb]

The ARM ELF? What arm elf???

[ptitSeb]

There is no easy way to trce any program anyway, arm or x86. Using emulator you get function like trace and all, but when you are running native software, there is only debugging and disassembling...

[ghost]

When I run Ra2 with box86 wine, there are many dlopen errors. Is this normal?

Not an ELF file (sign=MZ�) Warning: Cannot dlopen("/home/pi/.wine/dosdevices/z:/home/pi/Ra2/BINKW32.DLL"/0xb517d8, 2) Warning: Cannot dlopen("/usr/bin/../lib/i386-linux-gnu/wine/binkw32.dll.so"/0x3fca16c, 2) Using native(wrapped) libfreetype.so.6 Using native(wrapped) libfontconfig.so.1 Using emulated /usr/bin/../lib/i386-linux-gnu/wine/winex11.drv.so Using native(wrapped) libXext.so.6 Using native(wrapped) libX11.so.6 Using native(wrapped) libxcb.so.1 Using native(wrapped) libXau.so.6 Using native(wrapped) libXdmcp.so.6 Using native(wrapped) libXinerama.so.1 Using native(wrapped) libXxf86vm.so.1 Using native(wrapped) libXrender.so.1 Using native(wrapped) libXrandr.so.2 Using native(wrapped) libXcomposite.so.1 Using native(wrapped) libXi.so.6 Using native(wrapped) libXcursor.so.1 Using emulated /usr/bin/../lib/i386-linux-gnu/wine/uxtheme.dll.so 0029:err:ole:CoGetClassObject class {1440ad10-6aa8-11d1-b6f9-00a024ddafd1} not registered 0029:err:ole:CoGetClassObject class {1440ad10-6aa8-11d1-b6f9-00a024ddafd1} not registered 0029:err:ole:create_server class {1440ad10-6aa8-11d1-b6f9-00a024ddafd1} not registered 0029:err:ole:CoGetClassObject no class object {1440ad10-6aa8-11d1-b6f9-00a024ddafd1} could be created for context 0x7 Not an ELF file (sign=MZ�) Warning: Cannot dlopen("/home/pi/.wine/dosdevices/z:/home/pi/Ra2/Blowfish.dll"/0xb6b0b8, 2) Warning: Cannot dlopen("/usr/bin/../lib/i386-linux-gnu/wine/blowfish.dll.so"/0x4c887b5, 2) Not an ELF file (sign=MSFT) Warning: Cannot dlopen("/home/pi/.wine/dosdevices/z:/home/pi/Ra2/Ra2.tlb"/0xb6b308, 2) Warning: Cannot dlopen("/usr/bin/../lib/i386-linux-gnu/wine/ra2.tlb.so"/0x4b39808, 2)

[ptitSeb]

Yes, that's normal. It's just wine trying to see if the lib is a linux one (so part of wine itself). Box86 shows the dlopen attemps, for debugging purpose, but this error normally don't show up on regular x86 linux.

[ghost]

How much faster is box86 dynarec than qemu?

[ptitSeb]

I don't know, I haven't any qemu-i386 installed. Bt I'll see on discord if someone can do the 7zip bench on it.

[ghost]

Can you tell me how to learn about dynarec and binary translation? I really want to know.

[ptitSeb]

I don't know. I learned by meself. Look on the net, there are some tutorial on some dynarec, mostly on x86 but that fine, they gives an overview on how a dynarec work, and what are the main building brick that compose one.

[trapexit]

https://www.amazon.com/Virtual-Machines-Versatile-Platforms-Architecture/dp/1558609105

Virtual Machines by Smith and Nair is pretty good overview on the topic.

[ghost]

https://www.amazon.com/Virtual-Machines-Versatile-Platforms-Architecture/dp/1558609105

Virtual Machines by Smith and Nair is pretty good overview on the topic.

Is there anything about dynarec in this?

[trapexit]

Yes. It's an overview of the whole of the virtual machine topic which dynamic recompilation is a subset.

[ghost]

@ptitSeb why you handle /sys/devices/system/cpu/cpuX/topology/core_id? It already exists on arm.

[ptitSeb]

It didn't exist on my test machine... I'll add a test, to use existing one if they exists...

[ptitSeb]

this is a special case, if argv is NULL. If argv is not null, it has been treated in the block before.

[ghost]

Sorry,I didn't see the answer just now.

[ptitSeb]

anyway, the case of argv == NULL should not exist, the man page for execv says this should not be NULL IIRC

[ghost]

Can you add support for vulkan?

[ptitSeb]

Yes, it's on my roadmap, will be done during the 0.1.5 version most probably.

[ghost]

It seems that my_execvp has some problems. The first parameter "path" is not the full path of the file.

[ghost]

You cannot use the FileIsX86ELF function to detect whether the file is an x86 elf.