Embree fails building on OpenBSD 7.0

RHL120 commented 2 years ago

Godot version

dd3acd74bbd0c682f489693e31961aba54bde10a

System information

OpenBSD 7.0, OpenBSD clang version 11.1.0

Issue description

I am trying to build Godot on OpenBSD 7.0. I followed this guide https://docs.godotengine.org/en/stable/development/compiling/compiling_for_x11.html (the guide says I should install scons with pkg_add but that doesn't work becuase the python version that is used with it is 2.7 and godot uses type anotations which are not supported by python2.7 so I used pip install scons). when I run scons platform=linuxbsd -j2 use_llvm=yes I get the follwing error:

thirdparty/embree/common/sys/sysinfo.cpp:630:5: error: unknown type name 'cpu_set_t'
    cpu_set_t set;
    ^
1 error generated.
scons: *** [thirdparty/embree/common/sys/sysinfo.x11.tools.64.o] Error 1
scons: building terminated because of errors.
[Time elapsed: 00:08:05.616]

Steps to reproduce

doas pkg_add llvm python3
pip install scons # python version 3.8.12
git clone https://github.com/godotengine/godot.git -b 3.x
cd godot
scons platform=linuxbsd -j2 use_llvm=yes

Minimal reproduction project

No response

RHL120 commented 2 years ago

Sorry I used a newer version of the docs (for version 4) and I was compiling on version 3 I assume

RHL120 commented 2 years ago

Sorry I used a newer version of the docs (for version 4) and I was compiling on version 3 I assume

Nope same error :cry:

Calinou commented 2 years ago

It's likely that Embree (used in the CPU lightmapper in 3.x and raster occlusion culling in master) is not supported on OpenBSD. Try passing the module_embree_enabled=no SCons option.

RHL120 commented 2 years ago

sed in the CPU lightmapper in 3.x and raster

I tried it. I am still getting the same error...

Calinou commented 2 years ago

sed in the CPU lightmapper in 3.x and raster

I tried it. I am still getting the same error...

My mistake, it's module_lightmapper_cpu_enabled=no.

RHL120 commented 2 years ago

sed in the CPU lightmapper in 3.x and raster

I tried it. I am still getting the same error...

My mistake, it's module_lightmapper_cpu_enabled=no.

Sorry if this is starting to get annoying but I am still getting the same error :(

akien-mga commented 2 years ago

sed in the CPU lightmapper in 3.x and raster

I tried it. I am still getting the same error...

My mistake, it's module_lightmapper_cpu_enabled=no.

In the 3.x branch, it's module_raycast_enabled=no.

akien-mga commented 2 years ago

We probably need to do similar fixes for OpenBSD support as we've had to do for Android: https://github.com/godotengine/godot/blob/master/thirdparty/embree/patches/godot-changes-android.patch

Upstream Embree seems to have support for FreeBSD but not OpenBSD.

RHL120 commented 2 years ago

sed in the CPU lightmapper in 3.x and raster

I tried it. I am still getting the same error...

My mistake, it's module_lightmapper_cpu_enabled=no.

In the 3.x branch, it's module_raycast_enabled=no.

I tried that it kind of worked. I am getting a diffrent error now:

[Initial build] Compiling ==> thirdparty/libvpx/vpx_dsp/x86/vpx_subpixel_8t_intrin_ssse3.c
thirdparty/libvpx/vpx_dsp/x86/vpx_subpixel_8t_intrin_ssse3.c:92:18: error: always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3', but would be inlined into function 'vpx_filter_block1d4_h8_intrin_ssse3' that is compiled without support for 'ssse3'
    srcRegFilt1= _mm_shuffle_epi8(srcReg, shuffle1);
                 ^
thirdparty/libvpx/vpx_dsp/x86/vpx_subpixel_8t_intrin_ssse3.c:93:18: error: always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3', but would be inlined into function 'vpx_filter_block1d4_h8_intrin_ssse3' that is compiled without support for 'ssse3'
    srcRegFilt2= _mm_shuffle_epi8(srcReg, shuffle2);
                 ^
thirdparty/libvpx/vpx_dsp/x86/vpx_subpixel_8t_intrin_ssse3.c:96:19: error: always_inline function '_mm_maddubs_epi16' requires target feature 'ssse3', but would be inlined into function 'vpx_filter_block1d4_h8_intrin_ssse3' that is compiled without support for 'ssse3'
    srcRegFilt1 = _mm_maddubs_epi16(srcRegFilt1, firstFilters);
                  ^
thirdparty/libvpx/vpx_dsp/x86/vpx_subpixel_8t_intrin_ssse3.c:97:19: error: always_inline function '_mm_maddubs_epi16' requires target feature 'ssse3', but would be inlined into function 'vpx_filter_block1d4_h8_intrin_ssse3' that is compiled without support for 'ssse3'
    srcRegFilt2 = _mm_maddubs_epi16(srcRegFilt2, secondFilters);
                  ^
4 errors generated.
scons: *** [thirdparty/libvpx/vpx_dsp/x86/vpx_subpixel_8t_intrin_ssse3.x11.tools.64.o] Error 1
scons: building terminated because of errors.
[Time elapsed: 00:03:34.288]

Acually I just forgot to use llvm. Now I have this error:

clang++: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
clang++: warning: argument unused during compilation: '-static-libstdc++' [-Wunused-command-line-argument]
ld: error: unable to find library -l:libatomic.a
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
scons: *** [bin/godot.x11.tools.64.llvm] Error 1
scons: building terminated because of errors.
[Time elapsed: 00:53:55.710]

akien-mga commented 2 years ago

Indeed, that worked to bypass the Embree build issue. Now you have a build issue for webm/libvpx, which you can also disable with module_webm_enabled=no.

cederom commented 2 years ago

Nice to hear from BSD world :-) Greetings from FreeBSD :-) We have Godot 3.2 port at the moment created by @sambler, 3.3 did not build, will have to try with 3.4 :-) Thank you for this marvelous tool that I found after Blender removed Game Engine :-)

And I can cross compile mobile applications straight from my FreeBSD! I have enough of JavaScript + ReactNative + Expo ugly dirty mess.. and I will try to move with mobile application development to Godot. Godot seems the only alternative so far to JavaScript and it works out of the box on BSD :-) I also consider porting LVGL + Python but Godot seems to just work! :-)

A bit off topic question: what GUI does Godot use for UI components and is it possible to extend them with new components (i.e. calendar)? Does it have access to local data store? Will everything work on BSD in the future?

BIG TANK U FOR GODOT! :-)

Calinou commented 2 years ago

what GUI does Godot use for UI components

Godot uses its own UI system – it does not rely on an existing UI toolkit.

is it possible to extend them with new components (i.e. calendar)?

Yes, it's possible to create your own instanced scenes with custom behavior to act as a calendar.

Does it have access to local data store?

Yes, the File class can be used to read/write files to user:// (which is automatically mapped to IndexedDB on HTML5). You can also use the JavaScript singleton to call JavaScript code, which allows for using LocalStorage.

Will everything work on BSD in the future?

Yes, as long as contributors remain interested to test it and fix porting issues :slightly_smiling_face:

cederom commented 2 years ago

Thank you @Calinou ! Godot rox! :-)

@RHL120 you may want to take a look how things are solved on FreeBSD that may give you some hints on OpenBSD? :-)

https://cgit.freebsd.org/ports/tree/devel/godot

https://cgit.freebsd.org/ports/tree/graphics/embree

akien-mga commented 1 year ago

CC @omar-polo - would be good to assess whether this is still relevant for 3.6 and 4.1, and what changes we could possibly upstream in Godot to make it easier to build on OpenBSD.

omar-polo commented 1 year ago

@akien-mga embree needs patching on both godot 3.x and 4.x. I've sent a PR upstream https://github.com/embree/embree/pull/379 but it was ignored so far :)

I'm currently carrying two patches to build on OpenBSD:

I could have put more effort in the implementation of getVirtual/ResidentMemoryBytes, but for other OSes it returns zero too so I thought it was fine.

I've seen that in some cases Godot carries some patches for the things in thirdparty, but I'm not sure these are worthy enough to be included.

akien-mga commented 1 year ago

I think it's worth including in Godot, you could merge both changes as a single thirdparty/embree/patches/openbsd-support.patch (and apply it, we use the patches for documentation purposes but the code should be patched so it compiles out of the box).

The long patch might fail compiling on non-OpenBSD though since I think for most compilers and platforms long is the same as int64_t, so it would be seen as a duplicate of the existing define. You could enclose it in #if defined(__OpenBSD__) to be safe, at least in the version we'd merge in Godot (for upstream Embree, you can wait for them to decide how they want to handle this).

godotengine / godot