search

godotengine / godot-proposals

Godot Improvement Proposals (GIPs)
MIT License
1.06k stars 65 forks source link

Add official Linux ARM/Raspberry Pi editor and export template binaries #988

Closed Schweini07 closed 5 months ago

Schweini07 commented 4 years ago

Bugsquad edit: You can use https://github.com/hiulit/Unofficial-Godot-Engine-Raspberry-Pi until official builds are provided.

The re-open of https://github.com/godotengine/godot/issues/2671.

NOTE: This issue is more about Godot 3.2, as Godot 3.0.6 and 3.1 are possible to get on the pi, with the costs of a few bugs, for more information read below.

If you have anything to add, please tell me.

Takeaways:

Other interesting things I found:

Describe the project you are working on: Applies to any project made on a Pi.

Describe the problem or limitation you are having in your project: It's hard perhaps even impossible, to get Godot 3.2 working on the Pi.

Describe the feature / enhancement and how it helps to overcome the problem or limitation: It's hard to tell why Godot 3.2 doesn't work with Pis below 4. My assumption is that it has to do something with the missing GLES3 or because the Pi has an ARM processor (https://godotengine.org/qa/13114/export-godot-project-for-raspberry-pi). I think it could help to look into: https://github.com/efornara/frt

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams: As I don't know the exact problems, I can't tell.

If this enhancement will not be used often, can it be worked around with a few lines of script?: A few lines of code probably won't do it, but Godot 3.2 is desired from Raspberry Pi users.

Is there a reason why this should be core and not an add-on in the asset library?: It isn't possible.

efornara commented 3 years ago

As a clarification, FRT (export template) does support Godot 3.2. In fact, I have just noticed this proposal when publishing 3.2.2-beta4 (https://sourceforge.net/projects/frt/files/previews/). Godot 3.2.1-stable is still available (it is the default download).

The unmodified x11 platform of Godot 3.2 should work out of the box on a Pi 4 and, by changing the OpenGL system settings, on a Pi 2/3 too. You should add a compiler flag to select hard float when generating the binaries, and I think some people have problems when using GCC, but, apart from that, it should be fine.

realkotob commented 3 years ago

I've worked with godot 3.2 and 3.1 on the raspberry PI 3A and 3B+

It works pretty well for 2D.

However, even for 2D there are a lot of optimizations and graphical limitations that the user needs to do and figure out on their own.

I still think it would be useful to make an official RPI export template, and I would love to help to make it happen since it would be useful to a lot of users, but this can only happen after https://github.com/godotengine/godot-proposals/issues/877 is decided since RPI is considered a low-end platform.

Example limitations: Particles do not work at all, and there's a texture memory limit that means that it's better to load things as needed instead of loading everything into the scene at once (or else they will appear black ingame).

antv199 commented 3 years ago

I found this Vulkan update for the Raspberry Pi. This might come in handy.

Coolcreeper27 commented 3 years ago

This would help for beginning game devs to be able code any time

Calinou commented 3 years ago

@Coolcreeper27 Please don't bump issues without contributing significant new information. Use the :+1: reaction button on the first post instead.

dh928 commented 3 years ago

I've worked with godot 3.2 and 3.1 on the raspberry PI 3A and 3B+

It works pretty well for 2D.

However, even for 2D there are a lot of optimizations and graphical limitations that the user needs to do and figure out on their own.

I still think it would be useful to make an official RPI export template, and I would love to help to make it happen since it would be useful to a lot of users, but this can only happen after #877 is decided since RPI is considered a low-end platform.

Example limitations: Particles do not work at all, and there's a texture memory limit that means that it's better to load things as needed instead of loading everything into the scene at once (or else they will appear black ingame).

How did you get it to work on your pi 3B+?

realkotob commented 3 years ago

@dh928 You can use the FRT thing directly or use this thing which makes it run within RetroPie.

Note that some things are not supported, like particle effects or Websockets (but fortunately I used the other TCP option for multiplayer, so wasn't an issue since I changed 1 line).

ShalokShalom commented 3 years ago

Would be nice to find it here 🥳

Calinou commented 3 years ago

@ShalokShalom This can be handled by the Arch Linux ARM packagers already, just like other Linux distributions. This proposal is about distributing official, first-party ARM binaries for Linux. You'd download them on the Godot website just like x86 binaries.

ShalokShalom commented 3 years ago

Yeah, but getting in touch with the distros and actually coming up with a ready to go build script would not hurt the visibility of the project. In the end, its all about making the software available to the people. I could do that 😊

Edwardphantom commented 3 years ago

Yeah, but getting in touch with the distros and actually coming up with a ready to go build script would not hurt the visibility of the project. In the end, its all about making the software available to the people. I could do that 😊

man, plz, its gonna be amazing !

nobody5050 commented 3 years ago

Bumping because this would be an awesome thing for godot to do and would also bring more users to the engine.

Edit: the newly released pi 400 is surely powerful to run godot without limitations.

hiulit commented 3 years ago

Hi everybody!

I'm writing from a Raspberry Pi 4 4GB where I successfully compiled all major versions (2.1.6, 3.1.0, 3.1.1, 3.1.2, 3.2.0, 3.2.1, 3.2.2, 3.2.3 and also 3.2.4 beta 4) of Godot , both the editor and the export templates :tada:

I could't compile 3.0.6 because it doesn't support GLES3. I'm not sure if that's the cause, but I couldn't anyway.

I want to clarify that I'm not an expert at compilattion, architectures and all that stuff, but I'm tech-savy enough to understand them and tinker with them.

I'll try to explain how I did it, what is working and what is not.

But first of all, I want to credit all the sources where I got the information from:

How I compiled Godot (X11) for the Raspberry Pi

For all I can tell, they all seem to work the same.

I don't know the differences between all those flags, but I found the GCC ARM Options site where all those options all explained.

As you can see, all the binaries are compiled using Clang. GCC is suposed to be better (?), so I also compiled a version using GCC (without lto) to compare it with the Clang version and it seems to performe the same as the Clang binaries. I could't compile using GCC with lto because it's very RAM consuming and apparently my Raspberry Pi run out RAM.

Editor (IDE)

The only versions that work perfectly are 2.1.6 and 3.2.4 beta 4.

I'm guessing that 2.1.6 works well because it's less demanding (?). And for 3.2.4 beta 4, it seems that it woks great because of the amazing job the Godot team has done with the GLES2 batching.

All the versions inbetween are very laggy, at least when run at fullscreen (1920x1200). If the editor window is smaller, at between ~1/2 and ~1/3 of that resolution, it performs correctly. I guess if all the new batching features were ported back to those versions they would work perfectly as well.

Here's a comparison example between 3.2.3-stable and 3.2.4 beta 4.

Export templates

All versions seem to work fine.

Conclusions

I think, at least with the Raspberry 4, that it's worth investing the time on adding official support for it. If not for the editor, at least for the export templates.

I can't speak for other ARM single-board computers, but I can speak for the Raspberry Pi 4 and I think it's a perfectly capable device to be used for developing a 2D game.

The Raspberry Pi 4 is the most known single-board computer and it's reasonably priced, even cheap, if I might add.

Giving support to it would give a lot of people who can't afford high-end computers to be able to develop games.

Also, the Raspberry Pi is well know for being an emulation/gaming device and having the ability to play Godot games on it would be great for exposure.

Calinou commented 3 years ago

I could't compile using GCC with lto because it's very RAM consuming and apparently my Raspberry Pi run out RAM.

To compile an LTO-enabled build, you'll need a Raspberry Pi with 8 GB of RAM and lots of patience, or cross-compile the ARM binary from another machine with at least 8 GB of RAM.

hiulit commented 3 years ago

I could't compile using GCC with lto because it's very RAM consuming and apparently my Raspberry Pi run out RAM.

To compile an LTO-enabled build, you'll need a Raspberry Pi with 8 GB of RAM and lots of patience, or cross-compile the ARM binary from another machine with at least 8 GB of RAM.

Do you think that compiling with lto we would see noticeable improvements, performance-wise? Is it worth trying?

Calinou commented 3 years ago

Do you think that compiling with lto we would see noticeable improvements, performance-wise? Is it worth trying?

On x86, it makes the Godot binary 20-25% smaller and about 20-25% faster as well. I think it's very much worth it, but the performance gains may be different on ARM.

efornara commented 3 years ago

The official export templates (x11) work well on a Raspberry Pi 3 too. Today. Latest stable Godot release.

I am not complaining too much about a few additional stars and downloads that FRT is getting, but, really, most people who download FRT would be better served by downloading an official yet unsupported export template for Raspberry Pi 3/4.

In my opinion, at this point, FRT should become a niche "port", downloaded mostly by people targeting Raspberry Pi Zero, Raspberry 2/3 with Legacy driver or platforms where you have to use KMS/DRM (and possibly Wayland, if I find some time to add it).

I think that some sort of official support should be seriously considered for a future Godot 3.2 release, ideally 3.2.4-stable.

Calinou commented 3 years ago

I think that some sort of official support should be seriously considered for a future Godot 3.2 release, ideally 3.2.4-stable.

We want it to happen too, but I don't control the release process itself.

efornara commented 3 years ago

I suppose someone could publish unofficial templates for Godot, as occasionally happens. From what I have seen, these tend not to remain up-to-date for long. My guess is for lack of discoverability, which leads to lack of downloads / stars / likes, and ultimately to loss of interest from the person attempting it.

At the end of the day, I can live without an official export template. What I am missing is a reason, I suppose by who controls the release process itself, because, as things are now, the situation doesn't make much sense to me.

Calinou commented 3 years ago

What I am missing is a reason, I suppose by who controls the release process itself, because, as things are now, the situation doesn't make much sense to me.

The build containers need to be updated to allow for ARM cross-compilation. While feasible, it's not 100% trivial.

hiulit commented 3 years ago

@efornara Do you have any tips on which flags are best suited for compiliing for both the Raspberry Pi 3 and 4?

As you can see in my post above I used diferrent flags, but as far as I could see, they all seemed to perform equally.

efornara commented 3 years ago

@Calinou

The build containers need to be updated to allow for ARM cross-compilation. While feasible, it's not 100% trivial.

Fair enough. I am planning to soon focus on cross-compiling for FRT too. I did a quick test a few months ago, and I think I only had to make just a few minor changes to my detect.py to get a binary that started without crushing.

@hiulit

Do you have any tips on which flags are best suited for compiliing for both the Raspberry Pi 3 and 4?

Not really. I copied the flags that I am using from an unchallenged forum post on the Raspberry Pi forums ages ago. Today, I would use something different (i.e. less specific). I have never changed them because it wasn't broken, so there was no point in risk breaking it. I do plan to simplify them when I switch to cross compiling: I am making a break anyway, so I might as well go all the way.

Same boat as you when it comes to performance. I tried once a benchmark and the results were inconclusive and too noisy.

About LTO. I had problems generating a Godot 3.something editor build, even on a PC with 8G of RAM running qemu-user-static overnight. There is a possibility that a 8G Pi4 won't be enough to generate a 32-bit binary, and you might need to cross compile anyway.

hiulit commented 3 years ago

Hi everyone! In case anyone is interested, I've just released Unofficial Godot Engine for the Raspberry Pi: Unofficial Godot Engine editor and export templates binaries for the Raspberry Pi.

Edwardphantom commented 3 years ago

Hi everyone! In case anyone is interested, I've just released Unofficial Godot Engine for the Raspberry Pi: Unofficial Godot Engine editor and export templates binaries for the Raspberry Pi.

You are a hero bro, i wanna be like you when i grow up.

alexfreyre commented 3 years ago

Nice! personally I feel extremely happy because of this PR :)

I am a visual artist from the field of fine arts and I use Godot for what we call "new media art" so, I do applications as interactive virtual experiences, for that reason I don't make games nor publish anything like that,

I do exhibitions and I need to get them running on galleries, so SBC always have been the 1st option for me because its low cost. That said, for me too is very important this arm support, but not only for RPI, I explain why:

FRT project is amazing because it runs on almost a raw archlinux arm system from command-line and the performance is much better than from X11, but it is focused on RPIs, which is really nice! they are the most known SBC but not the cheapest nor powerful either.

For other boards we need to compile and run from X11 which is considerably slow while those boards are much more powerful than a RPIs, for example Odroid N2+ with a 850MHz GPU at $63, I have found other which I can't remember the brand nor the processor but it cost $19 with a 650MHz GPU... in my humble opinion it worth support a wide scenario (I am not a developer I can't imagine the time and the effort to do that) because arm powered custom devices/creations can be very useful/cheap and widely used from art piceces, design, custom gaming-machines, and who knows!!!

So my point/wishes/requests are:

  1. Please support in general arm and aarch64 (being arm6 useful on 7 and 8), would be great, because there are a lot of boards on the market with a lot of different processors
  2. if it is possible, please add the wonderful feature to run godot releases from command-line like FRT but not only limited for RPIs or the processors they have. (if this involve a complex workflow with several drivers or something like that, consider support a few of the most known GPUs)

amazing all the people concerned about this!!! (sorry about my english and this looong message :) )

silverkorn commented 3 years ago

@alexfreyre , kinda same here. I'm trying to show a friend in this field that they can use Godot for artistic exposition at low price and few knowledge (They were taught Python in their courses)

efornara commented 3 years ago

I hope the moderators don't mind if I comment about FRT here.

@alexfreyre

FRT isn't limited to Raspberry Pi. Here is my own experience with an Orange Pi PC (AllWinner H3 with Mali 400).

I started with a "server" buster image from Armbian and compiled a newer version of Mesa myself. The official FRT binaries work both on the console (KMS/DRM) and on X11. The Godot 2.1 2D platformer runs at 30 fps on the console and at 60 fps on X11.

My godot2-editor from https://github.com/efornara/godot2-rpi-repo (basically, Godot 2.1 patched to use EGL/ES2 instead of GLX/GL2) works too. I wouldn’t say that it is perfectly smooth, but I have found it quite useable, much more than a 3.2.4beta4 editor on a Pi 3B+.

I did try once a "desktop" buster image from Armbian, and I didn’t get any hardware acceleration, so that might have been the problem you got. I suggest to check es2_info and es2gears to make sure that it really is a Godot / FRT problem and not an OS problem. IIRC, I got a "DRI2: failed to authenticate" as warning that there was no hardware acceleration.

For anyone looking for cheap boards, I also tried the Godot 2.1 2D platformer (console) on a Orange Pi One (same as the PC, but with fewer ports and only 512M of RAM). Again, it works fine, but, to be fair, having to compile Mesa yourself might not be for everyone.

Also, see: https://github.com/efornara/frt/issues/25

alexfreyre commented 3 years ago

@efornara , I hope that in some moment Godot supports natively EGL, and if this PR is considered finally that the export templates could support EGL (only if performance is better)

but, how can platformer 2D runs at 30 fps on the console and at 60 fps on X11? isn't EGL faster than X11? x11 need some hardware resources..

efornara commented 3 years ago

Unless you are using a composer, or you are filling up your memory with heavy X11 applications, the overhead of using X11 instead of KMS/DRM should be negligible. I did my test with only openbox and xterm running. Pi 0-3 + legacy driver is a special case with different trade-offs.

I didn't investigate why KMS/DRM was slower. One explanation could be that the platformer size was smaller than the display. While on KMS/DRM the game "window" was the same size as on X11, maybe the size of actual framebuffer used was as big as the full display (1024x600 in my case). There might be some possible optimization to be made on the FRT side. When/if I put my hands on the kmsdrm module I might have a look.

xarses commented 3 years ago

I'm happy that you have gone on about FRT and other alternatives to official support here, as this helps lay the path for people that want understand how to do this themselves

samdze commented 3 years ago

I recently managed to build Godot 3.2.3 export templates on a Allwinner R16 (A33), the cpu contained in a portable device called GameShell. Not the easiest task, but in the end it turned out very well for 2D games, the only issue currently is the audio, which is a bit distorted but mostly tolerable. (EDIT: fixed! Thanks to @hiulit for pointing out the issue!)

I was also able to compile a release build using LTO even if the machine has only 1GB of ram. It should be possible to do the same thing on a Raspberry allocating a large enough swap area/file.

Here I described the complete build process: https://github.com/samdze/godot-gameshell

Godot running on a device like that makes me very happy and I'd love to get official support for at least a few very popular arm devices. (Raspberry Pi, for sure)

hiulit commented 3 years ago

Great work @samdze !

For the audio issues, this might help you https://github.com/godotengine/godot/pull/43928

nicolas-van commented 3 years ago

Just to say I would really be happy to have Godot working on a Raspberry pi too. Continue like that guys 😎

Calinou commented 3 years ago

@nicolas-van Please don't bump issues without contributing significant new information. Use the :+1: reaction button on the first post instead.

hiulit commented 3 years ago

If anyone is interested, I've added headless and server builds https://github.com/hiulit/Unofficial-Godot-Engine-Raspberry-Pi.

I couldn't compile them for 2.1.6 for whatever reason.

I don't have any experience with that kind of builds, so if would be great if someone could test that they work :)

JoaoSilverio255 commented 3 years ago

Hey, I don't know if the discussion is still active or not but I have tried to make it work and for some reason, the change_scene() method doesn't seem to work. Have anyone came across this error?

Calinou commented 3 years ago

Hey, I don't know if the discussion is still active or not but I have tried to make it work and for some reason, the change_scene() method doesn't seem to work. Have anyone came across this error?

Please report issues related to unofficial builds to their author after making sure you can't reproduce them on official builds.

AgentSmith0 commented 2 years ago

Is it possible to export a project for PinePhone devices, that run on Linux (ARM)? https://wiki.pine64.org/wiki/PinePhone

cumminsd commented 2 years ago

Like the "new media art" posts above, another non-game use case is for the small, touch screen, home automation devices. Using Godot to program the displays would be awesome. :-)

xarses commented 2 years ago

totally off topic at this point, but I'm working on a control app that was initially intended for use by streamers that I'm planning on using for that too, and someone asked for the support for ARM for similar reasons as yourself here (which is why I'm following the arm support thread) Ping me on discord if you want to chat about it

efornara commented 2 years ago

I think I mentioned cross-compiling in this thread.

In the latest versions of FRT, I did finally switch to cross compiling my binary releases (at least for arm32v7 and arm64v8, the architectures everyone is interested in anyway).

Quick start is here: https://github.com/efornara/frt/blob/2.0.0/doc/Compile.md. My dockerfiles (image based on debian/buster) are here: https://github.com/efornara/crossbuild and an example of detect.py is here: https://github.com/efornara/frt/blob/1.1.0/detect.py. Tested with both gcc and clang. I don't normally use LTO, but, when I tested it, it only worked on one. IIRC, it was clang.

Maybe there are problems I am not seeing yet, but so far it seems to be working. To be honest, it was easier than I expected. I just select the proper triple for the compiler and use the triple- version of pkg-config.

silverkorn commented 2 years ago

About the recent proposal renaming, wasn't the request not explicitly only about the export template but also the editor? If not, maybe create another proposal for the editor?

Calinou commented 2 years ago

About the recent proposal renaming, wasn't the request not explicitly only about the export template but also the editor? If not, maybe create another proposal for the editor?

This proposal also covers the editor :slightly_smiling_face:

hiulit commented 2 years ago

Hey! For anyone interested, I've added 64 bits binaries for the Pi 4 https://github.com/hiulit/Unofficial-Godot-Engine-Raspberry-Pi

hiulit commented 2 years ago

Also, I've created a PPA repository so you can easily install Godot using apt install godot https://github.com/hiulit/ppa

aaronfranke commented 2 years ago

Related to official ARM builds, I've been working on improving support for non-x86 architectures in Godot. Currently Godot's build system, export system, GDNative, and more have poor support.

The build system has a mix of bits and arch and android_arch, but it really should be unified into one architecture argument. I have a PR for this here https://github.com/godotengine/godot/pull/55778 and I also have another PR to do this in GDNative C++ in 3.x here https://github.com/godotengine/godot-cpp/pull/714 and in the GDNative C demos here https://github.com/godotengine/gdnative-demos/pull/63

The export system in 3.x was missing a dedicated class for LinuxBSD and instead used the "PC" class for it. In Godot 4.0 this has been fixed by @bruvzg, now there is an EditorExportPlatformLinuxBSD class https://github.com/godotengine/godot/pull/58455 . Because of this, we were able to update the logic for the export template file names to be less hard-coded for x86 https://github.com/godotengine/godot/pull/59416 . The export system also has a binary_format/64_bits flag instead of architecture selection and you need to specify custom templates to do an ARM export, this needs to be replaced too.

When loading a PCK file, currently it looks for the executable name, and then the executable name with the extension stripped. This system works for one executable, and works for multiple executables on Linux (like mygame.x86_64 and mygame.arm64 loads mygame.pck) but we currently can't have multiple executables load the same PCK on Windows. I have a PR to improve the PCK loading logic so that we can have mygame.x86_64.exe and mygame.arm64.exe etc load mygame.pck https://github.com/godotengine/godot/pull/59527

Calinou commented 2 years ago

We discussed this in a proposal review meeting. We largely agreed to the feature and it has significant community support. Some more comments:

[^1]: There's a native Android editor port already, but this can be useful for testing headless functionality on an Android device.

Faless commented 2 years ago

For those interested, I managed to get 3.x build cross compiled from debian for raspbian. This process is much faster then the usual compilation via qemu or on device (you can get an editor build in 10/15 minutes compiling with -j5). I managed to produce both release builds and editor builds by only disabling few modules (haven't tested LTO yet), but the result is actually pretty great and does not require code changes to Godot (though, fixing the disabled modules likely will).

Here is the dockerfile (you can achieve the same on a true debian bullseye running the various apt commands ofc):

FROM debian:bullseye-slim

RUN set -eux; \
        dpkg --add-architecture armhf && \
        apt-get update && \
        apt-get install -y g++-arm-linux-gnueabihf scons pkg-config build-essential git && \
        apt-get install -y libstdc++-10-dev:armhf libx11-dev:armhf libxcursor-dev:armhf libxinerama-dev:armhf libgl1-mesa-dev:armhf libglu-dev:armhf libasound2-dev:armhf libpulse-dev:armhf libudev-dev:armhf libxi-dev:armhf libxrandr-dev:armhf && \
        rm -rf /var/lib/apt/lists/*

CMD /bin/bash

From the image, you can then build Godot.

# Build the image
docker build -t pibuilder:latest -f Dockerfile.pi .
# Run a container with the built image
docker run -it --rm pibuilder:latest /bin/bash
# Checkout godot (only 3.x branch tested)
git clone https://github.com/godotengine/godot.git --depth 1 -b 3.x
cd godot
# Cross compile the editor
PKG_CONFIG_PATH=/usr/lib/arm-linux-gnueabihf/pkgconfig/ scons CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ LINK=arm-linux-gnueabihf-g++ AR=arm-linux-gnueabihf-ar platform=x11 target=release_debug arch=armhf module_denoise_enabled=no module_theora_enabled=no module_webm_enabled=no CCFLAGS=-Wno-attributes use_static_cpp=no module_raycast_enabled=no

The produced binary was tested working on both the PI3 and the PI4.

As you see, this did not require changes to godot itself, since our build system is pretty flexible and allows heavy customization via flags. I suspect we could achieve a similar result on the base fedora image with use for most builds (probably not the ubuntu one which has a toolchain from 5+ years ago). The choice of debian was mostly for two reason:

I hope this helps others that are trying to build for PI while we figure out how to properly integrate it in our release cycle. I could go ahead and open a PR on the build-containers/build-scripts repo but we probably want to iterate on this, so I'll drop a ping to @akien-mga and @hpvb for some feedback.

EDIT: Note that I've left out potential optimizations (-mfpu=neon-fp-armv8 -mcpu=... to retain maximum compatibility across SoCs instead of targeting Raspberry PI specifically).

hiulit commented 2 years ago

Thank you very much @Faless!! I've successfully compiled a 32 bits binary of the editor. I'll try it on my Pi and I'll report back. Could this image be used to compile 64 bits binaries? If so, how?

P.S. You are missing a cd godot after cloning Godot ;)

hiulit commented 2 years ago

Ok, so I've tested it on my Pi 4 and it didn't work, but because I'm still using Buster and the image uses Bullseye :)

FYI: I tried to create a new image with and old distro to have better compatibility and the errors started to pop up: