search

google / filament

Filament is a real-time physically based rendering engine for Android, iOS, Windows, Linux, macOS, and WebGL2
https://google.github.io/filament/
Apache License 2.0
17.72k stars 1.88k forks source link

Build vs Target build passes #4874

Open jwinarske opened 2 years ago

jwinarske commented 2 years ago

I'm working on a Yocto (Linux) recipe for Filament. This enables any Linux platform (NXP, RaspberryPi, Qualcomm, Renesas, etc) to easily use filament.

The recipe is currently part of my Yocto meta-vulkan layer: https://github.com/jwinarske/meta-vulkan/blob/jw/filament/recipes-graphics/filament/filament-vk_git.bb

In Yocto / OpenEmbedded (OE) there are in general two build passes: Native and Target

Native is used to generate host side tools that are used during the Target build pass. So no intent for display testing in a Native build pass. The expectation is that tests run on the target architecture (cross compiled).

Building sdl2. sdl2 for Wayland expects some libraries that are not present in the Native build environment, nor is it practical they get added.

Enter CMake variable FILAMENT_SKIP_SAMPLES. Using FILAMENT_SKIP_SAMPLES and adding minor CMake changes I can build Filament without sdl2 and things dependent on sdl2 (FilamentApp, tests, etc). Keeping in mind Native build is for creating the tools for Target build pass. Branch that builds for Native is here: https://github.com/meta-flutter/filament/tree/yocto With this branch and mentioned recipe I get a Native package generated that has the required tools for the Target build pass.

Tests should be independent of samples. So it seems there could be an additional flag FILAMENT_SKIP_TESTS. Then if either FILAMENT_SKIP_SAMPLES or FILAMENT_SKIP_TESTS is set sdl2 is built. If both are set, then sdl2 would be skipped, call it a minimal host build, with tools only. This logic would also gate what gets installed in this state. Effectively only host side tools.

If you're okay with me adding FILAMENT_SKIP_TESTS and associated logic to support the Native/Target model for Yocto I will create a PR.

Beyond this I would also add minimal CMake logic to support a Mobile build pass for Linux.

romainguy commented 2 years ago

You could build only what you need. For instance our Android builds only compile the following targets on the host: matc resgen cmgen filamesh. This means samples and visuals tests are not compiled.

jwinarske commented 2 years ago

I understand your point of building target of interest opposed to all. For this scenario it would save build time, and limit installed files. This is good.

The only caveat is if you set FILAMENT_SUPPORT_WAYLAND (PR waiting on your review) sdl2 will fail if required packages are not present. sdl2 will fail without these, so there needs to be a way to gate this in the first place. So by skipping the build of sdl2 it would enable individual target builds as you suggest.

romainguy commented 2 years ago

sdl2 shouldn't be built if all you're building are host tools. Do you need to build Filament itself in what you call the native build?

jwinarske commented 2 years ago

Filament only for Target build pass (cross compile). For sdl2 deps there are the samples, and a couple tests. I just don't think gating tests with FILAMENT_SKIP_SAMPLES is a good idea, hence addition of FILAMENT_SKIP_TESTS.

romainguy commented 2 years ago

Ok then just build the tools and sdl2 shouldn't be built in the native pass (since you won't build the samples nor Filament and its tests).

jwinarske commented 2 years ago

When CMake is run it will run sdl2 tnt cmake where it fails to find dependent packages. Regardless if sdl2 is actually built.

Two options:

  1. Remove required statement in sdl2 tnt cmake package check. Build failure will be far less obvious as to why.
  2. Gate external to prevent sdl2 tnt cmake from running.

After that the remainder is cmake logic to enable mobile build pass for linux.

romainguy commented 2 years ago

Ah, see that's exactly why I was commenting in your PR that we don't like using external CMake files and use our own even for dependencies instead.

jwinarske commented 2 years ago

In the case of my current open PR, in order to build/link sdl2 it requires system specific packages. I would agree in case of non-system libraries.

romainguy commented 2 years ago

And that's fine but can't we have those dependencies be checked/required only when sdl2 is actually built?

jwinarske commented 2 years ago

It would involve custom steps with dependencies. In my opinion messy and not fun to maintain.

The change without proposed FILAMENT_SKIP_TESTS is in this commit: https://github.com/meta-flutter/filament/commit/2351e1a38f16ae76ccabc4f4454d89e788182203

romainguy commented 2 years ago

FILAMENT_SKIP_SAMPLES is definitely wrong here. It would be better to define a FILAMENT_SKIP_SDL2 at the root level based on FILAMENT_SKIP_SAMPLES and maybe FILAMENT_SKIP_TESTS but even that seems a bit off to me. I'm not against a FILAMENT_SKIP_TESTS but that seems like a poor workaround (and brittle since we could create a visual tool for instance that requires sdl2, and then that would have to be taken into account, etc.).

jwinarske commented 2 years ago

I'm good with renaming proposed FILAMENT_SKIP_TESTS to FILAMENT_SKIP_SDL2. The commit using FILAMENT_SKIP_TESTS was just to see what was required for the build to pass.

jwinarske commented 2 years ago

So I have a working Yocto recipe. It only targets Vulkan. https://github.com/jwinarske/meta-vulkan/pull/2

There are two recipes:

These artifacts can also be added to a SDK build, which is used for host cross compilation.

Implementing target demos would require quite a bit more work. I may create a separate repo for this later.

Filament changes are here: https://github.com/meta-flutter/filament/commit/a4b3bc992d93475f40b84ece5a6d33e51234f247

These are based on the Vulkan Wayland PR. Once that gets merged I will submit new PR for the changes to support Yocto.

romainguy commented 2 years ago

Those changes look reasonable.

jwinarske commented 2 years ago

I went ahead and created a separate filament-samples repo and associated Yocto recipes. With this I can build all the samples with Vulkan Wayland Backend cross-compiled for target (any machine type), and install into Wayland based OS image. Master can be diffed with https://github.com/jwinarske/filament-samples to see the scope. It is a first pass, it builds, and I suspect it could be improved. If there is an appetite to pull this functionality into master I'm up for it.

Yocto recipe for Wayland based image: https://github.com/jwinarske/meta-vulkan/blob/main/recipes-graphics/filament/filament-samples-vk_git.bb

filament-vk-native -> builds host tools filament-vk -> builds filament static libs/headers based on Vulkan Wayland and creates -dev package filament-samples-vk -> builds samples with Vulkan Wayland back end and installs into /usr/bin/filament-samples