Android Support

[cart]

It should be possible to run Bevy Apps on Android

[PrototypeNM1]

I'm working on this issue now, can I get it assigned to me to help head off accidental parallel implementations?

[cart]

absolutely! thanks for picking it up. this will be huge :heart:

[cart]

haha just noticed @karroffel already beat me to it

[endragor]

@PrototypeNM1 could you please share your progress and what's left to be done? I have some time on weekends that I could dedicate to adding Android support to Bevy. Last weekend I added Android support to cpal, which was one of the major blockers. There are still some issues on android-ndk-rs side. I could look into that, but it would be nice if you broke this issue into subitems and clarify what is done and what isn't.

[PrototypeNM1]

To my knowledge there are 4 main blockers for Android support.

1. Lack of Android support for cpal (wip) (thanks for tacking this!) Once fixed, may reveal issues in it's transitive dependents Rodio or winit.
2. cargo apk is broken for how Bevy structures Cargo.toml (PR) Alt. removing glob imports from Bevy's Cargo.toml fixes this.
3. Mobile needs input support, touch and IMU
4. bevy-glsl-to-spirv requires a rewrite or replacement.

I'm working on 4. glsl-to-spirv as written depends on a bundled glslangValidator executable. This can't work on Android without hackery. Luckily glslangValidator has a recently added a C API.

@endragor since you started investigating cpal/oboe issues, I recommend figuring out why cargo apk had issues with including multiple shared libraries, doing so is helpful both for the Rust Android ecosystem and Bevy.

[MichaelHills]

@PrototypeNM1 FYI over at https://github.com/bevyengine/bevy/issues/87 I put up https://github.com/bevyengine/bevy/pull/324 where currently uses shaderc-rs instead of glsl-to-spirv for iOS. Feel free to give it a try.

I too have now run into the missing touch support... I might hack something in just so I can play with it, not exactly sure what the correct API is.

[enfipy]

Hey all! I was able to run Bevy 3d example on my Android phone.

Proof and repo:

Current issues are next:

1. Shaders still not working in runtime (so I did it in compile-time). As they precompiled - we can't use macros so that shadows are weird on the screenshot.
2. Bgra was not worked with Vulkan on my phone so I replaced them with Rgba. Probably we can just replace them with Rgba everywhere in Bevy but this should be tested on other platforms.
3. I couldn't make assets work at all on Android. It's pretty weird but with the newest version of android-ndk-rs - there are no assets when I trying to get them from asset_manager.
4. Audio, 3d models, textures, etc were not tested due to problems with assets.

This is the current status. Any feedback and advice are highly appreciated.

[cart]

Fantastic! This is a big step :smile:

1. Shaders still not working in runtime (so I did it in compile-time). As they precompiled - we can't use macros so that shadows are weird on the screenshot.

Yeah as we discussed elsewhere I think we'll want to solve this by adding pre-compiled shader support with the various permutations available. We'll want a system like that anyway to improve startup times in release builds. Alternatively we can try adding dynamic compilation support by (1) waiting for Naga or (2) somehow making shaderc compile on android

2. Bgra was not worked with Vulkan on my phone so I replaced them with Rgba. Probably we can just replace them with Rgba everywhere in Bevy but this should be tested on other platforms.

Yeah thats a bit annoying, but it seems like we can work around it by selecting formats according to the platform. Alternatively, it might be worth checking with the wgpu folks to see if its something they can fix on their end.

3. I couldn't make assets work at all on Android. It's pretty weird but with the newest version of android-ndk-rs - there are no assets when I trying to get them from asset_manager.

Hmm maybe we need to add them to some sort of asset manifest? I haven't worked with android for awhile, but i vaguely remember something like that.

[PrototypeNM1]

@enfipy Awesome! Referencing your changes for TextureFormat I got runtime shader generation working on Android.

[enfipy]

After about 10 hours of unsuccessful tries to make @PrototypeNM1 example with bevy-glsl-to-spirv work on my Mac (and Windows) - I added shaderc-rs support for Android (but it's still shitty) and was able to make shaders work in runtime.

Status update:

• Shaders are working on runtime but the build still needs to be enhanced (or I very hope Naga will land soon). (Screenshot_1)
• I added a temporary solution for assets loading on Android. I plan to add a better one to Bevy as @cart will finish asset refactoring.
• PNG images rendered successfully (Screenshot_2).
• Text and UI rendered successfully (Screenshot_1).

Things need to be resolved:

• [x] Audio. This PR looks good and hopefully will enable high-performance audio on Android.
• [x] TextureFormat. It looks like some issues still persist with Bgra/Rgba as I couldn't build a Monkey example and faced a similar issue that was before this (but I don't think it's very serious).
• [ ] Shaderc. As I specified above - it's not finished and tested on other platforms yet and hard to build.
• [ ] Request Permissions. From Android version 23 and higher, Android requires applications to request permissions at runtime. I want to add jni call to this in a couple of days.
• [x] Touch Support. I hope this will be available soon with this #696.

This is all I remembered.

Screenshots:

Screenshot_1: Screenshot_2:

[cart]

Very nice progress! I just merged the asset system changes. I'm making a few more changes to AssetIo to re-add wasm compatibility. It should also make integrating the android AssetIo backend slightly easier (as I'm boxing AssetIo)

[trezm]

I'd like to just say, I'm not working on this issue, but am wildly excited to see its progress. Huge thanks to everyone working on it and know that the community is ecstatic to see this moving forward!!

[PrototypeNM1]

@enfipy @cart Given Enfipy has more time to work on and organize this issue, I think we should transfer this issue to them.

[cart]

I think this is a big enough issue / problem space that I don't think anyone needs to own it.

[PrototypeNM1]

Added support for runtime spirv generation using a rewrite of glsl-to-spirv.

We're still waiting for a fix in cargo-apk's dependencies to land. The alternative of removing glob imports for bevy's workspaces is still a viable alternative if we wanted this working asap.

[mmacedoeu]

Would like to bring attention to proper implementation of winit events Event::Suspended and Event::Resumed at https://github.com/bevyengine/bevy/blob/master/crates/bevy_winit/src/lib.rs#L170 so it can proper release and reload resources

[francesca64]

Would there be any interest in using cargo-mobile to help with building, generating Android Studio projects, running on device, etc?

[cart]

Medium-to-long term I might be interested in adopting higher level abstractions. Short term I'd rather keep it simple, encourage people to become familiar with the "native" mobile tooling, and maintain control over "official" templates. We will likely create an official bevy_template repo in the near future.

A minor note on the current cargo-mobile bevy template: bevy has its own answer to #[mobile_entry_point] called #[bevy_main]. I'd prefer it if you used that instead (to support future non-mobile scenarios). Although thats only available on the master branch right now.

[Dimous]

Hello! I tried to run example in both debug and release mode on a hardware device, but result is the same. It starts with a black screen and after a while confirmation dialog "Application Bevy Example does not respond" pops up.

Name	Version
OS	Win10 x64 Home Edition
NDK	r21d
cargo-apk	v0.5.6
rustc	1.49.0 (e1884a8e3 2020-12-29)
bevy	4a0837048cba3028ca3ec136f72fd9c1dcb97edd
device	Samsung SM-A515F, Android 10, API 29

2021-01-09 16:24:19.678 11085-11085/? E/Zygote: isWhitelistProcess - Process is Whitelisted
2021-01-09 16:24:19.679 11085-11085/? E/Zygote: accessInfo : 1
2021-01-09 16:24:19.686 11085-11085/? I/example.androi: Late-enabling -Xcheck:jni
2021-01-09 16:24:19.718 11085-11085/? E/example.androi: Unknown bits set in runtime_flags: 0x8000
2021-01-09 16:24:19.776 11085-11085/rust.example.android D/ActivityThread: setConscryptValidator
2021-01-09 16:24:19.777 11085-11085/rust.example.android D/ActivityThread: setConscryptValidator - put
2021-01-09 16:24:19.856 11085-11085/rust.example.android W/System: ClassLoader referenced unknown path: 
2021-01-09 16:24:19.969 11085-11085/rust.example.android D/PhoneWindow: forceLight changed to true [] from com.android.internal.policy.PhoneWindow.updateForceLightNavigationBar:4274 com.android.internal.policy.DecorView.updateColorViews:1547 com.android.internal.policy.PhoneWindow.dispatchWindowAttributesChanged:3252 android.view.Window.setFlags:1153 com.android.internal.policy.PhoneWindow.generateLayout:2474 
2021-01-09 16:24:19.970 11085-11085/rust.example.android I/MultiWindowDecorSupport: [INFO] isPopOver = false
2021-01-09 16:24:19.970 11085-11085/rust.example.android I/MultiWindowDecorSupport: updateCaptionType >> DecorView@88404b2[], isFloating: false, isApplication: true, hasWindowDecorCaption: false, hasWindowControllerCallback: true
2021-01-09 16:24:19.970 11085-11085/rust.example.android D/MultiWindowDecorSupport: setCaptionType = 0, DecorView = DecorView@88404b2[]
2021-01-09 16:24:20.188 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: setView = com.android.internal.policy.DecorView@88404b2 TM=true MM=false
2021-01-09 16:24:20.217 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: Relayout returned: old=(0,0,1080,2400) new=(0,0,1080,2400) req=(1080,2400)0 dur=10 res=0x7 s={true 531023818752} ch=true
2021-01-09 16:24:20.218 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: ViewRootImpl >> surfaceCreated
2021-01-09 16:24:20.221 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: ViewRootImpl >> surfaceChanged W=1080, H=2400)
2021-01-09 16:24:20.237 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: MSG_WINDOW_FOCUS_CHANGED 1 1
2021-01-09 16:24:20.237 11085-11085/rust.example.android D/InputMethodManager: prepareNavigationBarInfo() DecorView@88404b2[NativeActivity]
2021-01-09 16:24:20.237 11085-11085/rust.example.android D/InputMethodManager: getNavigationBarColor() -855310
2021-01-09 16:24:20.240 11085-11085/rust.example.android D/InputMethodManager: prepareNavigationBarInfo() DecorView@88404b2[NativeActivity]
2021-01-09 16:24:20.240 11085-11085/rust.example.android D/InputMethodManager: getNavigationBarColor() -855310
2021-01-09 16:24:20.240 11085-11085/rust.example.android V/InputMethodManager: Starting input: tba=rust.example.android ic=null mNaviBarColor -855310 mIsGetNaviBarColorSuccess true , NavVisible : true , NavTrans : false
2021-01-09 16:24:20.240 11085-11085/rust.example.android D/InputMethodManager: startInputInner - Id : 0
2021-01-09 16:24:20.240 11085-11085/rust.example.android I/InputMethodManager: startInputInner - mService.startInputOrWindowGainedFocus
2021-01-09 16:24:20.255 11085-11085/rust.example.android D/InputMethodManager: prepareNavigationBarInfo() DecorView@88404b2[NativeActivity]
2021-01-09 16:24:20.255 11085-11085/rust.example.android D/InputMethodManager: getNavigationBarColor() -855310
2021-01-09 16:24:20.255 11085-11085/rust.example.android V/InputMethodManager: Starting input: tba=rust.example.android ic=null mNaviBarColor -855310 mIsGetNaviBarColorSuccess true , NavVisible : true , NavTrans : false
2021-01-09 16:24:20.256 11085-11085/rust.example.android D/InputMethodManager: startInputInner - Id : 0
2021-01-09 16:24:20.256 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: MSG_RESIZED: frame=(0,0,1080,2400) ci=(0,93,0,126) vi=(0,93,0,126) or=1
2021-01-09 16:24:20.356 11085-13264/rust.example.android I/OboeAudio: openStream() OUTPUT -------- OboeVersion1.4.3 --------
2021-01-09 16:24:20.356 11085-13264/rust.example.android D/OboeAudio: AAudioLoader():  dlopen(libaaudio.so) returned 0x459a7d9f91c217e9
2021-01-09 16:24:20.356 11085-13264/rust.example.android I/AAudio: AAudioStreamBuilder_openStream() called ----------------------------------------
2021-01-09 16:24:20.357 11085-13264/rust.example.android I/AudioStreamBuilder: rate   =  44100, channels  = 2, format   = 5, sharing = SH, dir = OUTPUT
2021-01-09 16:24:20.357 11085-13264/rust.example.android I/AudioStreamBuilder: device =      2, sessionId = -1, perfMode = 10, callback: ON with frames = 0
2021-01-09 16:24:20.357 11085-13264/rust.example.android I/AudioStreamBuilder: usage  =      1, contentType = 2, inputPreset = 6, allowedCapturePolicy = 0
2021-01-09 16:24:20.357 11085-13264/rust.example.android D/AudioStreamBuilder: build() MMAP not available because AAUDIO_PERFORMANCE_MODE_LOW_LATENCY not used.
2021-01-09 16:24:20.369 11085-13264/rust.example.android D/AudioStreamTrack: open(), request notificationFrames = 0, frameCount = 0
2021-01-09 16:24:20.376 11085-13264/rust.example.android D/AudioTrack: setVolume(1.000000, 1.000000) pid : 11085
2021-01-09 16:24:20.377 11085-13264/rust.example.android I/AAudio: AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#1 ----------------
2021-01-09 16:24:20.378 11085-13264/rust.example.android D/OboeAudio: AudioStreamAAudio.open() format=2, sampleRate=44100, capacity = 3544
2021-01-09 16:24:20.378 11085-13264/rust.example.android D/OboeAudio: AudioStreamAAudio.open: AAudioStream_Open() returned AAUDIO_OK
2021-01-09 16:24:20.378 11085-13264/rust.example.android D/AAudio: AAudioStream_requestStart(s#1) called --------------
2021-01-09 16:24:20.380 11085-13264/rust.example.android D/AAudio: AAudioStream_requestStart(s#1) returned 0 ---------
2021-01-09 16:24:20.381 11085-13185/rust.example.android D/AudioStreamLegacy: onAudioDeviceUpdate() devId 2 => 2
2021-01-09 16:24:20.389 11085-13264/rust.example.android E/event crates\bevy_gilrs\src\lib.rs:25:  Failed to start Gilrs. Gilrs does not support current platform.
2021-01-09 16:24:20.402 11085-13264/rust.example.android D/vulkan: searching for layers in '/data/app/rust.example.android-eS2NTJt_wA_NBHKUqWdyUA==/lib/arm64'
2021-01-09 16:24:20.402 11085-13264/rust.example.android D/vulkan: searching for layers in '/data/app/rust.example.android-eS2NTJt_wA_NBHKUqWdyUA==/base.apk!/lib/arm64-v8a'
2021-01-09 16:24:21.086 11085-13264/rust.example.android I/ExynosConfigStore: vendor::samsung_slsi::hardware::configstore::V1_0::IExynosHWCConfigs::getGrallocVersion retrieved: 0 (default)
2021-01-09 16:24:21.091 11085-13264/rust.example.android W/Gralloc3: mapper 3.x is not supported
2021-01-09 16:24:21.095 11085-13264/rust.example.android I/gralloc: Arm Module v1.0
2021-01-09 16:24:21.112 11085-13264/rust.example.android W/vulkan: vkAcquireNextImageKHR: non-infinite timeouts not yet implemented
2021-01-09 16:24:21.116 11085-13263/rust.example.android I/RustStdoutStderr: thread '<unnamed>' panicked at 'attachment's sample count 2 is invalid', C:\Users\kasim\.cargo\registry\src\github.com-1ecc6299db9ec823\wgpu-0.6.2\src\backend\direct.rs:1355:35
2021-01-09 16:24:21.116 11085-13263/rust.example.android I/RustStdoutStderr: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2021-01-09 16:27:03.424 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: MSG_WINDOW_FOCUS_CHANGED 0 1
2021-01-09 16:27:03.424 11085-11085/rust.example.android D/InputMethodManager: prepareNavigationBarInfo() DecorView@88404b2[NativeActivity]
2021-01-09 16:27:03.424 11085-11085/rust.example.android D/InputMethodManager: getNavigationBarColor() -855310
2021-01-09 16:27:03.532 11085-11085/rust.example.android D/InputTransport: Input channel destroyed: 'ClientS', fd=73
2021-01-09 16:27:03.986 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: Relayout returned: old=(0,0,1080,2400) new=(0,0,1080,2400) req=(1080,2400)8 dur=8 res=0x5 s={false 0} ch=true
2021-01-09 16:27:03.986 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: ViewRootImpl >> surfaceDestroyed
2021-01-09 16:27:04.015 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: stopped(true) old=false
2021-01-09 16:27:05.270 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: Relayout returned: old=(0,0,1080,2400) new=(0,0,1080,2400) req=(1080,2400)4 dur=8 res=0x1 s={false 0} ch=false
2021-01-09 16:27:05.271 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: stopped(false) old=true
2021-01-09 16:27:05.281 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: stopped(false) old=false
2021-01-09 16:27:05.293 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: Relayout returned: old=(0,0,1080,2400) new=(0,0,1080,2400) req=(1080,2400)0 dur=8 res=0x7 s={true 533386002432} ch=true
2021-01-09 16:27:05.294 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: ViewRootImpl >> surfaceCreated
2021-01-09 16:27:05.294 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: ViewRootImpl >> surfaceChanged W=1080, H=2400)
2021-01-09 16:27:05.322 11085-11085/rust.example.android I/ViewRootImpl@c160b0[NativeActivity]: MSG_WINDOW_FOCUS_CHANGED 1 1
2021-01-09 16:27:05.322 11085-11085/rust.example.android D/InputMethodManager: prepareNavigationBarInfo() DecorView@88404b2[NativeActivity]
2021-01-09 16:27:05.323 11085-11085/rust.example.android D/InputMethodManager: getNavigationBarColor() -855310
2021-01-09 16:27:05.324 11085-11085/rust.example.android D/InputMethodManager: prepareNavigationBarInfo() DecorView@88404b2[NativeActivity]
2021-01-09 16:27:05.324 11085-11085/rust.example.android D/InputMethodManager: getNavigationBarColor() -855310
2021-01-09 16:27:05.324 11085-11085/rust.example.android V/InputMethodManager: Starting input: tba=rust.example.android ic=null mNaviBarColor -855310 mIsGetNaviBarColorSuccess true , NavVisible : true , NavTrans : false
2021-01-09 16:27:05.324 11085-11085/rust.example.android D/InputMethodManager: startInputInner - Id : 0
2021-01-09 16:27:05.324 11085-11085/rust.example.android I/InputMethodManager: startInputInner - mService.startInputOrWindowGainedFocus
2021-01-09 16:27:17.567 11085-13155/rust.example.android I/example.androi: Thread[3,tid=13155,WaitingInMainSignalCatcherLoop,Thread*=0x7c30405c00,peer=0x14280000,"Signal Catcher"]: reacting to signal 3
2021-01-09 16:27:17.588 11085-13330/rust.example.android W/example.androi: 0xebadde09 skipped times: 0
2021-01-09 16:27:17.589 11085-13330/rust.example.android A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x6d65747379733f in tid 13330 (AudioTrack), pid 11085 (example.android)

[blaind]

I've got the same error as @Dimous

Tried enabling backtrace (by setting std::env::set_var("RUST_BACKTRACE", "full"); to main() beginning), but probably because of https://github.com/rust-windowing/android-ndk-rs/issues/101 the backtrace is not too helpful:

01-23 12:23:43.663 23103 23120 I RustStdoutStderr: thread '<unnamed>' panicked at 'attachment's sample count 2 is invalid', /.../.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.6.2/src/backend/direct.rs:1355:35
01-23 12:23:43.663 23103 23120 I RustStdoutStderr: stack backtrace:
01-23 12:23:43.663 23103 23120 I RustStdoutStderr:    0:       0x74b01e5384 - <unknown>
01-23 12:23:43.663 23103 23120 I RustStdoutStderr:    1:       0x74b01ff720 - <unknown>

Edit 1: turns out the reason is msaa sampling, after commenting it out (//.add_resource(Msaa { samples: 2 })), reinstallation and force-stopping the app, the example works.

Edit 2: the example runs, but between switching windows etc. got a few stacktraces multiple times:

01-23 12:42:25.156 24762 24779 I RustStdoutStderr: thread '<unnamed>' panicked at 'Unable to query surface capabilities: ERROR_SURFACE_LOST_KHR', /../.cargo/registry/src/github.com-1ecc6299db9ec823/gfx-backend-vulkan-0.6.5/src/window.rs:343:10

01-23 12:39:13.897 23559 23857 I RustStdoutStderr: thread '<unnamed>' panicked at 'Could not set global default tracing subscriber. If you've already set up a tracing subscriber, please disable LogPlugin from Bevy's DefaultPlugins: SetGlobalDefaultError { _no_construct: () }', crates/bevy_log/src/lib.rs:101:18

01-23 12:38:58.774 23559 23591 I RustStdoutStderr: thread '<unnamed>' panicked at 'Cannot get the native window, it's null and will always be null before Event::Resumed and after Event::Suspended. Make sure you only call this function between those events.', /../.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.24.0/src/platform_impl/android/mod.rs:530:13

Edit 3: after investigating the ERROR_SURFACE_LOST_KHR error, found https://github.com/gfx-rs/gfx/issues/3420 as a possible solution. Another alternative that'd require more work, is to hook bevy into android activity lifecycle. Possible way to implement this is to use ndk_glue::poll_events() -> Event which may return the lifecycle events as per documentation. I could try to implement this, but would require more pointers about bevy internals - possibly hooking up to draw_state.can_draw or to some code path at bevy_wgpu?

[blaind]

I've been investigating the ERROR_SURFACE_LOST_KHR further. It seems that the ndk_glue::poll_events() part mentioned above is already handled by bevy_winit crate which relies on winit crate. Relevant code path: bevy_winit/src/lib.rs::winit_runner_with

Problem number 1

Bevy runs render loop regardless of whether the Activity has gone away (not running), and there's currently no way to communicate that flag from winit.

Android loop part of winit can be found from https://github.com/rust-windowing/winit/blob/6db308f1e9388986f8c0332d37adf95d051712a3/src/platform_impl/android/mod.rs#L89

Winit has internally (see above code) self.running boolean flag, which is handled through Event::{Pause, Resume} events. Winit sends a MainEventsCleared event for each loop iteration, and seems to rely on that for running the render loop. However, MainEventsCleared is sent regarless of the self.running state. Other events WindowEvent and RedrawRequested are dependent on additional flags, and are not sent on each frame.

When user closes the activity, bevy still tries to render and run systems. Turned out, some system gets in (mutex?) lock state at https://github.com/bevyengine/bevy/blob/7166a28bafa669eac40f5864cce8a150b330dd5f/crates/bevy_wgpu/src/renderer/wgpu_render_graph_executor.rs#L73 hence blocking the execution (this was quite hard to find/debug...).

Problem number 2

When resuming the Activity, the surface error persists (panicked at 'Unable to query surface capabilities: ERROR_SURFACE_LOST_KHR', /.../gfx-backend-vulkan-0.6.5/src/window.rs:343:10)

I guess the best way would be to reinitialize the renderer, or some other relevant part of the pipeline (did not investigate further this part, would need to know about the renderer pipeline functionality on other platforms). Maybe through an event after Event::Resumed. See TODO part at the above collapsed code snippet.

PoC solution

See the following diffs:

• Bevy: https://github.com/bevyengine/bevy/compare/master...blaind:master
• Winit: https://github.com/rust-windowing/winit/compare/master...blaind:master

==> now in pull request #1293

Some tips for faster development iteration speed on Android:

• Change bevy Cargo.toml build_targets = ["aarch64-linux-android", "armv7-linux-androideabi"] to include only the desired target, not two
• If using rust-analyzer, add to workspace settings: "rust-analyzer.cargo.target": "aarch64-linux-android" (or arm7...) so that x86_64 build won't conflict with the targets
• Remove the largest directories (models, sounds at least) in bevy assets folder (temporarily) - this will reduce the APK size from about 80MB to about 60MB
• For future, probably gdb would be good, log printing is annoying to debug

[blaind]

Linking to #2432 (lifecycle API), which could help driving Android support forward too. Same challenges also in IOS (see #2296)

[alice-i-cecile]

Adding to the 0.8 milestone; I'd like to get Android support unbroken by then.

[rib]

Heya, not sure the best place to suggest this, but with Bevy's Android support still being somewhat in flux atm, it might be worth considering how to avoid being tied to NativeActivity and allowing for situations where developers need to use a custom Activity subclass.

NativeActivity only provides a quite limited shim over the Activity class and although it's possible to subclass NativeActivity in Java to potentially make up for some of its limitations, one thing that seems very common on Android is to derive from AppCompatActivity which opens the door to a load of androidx compatibility code, including using jetpack packages.

One specific base Activity that could make sense in the context of Bevy is to investigate the Android Game Development Kit which includes a GameActivity that could be considered a more modern replacement for NativeActivity.

See: https://developer.android.com/games/agdk https://developer.android.com/games/agdk/integrate-game-activity

In addition to the GameActivity they also have a library for frame pacing, called Swappy that may be worth considering, for the sake of not having to re-implement fiddly/error-prone logic for synchronizing/throttling frame submissions for GL and Vulkan. (See: https://developer.android.com/games/sdk/frame-pacing). They also have a library for text input (https://developer.android.com/games/agdk/add-support-for-text-input) and a library for supporting game controllers (https://developer.android.com/games/sdk/game-controller) that could be useful.

Considering my own use case currently, working on a Bluetooth library that runs on Android which I'd also like to be able to use with Bevy then it's notable that I have to subclass Activity in order to implement/override onActivityResult so I can use Activity::startIntentSenderForResult (The way Android's Bluetooth Companion API works is via an Intent that sends back a result when the user chooses a device - and that requires a subclass of Activity to implement onActivityResult). I'm hoping that however Android is supported that it will be fairly straightforward to use a custom Activity subclass. Besides very simple demos then I think most real-world Android applications are quite likely to end up wanting/needing this.

An alternative to GameActivity could be to create some kind of RustActivity or BevyActivity that would hopefully subclass AppCompatActivity, along with glue/bindings that re-implement the features of GameActivity but my initial impression is that it could be good to try and leverage what's already been implemented + tested as part of AGDK instead of re-implementing the same functionality.

[rib]

As a follow up to above, I've been working on a Rust "glue" layer (similar to ndk-glue) that can support building applications based on GameActivity instead of NativeActivity:

https://github.com/rib/agdk-rust/tree/main/game-activity (the README there also has a bit more context/info)

I've also built a corresponding branch of winit that uses this glue in the Android backend: https://github.com/rib/winit/tree/agdk-game-activity

For initial testing I've made:

1. A minimal application that just verifies running a mainloop only based on the glue API itself: https://github.com/rib/agdk-rust/tree/main/examples/agdk-mainloop
2. A minimal winit-based application that demonstrates drawing a triangle with wgpu; demonstrating handling lifecycle events and updating surface state each time we get a new native window: https://github.com/rib/agdk-rust/tree/main/examples/agdk-winit-wgpu

I'm pretty happy with how it's taken shape so far.

It's maybe worth mentioning that in the process of adapting existing ndk-glue based code I think I also stumbled across a number of issues that I'm hoping to follow up on separately - such as some concerns about how synchronization is currently handled in ndk-glue. Regardless of whether GameActivity or NativeActivity are used as a base Activity for Rust Android apps I think the winit backend implementation also needs some attention in this context.

Locally I've created a minimal Bevy Android app that I've started to poke at, and I see that in general the big difficulty with running Bevy apps on Android atm comes down to how lifecycle events are handled (which I've seen that @blaind has been investigating). In particular the first panic I hit was due to how Bevy tends to expect it can create a "primary" window while initializing - without waiting for any kind of lifecycle event - and it's not generally prepared to handle that window being destroyed and re-created repeated over the life of the application.

I circled back to the above agdk-winit-wgpu test application after I started to understand the main issue being hit in Bevy so I could first get a clearer idea of how to handle those lifecycle events in a way that can support desktop and mobile applications.

Hopefully I can make a bit more progress with getting the Bevy app working now that I feel like I have a clearer understanding of the problem.

It would be great to get any initial thoughts / feedback on any of the above work-in-progress.

[alice-i-cecile]

Removing from the 0.8 milestone; even if we merge #4913 I don't think this should be closed for 0.8 until we have significantly more testing.

[Leinnan]

Is there any update on that? What is the status of Android in 0.9?

[rib]

Although I haven't looked at Bevy recently we did finally get the android-activity backend merged for Winit which should help with Android support but it will also require some integration work in Bevy when it updates to Winit 0.28.

One of the other notable thing upstreamed in Winit was that all backends now consistently deliver a Resume event which helps with writing portable code that runs on Android (where handling Resume is very important) and other window systems.

I'm not sure atm how tricky it's going to be to rebase #4913 (or maybe something equivalent was done in the mean time) but something equivalent to that will be one of the main blockers still I expect.

[mockersf]

Is there any update on that? What is the status of Android in 0.9?

Works on some device but without audio, doesn't work on others at all.

android-activity backend merged for Winit

Do you know if there has been work to replace it also in things like cpal?

[rib]

cpal has switched over to using ndk-context now so that it's no longer dependent on any glue crate so should hopefully just work now 🤞

E.g. for android-activity I created an example to test cpal here: https://github.com/rib/android-activity/tree/main/examples/agdk-cpal which works with cpal = "0.14"

[rib]

here's the cpal PR that switched to ndk-context: https://github.com/RustAudio/cpal/pull/641 for reference

[enfipy]

Also, our team was working on the toolkit for plugins and build cli for Android (iOS in the future): https://github.com/dodorare/crossbow.

We succeeded in running different Google Play plugins on Android but Bevy was also blocked by stuff like audio, etc. @rib it would be awesome to take a look at how we can integrate with AGDK, we tried to design crossbow similar to Xamarin and Godot - so I think there will be no big problems with it.

Here's an example with Macroquad (but will work with Bevy too):

Honestly, I think if we all here put all this stuff together, we can make pretty fine Android support in Bevy v0.10, IMO.

[rib]

cool, I'd heard of crosssbow recently but hadn't had a chance to take a look so thanks for the pointer @enfipy

Skimming that example I see it has a main entry point like

#[macroquad::main("Macroquad UI")]
async fn main() -> anyhow::Result<()> {
}

which makes me think that you're possibly defining your own glue layer between native and jvm code? Or does this maybe currently build on the ndk-glue macro perhaps?

Winit (which Bevy usually builds on) is currently (for 0.28) implies needing to use android-activity as a glue crate (previously ndk-glue) for handling integration with the Activity class (either NativeActivity or GameActivity from AGDK) and both of those Activity crates also generally define the architecture for spawning a native thread that will run your application's main() function.

I'd need to poke into crossbow more to understand how it defines its main entry point macro to see if we can figure out how to join the dots with android-activity.

You should have some flexibility from the proc macro I guess but one notable thing about the entry point ABI for android-activity is that it is passed an app: AndroidApp argument that represents the app and represents a kind of connection between a native thread and JVM Activity (to avoid global static state) and I wonder how that would fit with crossbow's current main function definition.

[rib]

okey, I realize now that the example main() entry point is specific to the macroquad library - not something imposed by crossbow

[rib]

btw another build tool that's possibly in a similar space to crosstool is xbuild. I guess you've maybe seen it already @enfipy but figured I'd mentioned it while I didn't see it in the list to existing tools mentioned in the readme for crosstool.

[slyedoc]

Got bevy_audio working if I tell cpal to enable oboe "shared-stdcxx" feature, meaning I am having an issue with "c++_static" linking. Still working on why, and not sure if its just me or not.

[rib]

In case it might give some insight/clue you could maybe poke at this minimal cpal example: https://github.com/rib/android-activity/tree/main/examples/agdk-cpal

That example doesn't currently do anything special to choose the stdc++_shared vs static implementation and seemed to work.

There's currently no standard convention amongst Rust Android crates for being able to configure whether to use the shared or static implementation which is a bit awkward, since it wouldn't really be desirable to have a mix and match amongst crates.

It sounds kinda surprising that it works when enabling shared stdcxx. I would have thought you'd also have to make sure to copy the libc++_shared.so out of your NDK/toolchain into your package for that to run - but maybe cargo apk automagically does that.

[rib]

oh sorry, I lost track of which bug I was responding too and see I already linked that example quite recently in this thread so guess you've probably already seen it.

[slyedoc]

but maybe cargo apk automagically does that.

It does, and your examples have been my starting point for everything in this PR. Used na-winit-wgpu quite a bit, and made sure I could get agdk-cpal working before anything else this morning.

I do want to add AGDK bevy example (you put in so much work getting that working), plan to after I am happy with native activity with apk.

[MarijnS95]

It sounds kinda surprising that it works when enabling shared stdcxx. I would have thought you'd also have to make sure to copy the libc++_shared.so out of your NDK/toolchain into your package for that to run - but maybe cargo apk automagically does that.

cargo-apk takes care of all your library linking problems :wink:, and we landed the same feature in xbuild just a few days ago :slightly_smiling_face:

[rib]

@MarijnS95 does cargo apk maybe default to making crates link against libc++_shared.so as opposed to static linking?

As a guess, that seem like it could explain a discrepancy between the agdk-cpal example working and needing to enable shared-stdcxx with Bevy (while building with cargo apk)

[MarijnS95]

@rib cargo apk looks at what libraries the final .so requests linking against; if you configure the library to not link against dynamic libc++ (via i.e. the cc crate when compiling native code), it won't embed libc++_shared.so in the final APK.

[rib]

@rib cargo apk looks at what libraries the final .so requests linking against; if you configure the library to not link against dynamic libc++ (via i.e. the cc crate when compiling native code), it won't embed libc++_shared.so in the final APK.

right, makes sense

[alice-i-cecile]

Android is reasonably well supported now, following #9937. #10158 will fix an important bug with audio playing while suspended, but in general, it should be functional.

It needs more and better documentation still, and better multi-touch + haptic support, but the foundations are all there. If you run into further problems, please open a new issue!