iOS Support

[cart]

It should be possible to run Bevy Apps on iOS

[MichaelHills]

Hi, what’s involved in setting up Bevy on iOS? My understanding is that it uses winit which should already support iOS?

[MichaelHills]

I followed this guide to just get a Rust lib working from an ios project. Then I tried to substitute a bevy example app in. During cargo lipo --release it fails to build bevy-glsl-to-spirv.

It looks like there's no prebuilt binary in that repo so it tries to build glslang and then cmake fails

  -- Check for working C compiler: /usr/bin/cc -- broken
...
Building C object CMakeFiles/cmTC_97014.dir/testCCompiler.c.o
      /usr/bin/cc   -fPIC -m64 -m64 -mios-simulator-version-min=7.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.2.sdk -fembed-bitcode  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk   -o CMakeFiles/cmTC_97014.dir/testCCompiler.c.o   -c /Users/mikeh/rust-ios-example/rust/target/x86_64-apple-ios/release/build/bevy-glsl-to-spirv-51cc402e67486116/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
      clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]

Apparently that last line is a key sign something is off. I take it the build for glslang doesn't work properly to target aarch64-apple-ios or x86_64-apple-ios.

[MichaelHills]

I tried a couple of things, but I'm new to rust and new to cross-compiling to iOS, though slowly figuring things out. The approach of spawning the glslValidator process probably isn't going to fly on iOS. MoltenVK apparently can build the iOS static lib libglslang.a so I considered trying to interface with that. The bevy-glsl-to-spirv repo says TODO to switch to shaderc-rs so I figured I'd have a crack at that but had trouble cross-compiling for iOS.

I ended up stubbing out the code in crates/bevy_render/src/shader/shader.rs just to see how far I could get. I also had to disable the optional bevy audio as it didn't compile either (forgot why).

I managed to finally get something to build, on the simulator gfx-rs / gfx-backend-metal failed with Target OS is incompatible: library was not compiled for the simulator so I ran it on my phone instead to which I got

EventLoop cannot be run after a call to UIApplicationMain on iOS.

Stopping for tonight. Here's my diff so far to get where i got to...

diff --git a/Cargo.toml b/Cargo.toml
index 90b2368d..3254fd6c 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -13,7 +13,8 @@ readme = "README.md"
 exclude = ["assets/**/*", "tools/**/*", ".github/**/*", "crates/**/*"]

 [features]
-default = ["bevy_audio", "bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr", "mp3"]
+#default = ["bevy_audio", "bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr", "mp3"]
+default = ["bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr"]
 profiler = ["bevy_ecs/profiler", "bevy_diagnostic/profiler"]
 wgpu_trace = ["bevy_wgpu/trace"]

@@ -22,10 +23,10 @@ png = ["bevy_render/png"]
 hdr = ["bevy_render/hdr"]

 # Audio format support (MP3 is enabled by default)
-mp3 = ["bevy_audio/mp3"]
-flac = ["bevy_audio/flac"]
-wav = ["bevy_audio/wav"]
-vorbis = ["bevy_audio/vorbis"]
+#mp3 = ["bevy_audio/mp3"]
+#flac = ["bevy_audio/flac"]
+#wav = ["bevy_audio/wav"]
+#vorbis = ["bevy_audio/vorbis"]

 serialize = ["bevy_input/serialize"]

@@ -56,7 +57,7 @@ bevy_ui = { path = "crates/bevy_ui", version = "0.1" }
 bevy_window = { path = "crates/bevy_window", version = "0.1" }

 # bevy (optional)
-bevy_audio = { path = "crates/bevy_audio", optional = true, version = "0.1" }
+#bevy_audio = { path = "crates/bevy_audio", optional = true, version = "0.1" }
 bevy_gltf = { path = "crates/bevy_gltf", optional = true, version = "0.1" }
 bevy_wgpu = { path = "crates/bevy_wgpu", optional = true, version = "0.1" }
 bevy_winit = { path = "crates/bevy_winit", optional = true, version = "0.1" }
diff --git a/crates/bevy_render/Cargo.toml b/crates/bevy_render/Cargo.toml
index ea29b856..53fe2af4 100644
--- a/crates/bevy_render/Cargo.toml
+++ b/crates/bevy_render/Cargo.toml
@@ -24,7 +24,7 @@ bevy_window = { path = "../bevy_window", version = "0.1" }

 # rendering
 spirv-reflect = "0.2.3"
-bevy-glsl-to-spirv = "0.1.7"
+#bevy-glsl-to-spirv = { path = "/Users/mikeh/repos/glsl-to-spirv" } #"0.1.7"
 image = { version = "0.23", default-features = false }

 # misc
diff --git a/crates/bevy_render/src/shader/shader.rs b/crates/bevy_render/src/shader/shader.rs
index f963c0c9..dd448e74 100644
--- a/crates/bevy_render/src/shader/shader.rs
+++ b/crates/bevy_render/src/shader/shader.rs
@@ -1,6 +1,6 @@
 use super::ShaderLayout;
 use bevy_asset::Handle;
-use bevy_glsl_to_spirv::compile;
+//use bevy_glsl_to_spirv::compile;
 use std::{io::Read, marker::Copy};

 /// The stage of a shader
@@ -11,25 +11,26 @@ pub enum ShaderStage {
     Compute,
 }

-impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
-    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
-        match self {
-            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
-            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
-            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
-        }
-    }
-}
+//impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
+//    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
+//        match self {
+//            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
+//            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
+//            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
+//        }
+//    }
+//}

 fn glsl_to_spirv(
     glsl_source: &str,
     stage: ShaderStage,
     shader_defs: Option<&[String]>,
 ) -> Vec<u32> {
-    let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
+    //let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
     let mut spv_bytes = Vec::new();
-    output.read_to_end(&mut spv_bytes).unwrap();
-    bytes_to_words(&spv_bytes)
+    //output.read_to_end(&mut spv_bytes).unwrap();
+    //bytes_to_words(&spv_bytes)
+    spv_bytes
 }

 fn bytes_to_words(bytes: &[u8]) -> Vec<u32> {

[MichaelHills]

Looks like cross-compiling iOS stuff with cmake-rs is currently broken https://github.com/alexcrichton/cmake-rs/issues/96 so that's probably the source of some of my woes

[MichaelHills]

Success! I managed to get the 3d_scene demo loading in portrait mode (albeit a bit warped) on my phone.

• I pinned cmake-rs to 0.1.43 because that's the version which had the working iOS support
• Dropped bevy-glsl-to-spirv in favour of using shaderc-rs directly from crates/bevy_render/src/shader/shader.rs
• Created a main.swift to call out to my bevy_main() function directly which was the 3d_sprite main function renamed like this

  #[no_mangle]
  pub extern fn bevy_main() {

In xcode I had to add the static lib / header and search paths as per this site and I also had to link against lc++. Basically atm I build my "game" as a static lib and then call out to it manually from in an ios xcode project.

Trying to figure out how to get it into landscape mode now.

[cart]

Whoa that's huge! Thanks for the hard work here. People are going to love this!

[MichaelHills]

No worries. :) When I saw the post on Bevy on HN I thought it looked pretty awesome. I previously had started making a PC game in UE4 but life got busy. Decided I wanted to prototype it out as a mobile game instead and learn some Rust/ECS while I'm at it.

How should I go about getting stuff landed? I think the switch to shaderc-rs in general can be done now (tested a couple of examples still work). Did you want that to be done inside of bevy-glsl-to-spirv? I've currently just implemented it directly in crates/bevy_render/src/shader/shader.rs while hacking this all up.

In order to get iOS integration to "just work" the following issues need to be solved properly:

• Land using shaderc-rs in bevy
• Get that iOS support commit in cmake-rs fixed up and re-landed there
• In shaderc-rs I had to edit CMakeLists.txt in the glsl and shaderc submodules (see below), so maybe... get those landed there?
• Figure out why coreaudio-sys v0.2.5 panics during compile (that was why I had to disable audio)

shaderc

diff --git a/glslc/CMakeLists.txt b/glslc/CMakeLists.txt
index acf6fb0..4ec0067 100644
--- a/glslc/CMakeLists.txt
+++ b/glslc/CMakeLists.txt
@@ -40,6 +40,7 @@ shaderc_add_asciidoc(glslc_doc_README README)

 if(SHADERC_ENABLE_INSTALL)
   install(TARGETS glslc_exe
+    BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
     RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
 endif(SHADERC_ENABLE_INSTALL)

glslang

diff --git a/StandAlone/CMakeLists.txt b/StandAlone/CMakeLists.txt
index 2cf2899c..f84fdd2b 100644
--- a/StandAlone/CMakeLists.txt
+++ b/StandAlone/CMakeLists.txt
@@ -50,11 +50,13 @@ endif(WIN32)

 if(ENABLE_GLSLANG_INSTALL)
     install(TARGETS glslangValidator EXPORT glslangValidatorTargets
+            BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
             RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
     install(EXPORT glslangValidatorTargets DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake)

     if(ENABLE_SPVREMAPPER)
         install(TARGETS spirv-remap EXPORT spirv-remapTargets
+            BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
             RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
         install(EXPORT spirv-remapTargets DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake)
     endif()

Also here's the change to shader.rs

diff --git a/crates/bevy_render/src/shader/shader.rs b/crates/bevy_render/src/shader/shader.rs
index f963c0c9..2e710cd0 100644
--- a/crates/bevy_render/src/shader/shader.rs
+++ b/crates/bevy_render/src/shader/shader.rs
@@ -1,7 +1,6 @@
 use super::ShaderLayout;
 use bevy_asset::Handle;
-use bevy_glsl_to_spirv::compile;
-use std::{io::Read, marker::Copy};
+use std::{marker::Copy};

 /// The stage of a shader
 #[derive(Hash, Eq, PartialEq, Copy, Clone, Debug)]
@@ -11,14 +10,14 @@ pub enum ShaderStage {
     Compute,
 }

-impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
-    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
-        match self {
-            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
-            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
-            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
-        }
-    }
+impl Into<shaderc::ShaderKind> for ShaderStage {
+   fn into(self) -> shaderc::ShaderKind {
+       match self {
+           ShaderStage::Vertex => shaderc::ShaderKind::Vertex,
+           ShaderStage::Fragment => shaderc::ShaderKind::Fragment,
+           ShaderStage::Compute => shaderc::ShaderKind::Compute,
+       }
+   }
 }

 fn glsl_to_spirv(
@@ -26,10 +25,19 @@ fn glsl_to_spirv(
     stage: ShaderStage,
     shader_defs: Option<&[String]>,
 ) -> Vec<u32> {
-    let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
-    let mut spv_bytes = Vec::new();
-    output.read_to_end(&mut spv_bytes).unwrap();
-    bytes_to_words(&spv_bytes)
+    let mut compiler = shaderc::Compiler::new().unwrap();
+    let mut options = shaderc::CompileOptions::new().unwrap();
+    if let Some(shader_defs) = shader_defs {
+        for def in shader_defs.iter() {
+            options.add_macro_definition(def, None);
+        }
+    }
+
+    let binary_result = compiler.compile_into_spirv(
+        glsl_source, stage.into(),
+        "shader.glsl", "main", Some(&options)).unwrap();
+
+    binary_result.as_binary().to_vec()
 }

 fn bytes_to_words(bytes: &[u8]) -> Vec<u32> {

[MichaelHills]

Stumbled across this issue in CMake https://gitlab.kitware.com/cmake/cmake/-/issues/20956 which suggests that the whole BUNDLE DESTINATION thing might be due to some CMake default that doesn't make sense when building libs, and I don't need to try and get BUNDLE DESINATION added to shaderc and glsl.

[naithar]

Basically atm I build my "game" as a static lib and then call out to it manually from in an ios xcode project.

Godot works in same way, though it implements it's own main which gets used by iOS application.

Obj-C can be used too, but this would require only main.mm file like this:

@import UIKit;

extern "C" void bevy_main(void);

int main(int argc, char * argv[]) {
    bevy_main();
}

And additional values in Other C Flags in Build Settings: -fmodules -fcxx-modules.

I've tested with 2d -> sprite.rs example, but got empty window. I guess asset loading would require some work for iOS, but correct (?) texture handle was returned Handle<Texture>(82f42690-2c58-4843-a88a-aa066583f97d), so I'm not sure.

• Figure out why coreaudio-sys v0.2.5 panics during compile (that was why I had to disable audio)

This PR (https://github.com/RustAudio/coreaudio-sys/pull/33) seems to be addressing this issue, but I wasn't able to make it work even with this changes. But maybe it's just me as I don't really have much experience with rust.

[MichaelHills]

@naithar oh cool glad to see someone else getting it working too. Yeah I didn’t get the sprite example working, I also assumed it was related to asset loading but I haven’t looked into that yet.

Interesting I’ll try pulling that audio PR later to see if it works.

[MichaelHills]

By the way were you able to get the simulator working? I’m still getting the issue with Target OS is incompatible: library was not compiled for the simulator but supposedly there should be simulator metal support https://github.com/gfx-rs/gfx/pull/2873 so not sure yet what the problem with that is.

[naithar]

No, I've got some error about unsupported architecture from dependency. I'll try to look more into it after I figure out asset loading :)

Godot uses 13.0 sdk version for simulator, since this the only simulator version supporting Metal. But while it does build, MoltenVK fails and application doesn't launch in the end. So cargo lipo might be using lower sdk version that actually required for bevy.

[MichaelHills]

Looks like the ios audio support is still in-progress as per your link, I tried and failed to get that branch to work as well. I'll leave audio disabled for now and might have a try at getting the simulator going.

[naithar]

So compiled binary of bevy faces the same problem - not displaying sprite at specified path (icon.png). Providing a full path fixed this for macOS, but haven't for iOS.

[MichaelHills]

I played around with the paths on iOS on my phone, I added icon.png to my xcode project and examined the various paths...

Bundle.main.bundlePath -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app std::env::current_exe() -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/rust-ios Bundle.main.path(forResource: "icon", ofType: "png") -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/icon.png

I tried std::fs::canonicalize(std::env::current_exe().unwrap().join("..").join("icon.png")).unwrap(); to produce /private/var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/icon.png and pass that to the asset loader. Logs in the loader showed it was successfully loading the file. I guess you already checked the texture handle... but still nothing shows up on screen.

I tried adding some color to the texture to see if it'd help, it didn't. I also tried just the color by itself and no texture, but that doesn't seem to work even on mac.

        .spawn(SpriteComponents {
            material: materials.add(ColorMaterial {
                color: Color::RED,
                texture: texture_handle.into(),
            }),
            ..Default::default()
        });

If we can get a red square rendering, that'd be a good first step. I edited the fragment shader to show only red, worked on mac, didn't work on iOS.

[naithar]

It seems like iOS doesn't really handle/load assets in any way. Also compiled binaries doesn't really like relative paths too, as I've reported in #344

I'm looking for a way to debug rust on iOS, so problem could be traced, but I've got no progress yet.

[MichaelHills]

In crates/bevy_render/src/render_graph/nodes/pass_node.rs I added some logs, basically can_draw_indexed() returns false because one of the "bind groups" is None. I don't know what a bind group is though.

                                    println!("maybe draw indexed can_draw={} index_buf={} bgs={} vbs={}", draw_state.can_draw(), draw_state.index_buffer.is_some(), draw_state.bind_groups.iter().all(|b| b.is_some()), draw_state.vertex_buffers.iter().all(|v| v.is_some()));
                                    for bg in draw_state.bind_groups.iter() {
                                        println!("bg is some {}", bg.is_some());

                                    }
                                    if draw_state.can_draw_indexed() {

logs are

1 visible entities
maybe draw indexed can_draw=false index_buf=true bgs=false vbs=true
bg is some true
bg is some false
bg is some true

[MichaelHills]

Oh nevermind, it was because I had left this in the frag shader

# ifdef COLORMATERIAL_TEXTURE
layout(set = 1, binding = 1) uniform texture2D ColorMaterial_texture;
layout(set = 1, binding = 2) uniform sampler ColorMaterial_texture_sampler;
# endif

I removed it and now it looks like it is issuing the draw call. Still no red square. :(

[naithar]

I've put some logs into fn load_asset(&self, load_request: &LoadRequest) -> Result<TAsset, AssetLoadError> and that's the result.

loading untyped at /private/var/containers/Bundle/Application/DF750ECE-8232-4398-B02E-F9749CCA6A03/BevyTest.app/icon.png
loading: Handle<Texture>(7841c797-5cf9-401d-b1ce-67245a08cd42)
loading asset: LoadRequest { path: "/private/var/containers/Bundle/Application/DF750ECE-8232-4398-B02E-F9749CCA6A03/BevyTest.app/icon.png", handle_id: HandleId(7841c797-5cf9-401d-b1ce-67245a08cd42), handler_index: 1, version: 0 }
file exists: true
bytes read: [... very big array ...]

So iOS actually seems to be loading assets correctly in the end if full path is given.

I removed it and now it looks like it is issuing the draw call. Still no red square. :(

Maybe iOS handles drawing differently or shaders are not compiled so it fails to draw anything?

Edit: Seems like shaders are also compiled correctly, so I guess something might be wrong with shader itself or something is not supported on iOS.

[MichaelHills]

I finally got my red square working. WIthout the image/texture to determine the sprite size, you have to set the size manually:

        .spawn(SpriteComponents {
            material: materials.add(ColorMaterial {
                color: Color::RED,
                // texture: texture_handle.into(),
            }),
            sprite: Sprite { size: Vec2::new(100.0, 100.0) },
            ..Default::default()
        });

so I think the shaders are fine... but still something going wrong in image or texture loading, not sure what yet.

[MichaelHills]

in crates/bevy_render/src/texture/image_texture_loader.rs

        println!("do dyn img");
        let dyn_img = image::load_from_memory_with_format(bytes.as_slice(), img_format)?;
        println!("got dyn img");

It seems to never finish load_from_memory_with_format (at least I don't see the log afterwards), so this explains why the texture loading is not working.

Now the whole "it has a handle" thing is actually because you get a handle back but the png is loaded into a texture on a background thread. Eventually the handle will resolve to a valid texture, but only once it has loaded.

Discovered this initially by adding logs to crates/bevy_sprite/src/sprite.rs

pub fn sprite_system(
    materials: Res<Assets<ColorMaterial>>,
    textures: Res<Assets<Texture>>,
    mut query: Query<(&mut Sprite, &Handle<ColorMaterial>)>,
) {
    println!("sprite system");
    for (mut sprite, handle) in &mut query.iter() {
        let material = materials.get(&handle).unwrap();
        println!("got material with color {} {} {}", material.color.r, material.color.g, material.color.b);
        if let Some(texture_handle) = material.texture {
            println!("got texture handle");
            if let Some(texture) = textures.get(&texture_handle) {
                println!("got texture size {} {}", texture.size.x(), texture.size.y());
                sprite.size = texture.size;
            }
        }
    }
}

and noticed that on iOS it never printed the texture size but on mac it did.

Edit: finally got the error message out from the image loading... Format error decoding Png: CgBI chunk appeared before IHDR chunk

I think somehow the PNG is getting corrupted when added to the iOS project? It's smaller...

ios bytes=12244 mac bytes=15713

[MichaelHills]

Well, I found our problem! You need to disable XCode "Remove text metadata from PNG". I disabled everything though just to be sure... I guess the image loading library can't handle whatever XCode does to it.

My phone is now showing a red bevy bird. 👍

[cart]

Very exciting! Great work here!

[naithar]

Adding std::env::set_current_dir(std::env::current_exe().unwrap().parent().unwrap()); works as workaround for iOS and macOS .app not loading assets at relative path.

[naithar]

@cart is there a reason for AssetServer not using get_root_path method for loading assets at relative paths?

[MichaelHills]

@naithar how did you create a mac .app bundle?

[MichaelHills]

Nevermind found this https://stackoverflow.com/questions/40309538/creatin-a-minimal-macos-app-bundle

So I made

sprite.app/Contents/MacOS/sprite
sprite.app/Contents/Resources/assets/branding/icon.png
sprite.app/Contents/Info.plist

where Info.plist is

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>CFBundleExecutable</key>
    <string>sprite</string>
    <key>CFBundleSignature</key>
    <string>MYAP</string>
    <key>CFBundlePackageType</key>
    <string>APPL</string>
    <key>CFBundleVersion</key>
    <string>1.0.0</string>
    <key>CFBundleIdentifier</key>
    <string>com.myself.myapp</string>
    <key>CFBundleDisplayName</key>
    <string>sprite</string>
    <key>CFBundleName</key>
    <string>sprite</string>
</dict>
</plist>

as per https://github.com/bevyengine/bevy/issues/344 my app doesn't work because I haven't tweaked the asset path yet.

[MichaelHills]

@cart is there a reason for AssetServer not using get_root_path method for loading assets at relative paths?

Oh I see, get_root_path is used in load_asset_folder and filesystem_watcher_system but not in load / load_untyped. It's also not a public function so probably can't try to use it ourselves.

Also, this code path of get_root_path doesn't seem very useful?

        match std::env::current_exe() {
            Ok(exe_path) => exe_path
                .parent()
                .ok_or(1)
                .map(|exe_parent_path| exe_parent_path.to_owned()),
            Err(err) => Err(2),
        }

[naithar]

how did you create a mac .app bundle?

I've simply used Godot's .app template for iOS and tweaked it a little 😃

Also, this code path of get_root_path doesn't seem very useful?

Well, that's basically what I'm passing into set_current_path atm, so binary run by double clicking, .app and iOS application could work correctly. But I guess there is a reason for bevy not using this for asset loading

[MichaelHills]

I've simply used Godot's .app template for iOS and tweaked it a little 😃

Oh I see. :)

Btw I was going to file a bug on the image crate but it turns out the XCode PNG optimization converts it to a different file format https://iphonedevwiki.net/index.php/CgBI_file_format which is not valid PNG anymore, so actually there's no bug.

I'll open a PR here though for better error reporting. Took many hours to track down the issue which some simple error reporting could've highlighted immediately.

Edit: looks like there already is error reporting on asset load failure... odd that I never saw any logs though, will investigate

                Err(err) => {
                    asset_server
                        .set_load_state(result.handle.id, LoadState::Failed(result.version));
                    log::error!("Failed to load asset: {:?}", err);
                }

[MichaelHills]

Okay so logging needs to be configured

diff --git a/Cargo.toml b/Cargo.toml
index 1d8a1bcf..8f437052 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -62,6 +62,9 @@ bevy_gltf = { path = "crates/bevy_gltf", optional = true, version = "0.1" }
 bevy_wgpu = { path = "crates/bevy_wgpu", optional = true, version = "0.1" }
 bevy_winit = { path = "crates/bevy_winit", optional = true, version = "0.1" }

+log = { version = "0.4", features = ["release_max_level_info"] }
+env_logger = "0.7.1"
+
 [dev-dependencies]
 rand = "0.7.2"
 serde = { version = "1", features = ["derive"]}
diff --git a/examples/2d/sprite.rs b/examples/2d/sprite.rs
index d91caefc..3199e72f 100644
--- a/examples/2d/sprite.rs
+++ b/examples/2d/sprite.rs
@@ -1,6 +1,9 @@
 use bevy::prelude::*;

 fn main() {
+    env_logger::init();
+    log::warn!("hi");
+
     App::build()
         .add_default_plugins()
         .add_startup_system(setup.system())

and then you see this :) much better!

$ RUST_LOG=warn cargo run --example sprite
    Finished dev [unoptimized + debuginfo] target(s) in 0.73s
     Running `target/debug/examples/sprite`
[2020-08-29T13:41:11Z WARN  sprite] hi
[2020-08-29T13:41:12Z ERROR bevy_asset::loader] Failed to load asset: LoaderError(Format error decoding Png: CgBI chunk appeared before IHDR chunk

    Caused by:
        CgBI chunk appeared before IHDR chunk)

[naithar]

I managed to finally get something to build, on the simulator gfx-rs / gfx-backend-metal failed with Target OS is incompatible: library was not compiled for the simulator so I ran it on my phone instead to which I got

EventLoop cannot be run after a call to UIApplicationMain on iOS.

Well, I have some progress on this:

This requires changes to gfx's Makefile and some other files:

Changes

`Cargo.toml` for bevy game will require this changes as well as cloning `gfx` before https://github.com/gfx-rs/gfx/commit/b373ebe0f057300fff76534afba25b261527db7a commit ``` [patch.crates-io] gfx-backend-metal = { path = "./gfx/src/backend/metal" } gfx-hal = { path = "./gfx/src/hal" } gfx-warden = { path = "./gfx/src/warden" } ``` Changes for `gfx`: Running `make shader-binaries` will also be required after this changes. ``` diff --git a/Makefile b/Makefile index 3a46869c..a99511b7 100644 --- a/Makefile +++ b/Makefile @@ -32,6 +32,8 @@ else endif ifeq ($(TARGET),aarch64-apple-ios) EXCLUDES+= --exclude gfx-backend-vulkan --exclude gfx-backend-gl + else ifeq ($(TARGET),x86_64-apple-ios) + EXCLUDES+= --exclude gfx-backend-vulkan --exclude gfx-backend-gl else FEATURES_GL=gl endif @@ -96,4 +98,9 @@ ifeq ($(UNAME_S),Darwin) xcrun -sdk iphoneos metal -c *.metal -mios-version-min=11.4 && \ xcrun -sdk iphoneos metallib *.air -o gfx-shaders-ios.metallib && \ rm *.air + # iOS Simulator + cd ./src/backend/metal/shaders && \ + xcrun -sdk iphonesimulator metal -c *.metal -mios-simulator-version-min=13.0 && \ + xcrun -sdk iphonesimulator metallib *.air -o gfx-shaders-ios-simulator.metallib && \ + rm *.air endif diff --git a/examples/Cargo.toml b/examples/Cargo.toml index f9fc6e4e..415b0f5f 100644 --- a/examples/Cargo.toml +++ b/examples/Cargo.toml @@ -58,7 +58,7 @@ version = "0.6" features = ["x11"] optional = true -[target.'cfg(any(target_os = "macos", all(target_os = "ios", target_arch = "aarch64")))'.dependencies.gfx-backend-metal] +[target.'cfg(any(target_os = "macos", all(target_os = "ios")))'.dependencies.gfx-backend-metal] path = "../src/backend/metal" version = "0.6" optional = true diff --git a/src/backend/metal/Cargo.toml b/src/backend/metal/Cargo.toml index 4a7734aa..7a279c8d 100644 --- a/src/backend/metal/Cargo.toml +++ b/src/backend/metal/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "gfx-backend-metal" -version = "0.6.0" +version = "0.6.1" description = "Metal API backend for gfx-rs" homepage = "https://github.com/gfx-rs/gfx" repository = "https://github.com/gfx-rs/gfx" diff --git a/src/backend/metal/shaders/gfx-shaders-ios.metallib b/src/backend/metal/shaders/gfx-shaders-ios.metallib index f20b7f71..f946086a 100644 Binary files a/src/backend/metal/shaders/gfx-shaders-ios.metallib and b/src/backend/metal/shaders/gfx-shaders-ios.metallib differ diff --git a/src/backend/metal/shaders/gfx-shaders-macos.metallib b/src/backend/metal/shaders/gfx-shaders-macos.metallib index b331b9ac..642ae185 100644 Binary files a/src/backend/metal/shaders/gfx-shaders-macos.metallib and b/src/backend/metal/shaders/gfx-shaders-macos.metallib differ diff --git a/src/backend/metal/src/internal.rs b/src/backend/metal/src/internal.rs index ca37701a..e662a07b 100644 --- a/src/backend/metal/src/internal.rs +++ b/src/backend/metal/src/internal.rs @@ -452,8 +452,10 @@ impl ServicePipes { pub fn new(device: &metal::DeviceRef) -> Self { let data = if cfg!(target_os = "macos") { &include_bytes!("./../shaders/gfx-shaders-macos.metallib")[..] - } else { + } else if cfg!(target_arch = "aarch64") { &include_bytes!("./../shaders/gfx-shaders-ios.metallib")[..] + } else { + &include_bytes!("./../shaders/gfx-shaders-ios-simulator.metallib")[..] }; let library = device.new_library_with_data(data).unwrap(); diff --git a/src/warden/Cargo.toml b/src/warden/Cargo.toml index 310056ce..8f7fa0a5 100644 --- a/src/warden/Cargo.toml +++ b/src/warden/Cargo.toml @@ -52,7 +52,7 @@ path = "../../src/backend/dx11" version = "0.6" optional = true -[target.'cfg(any(target_os = "macos", all(target_os = "ios", target_arch = "aarch64")))'.dependencies.gfx-backend-metal] +[target.'cfg(any(target_os = "macos", all(target_os = "ios")))'.dependencies.gfx-backend-metal] path = "../../src/backend/metal" version = "0.6" features = ["auto-capture"] ```

After this running cargo lipo --targets aarch64-apple-ios x86_64-apple-ios will produce a binary that could be used to run application on both simulator starting from iOS 13.0 and device with metal support.

[naithar]

After having #334, #324 and some workarounds for #344 doing iOS stuff should be a bit easier.

Edit:

Cargo.toml for bevy game will require this changes as well as cloning gfx before gfx-rs/gfx@b373ebe commit

Cloning gfx before this commit is required for bevy to compile - current version of wgpu on bevy is using old version of gfx and there are a breaking changes.

[MichaelHills]

Nice work on the simulator! I've actually started prototyping my game and learning how to use systems and Query etc. Using mac for dev, then deploy on phone when ready. I've hacked up some touch support so I can move my character (a sphere...) around.

Here's my diffs for the touch if you want to play with it, I'll try and make a PR when I have time.

diff --git a/crates/bevy_input/src/lib.rs b/crates/bevy_input/src/lib.rs
index 1893d86b..a9fa49fe 100644
--- a/crates/bevy_input/src/lib.rs
+++ b/crates/bevy_input/src/lib.rs
@@ -1,6 +1,7 @@
 mod input;
 pub mod keyboard;
 pub mod mouse;
+pub mod touch;
 pub mod system;

 pub use input::*;
@@ -12,6 +13,7 @@ pub mod prelude {
 use bevy_app::prelude::*;
 use keyboard::{keyboard_input_system, KeyCode, KeyboardInput};
 use mouse::{mouse_button_input_system, MouseButton, MouseButtonInput, MouseMotion, MouseWheel};
+use touch::{touch_input_system, TouchInput, TouchInputState};

 use bevy_ecs::IntoQuerySystem;

@@ -25,6 +27,7 @@ impl Plugin for InputPlugin {
             .add_event::<MouseButtonInput>()
             .add_event::<MouseMotion>()
             .add_event::<MouseWheel>()
+            .add_event::<TouchInput>()
             .init_resource::<Input<KeyCode>>()
             .add_system_to_stage(
                 bevy_app::stage::EVENT_UPDATE,
@@ -34,6 +37,11 @@ impl Plugin for InputPlugin {
             .add_system_to_stage(
                 bevy_app::stage::EVENT_UPDATE,
                 mouse_button_input_system.system(),
+            )
+            .init_resource::<TouchInputState>()
+            .add_system_to_stage(
+                bevy_app::stage::EVENT_UPDATE,
+                touch_input_system.system(),
             );
     }
 }
diff --git a/crates/bevy_input/src/touch.rs b/crates/bevy_input/src/touch.rs
new file mode 100644
index 00000000..1c681231
--- /dev/null
+++ b/crates/bevy_input/src/touch.rs
@@ -0,0 +1,103 @@
+use bevy_ecs::{Local, ResMut, Res};
+use bevy_app::{Events, EventReader};
+use std::collections::{HashMap, HashSet};
+
+
+/// A touch input event
+#[derive(Debug, Clone)]
+pub struct TouchInput {
+    pub phase: TouchPhase,
+    pub x: f64,
+    pub y: f64,
+    ///
+    /// ## Platform-specific
+    ///
+    /// Unique identifier of a finger.
+    pub id: u64,
+}
+
+/// Describes touch-screen input state.
+#[derive(Debug, Hash, PartialEq, Eq, Clone, Copy)]
+#[cfg_attr(feature = "serialize", derive(serde::Serialize, serde::Deserialize))]
+pub enum TouchPhase {
+    Started,
+    Moved,
+    Ended,
+    Cancelled,
+}
+
+#[derive(Default)]
+pub struct TouchSystemState {
+    touch_event_reader: EventReader<TouchInput>,
+}
+
+#[derive(Debug, Clone)]
+pub struct ActiveTouch {
+    start_x: f64,
+    start_y: f64,
+    cur_x: f64,
+    cur_y: f64,
+}
+
+impl ActiveTouch {
+    pub fn dx(&self) -> f64 {
+        self.cur_x - self.start_x
+    }
+    pub fn dy(&self) -> f64 {
+        self.cur_y - self.start_y
+    }
+}
+
+#[derive(Default)]
+pub struct TouchInputState {
+    pub active_touches: HashMap<u64, ActiveTouch>,
+    pub just_pressed: HashSet<u64>,
+    pub just_released: HashSet<u64>,
+    pub just_cancelled: HashSet<u64>,
+}
+
+
+/// Updates the TouchInputState resource with the latest TouchInput events
+pub fn touch_input_system(
+    mut state: Local<TouchSystemState>,
+    mut touch_state: ResMut<TouchInputState>,
+    touch_input_events: Res<Events<TouchInput>>,
+) {
+    touch_state.just_pressed.clear();
+    touch_state.just_released.clear();
+    touch_state.just_cancelled.clear();
+
+    for event in state
+        .touch_event_reader
+        .iter(&touch_input_events)
+    {
+        // println!("{:?}", event);
+
+        let active_touch = touch_state.active_touches.get(&event.id);
+        match event.phase {
+            TouchPhase::Started => {
+                touch_state.active_touches.insert(event.id, ActiveTouch {
+                    start_x: event.x,
+                    start_y: event.y,
+                    cur_x: event.x,
+                    cur_y: event.y,
+                });
+            },
+            TouchPhase::Moved => {
+                let prev = active_touch.unwrap();
+                let mut cur = prev.clone();
+                cur.cur_x = event.x;
+                cur.cur_y = event.y;
+                touch_state.active_touches.insert(event.id, cur);
+            },
+            TouchPhase::Ended => {
+                touch_state.active_touches.remove(&event.id);
+                touch_state.just_released.insert(event.id);
+            },
+            TouchPhase::Cancelled => {
+                touch_state.active_touches.remove(&event.id);
+                touch_state.just_cancelled.insert(event.id);
+            },
+        };
+    }
+}
diff --git a/crates/bevy_winit/src/lib.rs b/crates/bevy_winit/src/lib.rs
index b6261ae8..5659086f 100644
--- a/crates/bevy_winit/src/lib.rs
+++ b/crates/bevy_winit/src/lib.rs
@@ -4,6 +4,7 @@ mod winit_windows;
 use bevy_input::{
     keyboard::KeyboardInput,
     mouse::{MouseButtonInput, MouseMotion, MouseScrollUnit, MouseWheel},
+    touch::{TouchInput, TouchPhase}
 };
 pub use winit_config::*;
 pub use winit_windows::*;
@@ -192,6 +193,22 @@ pub fn winit_runner(mut app: App) {
                         });
                     }
                 },
+                WindowEvent::Touch(touch) => {
+                    let mut touch_input_events =
+                        app.resources.get_mut::<Events<TouchInput>>().unwrap();
+                    // TODO use convert* thing
+                    touch_input_events.send(TouchInput {
+                        phase: match touch.phase {
+                            event::TouchPhase::Started => TouchPhase::Started,
+                            event::TouchPhase::Moved => TouchPhase::Moved,
+                            event::TouchPhase::Ended => TouchPhase::Ended,
+                            event::TouchPhase::Cancelled => TouchPhase::Cancelled,
+                        },
+                        x: touch.location.x,
+                        y: touch.location.y,
+                        id: touch.id,
+                    });
+                },
                 _ => {}
             },
             event::Event::DeviceEvent { ref event, .. } => {

You can use it as follows:

The app setup...

        .init_resource::<TouchHandlerState>()
        .add_system(touch_handler.system())

And then the code

struct TouchHandlerState {
    touch_event_reader: EventReader<TouchInput>,
}

fn touch_handler(
    mut state: ResMut<TouchHandlerState>,
    touch_events: Res<Events<TouchInput>>,
    touch_input_state: Res<TouchInputState>,
) {
    for event in state.touch_event_reader.iter(&touch_events) {
        println!("{:?}", event);
    }
    // touch_input_state.active_touches;
    // touch_input_state.just_pressed;
    // touch_input_state.just_released;
    // touch_input_state.just_cancelled;
}

in my prototype game I didn't actually use the event stream and just did this to get a finger direction and move my character in the direction the held-finger has moved from the original touch position.

    let mut dx = 0.0;
    let mut dy = 0.0;
    for id in touch_input_state.active_touches.keys() {
        let t = touch_input_state.active_touches.get(&id).unwrap();
        dx = t.dx();
        dy = t.dy();
        break;
    }

[naithar]

Cool 👍 For iOS's touch processing this might be relevant: https://github.com/godotengine/godot/pull/39624 as winit seems to be using touchesBegan/touchesMoved methods for touch recognition too. I'm not 100% sure about this being a problem for bevy, but might be something to take into account.

[MichaelHills]

Oh yeah click emulation. I also have no idea if winit supports pinch to zoom or other gestures which would be nice to have.

[naithar]

Hardly. From what I've seen it gives only basic touch info, just as iOS would without using gesture recognizers. But this could be used for making all zoom, pitch, pan and whatever gestures required, so I don't really see a problem.

Godot's required this change for touch processing because newest iPhones send moved events to early which broke gui input handling due to the way it's made (emulation of mouse from touches).

[CleanCut]

Thank you for your awesome work on this, @MichaelHills & @naithar! I think a lot of us are watching your efforts with interest in anticipation of iOS support. 😄

One comment: I encourage just getting the basic touch info working, and forgetting about gestures/mouse-emulation. Gestures/mouse-emulation support (if any) would be best implemented as an optional driver or ui-level system anyway, not handled up at the raw input level. See the excellent Event Chaining as a Decoupling Method in Entity-Component-System post about such designs.

[naithar]

One comment: I encourage just getting the basic touch info working, and forgetting about gestures/mouse-emulation.

Yeah, that's what I think too.

Basically there are no stoppers for iOS support left:

• There are already PR for shader fix for iOS #324
• We also have a PR for touches #334 which seems to be as basic as it could be (which is great for now).
• I'm planning on making a PR for gfx to allow compilation for iOS Simulators. But I just want to test everything before I actually open it.
• Also wgpu crate update might be required for everything to work.

iOS could probably be hidden behind default_ios feature for now as coreaudio is not yet ready for iOS. It would probably required to make a templates so iOS application could be tested easily.

[MichaelHills]

Ohhh I didn't realise https://github.com/bevyengine/bevy/pull/334 was a PR, I assumed it was the issue https://github.com/bevyengine/bevy/issues/300 I was looking at earlier. Cool. :) I didn't need to spend that time writing touch support haha, but it's good I'm learning how to use Bevy at the same time.

[naithar]

gfx simulator support should land soon-ish (If it didn't already) for both master and 0.6 version which is used atm by bevy.

cmake-rs could be bumped to 0.1.43 by project with

[dependencies]
cmake = "=0.1.43"

Modified shaderc is still required, so I guess something should be done on that part too in shaderc and glslang.

I've also modified Cargo.toml to simplify building:

index 56f70cc..8ccd284 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -8,17 +8,17 @@ edition = "2018"

 [dependencies]
 cmake = "=0.1.43"
-bevy = { version = "0.1.3" }
+bevy = { version = "0.1.3", default-features = false, features=["default_ios"]  }

 [patch.crates-io]
+bevy = { path = "./bevy", default-features = false, features=["default_ios"] }
+shaderc = { path = "./shaderc-rs/shaderc-rs" }
--- a/bevy/Cargo.toml
+++ b/bevy/Cargo.toml
@@ -17,6 +17,8 @@ default = [
     "bevy_audio", "bevy_gltf", "bevy_wgpu", "bevy_winit",
     "dynamic_plugins", "png", "hdr", "mp3", "x11"
 ]
+default_ios = ["bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr"]
+

[MichaelHills]

gfx simulator support should land soon-ish (If it didn't already) for both master and 0.6 version which is used atm by bevy.

Awesome.

Modified shaderc is still required, so I guess something should be done on that part too in shaderc and glslang.

Yeah either an update to cmake-rs or to shaderc/glslang is required due to the bundle destination thing referenced earlier.

+    BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}

My opinion is that cmake should not default CMAKE_MACOSX_BUNDLE to ON https://gitlab.kitware.com/cmake/cmake/-/issues/20956#

Failing that I think it can be fixed in cmake-rs as I mentioned here https://github.com/alexcrichton/cmake-rs/issues/95 by adding cmd.arg("-DCMAKE_MACOSX_BUNDLE=FALSE");. This way cmake-rs can workaround cmake's bad defaults.

Haven't really received any response from the above projects though.

In the meantime... waiting for Naga to be ready so we can ditch shaderc. :)

[naithar]

Awesome.

It's already landed, so updating gfx-metal-backend to 0.6.2 should do the trick.

In the meantime... waiting for Naga to be ready so we can ditch shaderc. :)

Yeah, naga looks very promising. But at least now we can already do some stuff :)

While at it, it seems iOS will need some WindowMode that tries to save window aspect ratio or scales content based on its size. But maybe it' already there and I just don't know it :)

Edit: This does look kinda related https://github.com/bevyengine/bevy/issues/195, but it seems to be only for UI

[MichaelHills]

I did a cargo update and now simulator works. 👍

[MichaelHills]

I tried to add a build script step to XCode so I don't need to manually run my cargo build script in the terminal. I couldn't get it to work though, have you had any luck with that?

[MichaelHills]

Also my staticlib with --release using opt-level = 'z' is like 192MB. Am I doing something wrong or is it meant to be that large?

[naithar]

Yeah, it's mostly okay. Binary has lots of unused symbols and multiple archs. Once you upload a binary to AppStore it would shrink, since App Slicing would be used on Apple side.

I tried to add a build script step to XCode so I don't need to manually run my cargo build script in the terminal. I couldn't get it to work though, have you had any luck with that?

I didn't do it. But you should have some luck if you create a custom script target and then link .a file to library scheme as well as resulting application target dependencies. Godot should have some tutorials on this. I can probably make one, but not right now :)

[MichaelHills]

I managed to create an archive (disabled bitcode, otherwise get some error about Rust using llvm10 and apple using llvm11). It is 91MB. Not too bad I guess. Was hoping for smaller. :)

[naithar]

Once you upload it will be smaller - resulting app should be around 20-30MB in size depending on libraries used. My test Godot library is about 600mb in size, but results in 28mb as application on TestFlight

[MichaelHills]

There are a couple of issues around the window on iOS. I've set the window resolution to 1334 by 750 which is the iPhone 6 resolution. However it doesn't reach the edge of the screen, stops a tiny bit short. i.e. black bars on the right and bottom. Increasing width did nothing.

I also can't get rid of the status bar. I configured it to off in the XCode project, but it seems to want to stay on. I tried to hook into the window created event, and access the winit window directly to call set_decorations(false) but that did nothing.

Edit: I've fixed my game to landscape in the XCode project, that works to prevent portrait mode.