search

tracel-ai / burn

Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.
https://burn.dev
Apache License 2.0
8.86k stars 435 forks source link

Provide usage examples on Android and iOS #645

Open tangxuesong6 opened 1 year ago

tangxuesong6 commented 1 year ago

It would be great if there are examples of using Android or iOS devices.

nathanielsimard commented 1 year ago

We don't have plans to provide examples on mobile devices just yet, but it shouldn't be different than normal rust projects: https://source.android.com/docs/setup/build/rust/building-rust-modules/overview

elbaro commented 5 months ago

Which backend performs the best on Android / iOS? Can wgpu-backend utilize neural engines?

antimora commented 5 months ago

Which backend performs the best on Android / iOS? Can wgpu-backend utilize neural engines?

I haven't tried this myself but here is what supported for Android / iOS:

burn-ndarray can be built for Android and iOS for CPU. For iOS, ndarray supports Accelerate library (use accelerate feature flag), which can take advantage platform specific hardware.

burn-wgpu uses https://github.com/gfx-rs/wgpu which supports Vulcan (on Android) and Metal (on iOS).

I do not think we have anything specific to neural engines on iOS but support on Metal is your closest bet.

CC @laggu, @louisfd and @syl20bnr

soupslurpr commented 3 months ago

On Android, Burn crashes (at least with the wgpu backend) when trying to persist the autotune cache because the dirs crate returns None.

antimora commented 3 months ago

On Android, Burn crashes (at least with the wgpu backend) when trying to persist the autotune cache because the dirs crate returns None.

You can disable auto-tune.

soupslurpr commented 3 months ago

You can disable auto-tune.

Wouldn't it result in worse performance? I'll still try, could you tell me how to do that?

antimora commented 3 months ago

You can disable auto-tune.

Wouldn't it result in worse performance? I'll still try, could you tell me how to do that?

auto tune is controlled via feature flag. Please see web classification inference example

soupslurpr commented 3 months ago

Thanks, I just disabled the default features for burn-wgpu as the example did and it doesn't run auto tune anymore. The speeds (at least with Whisper) are the same as with auto tune on my Google Pixel 7.

soupslurpr commented 3 months ago

With more testing perhaps there is a speed difference of around 36%. I modified dirs-rs to support Android by reading an environment variable for the home directory. In the first function in the Rust library I call I set the environment variable to the path of the Android application.filesDir.

Autotune works perfectly now.

soupslurpr commented 3 months ago

Also wanted to say thanks for working on Burn, I've been liking it so far, interested to see how much f16 support in wgpu or switching to dawn will improve performance (and later quantization as it seems to be in progress).