search

tracel-ai / burn

Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.
https://burn.dev
Apache License 2.0
7.82k stars 372 forks source link

Cross-Platform Integration using `xtask-wasm` #851

Open antimora opened 9 months ago

antimora commented 9 months ago

Description:

As we aim to make our project more cross-platform compatible, leveraging xtask-wasm appears to be a viable solution.

Objectives:

  1. Translate existing scripts to their xtask-wasm equivalents:
    • examples/image-classification-web/build-for-web.sh
    • examples/image-classification-web/run-server.sh
    • examples/mnist-inference-web/build-for-web.sh
    • examples/mnist-inference-web/run-server.sh

About xtask-wasm:

Features of xtask-wasm include:

Reason for Integration:

The primary incentive for integrating xtask-wasm is its cross-platform capabilities. Ensuring our project's scripts are compatible across different platforms will significantly boost accessibility and usability.

We would appreciate insights, suggestions, or any potential challenges foreseen in this transition.

antimora commented 9 months ago

@Luni-4, wanna give a try? =)

Luni-4 commented 9 months ago

If it's not urgent, feel free to assign to me this task :)

antimora commented 9 months ago

No, it's not urgent. I just could not complete it in time for the example. I'll be traveling soon so I can't finish it.

Luni-4 commented 8 months ago

@antimora

We can rewrite the https://github.com/burn-rs/burn/blob/main/examples/image-classification-web/https_server.py in https://github.com/tokio-rs/axum in order to simplify the xtask-wasm transition. Are you willing to do that?

antimora commented 8 months ago

@antimora

We can rewrite the https://github.com/burn-rs/burn/blob/main/examples/image-classification-web/https_server.py in https://github.com/tokio-rs/axum in order to simplify the xtask-wasm transition. Are you willing to do that?

In the end, I think you might end up recreating xtask-wasm. It is better to give a try to use the lib. If we can get it working, the same solution can be applied to other projects (outside burn), including mnist digits.

Luni-4 commented 8 months ago

@antimora We can rewrite the https://github.com/burn-rs/burn/blob/main/examples/image-classification-web/https_server.py in https://github.com/tokio-rs/axum in order to simplify the xtask-wasm transition. Are you willing to do that?

In the end, I think you might end up recreating xtask-wasm. It is better to give a try to use the lib. If we can get it working, the same solution can be applied to other projects (outside burn), including mnist digits.

The Start command runs what we have previously defined in Dist. In our case, building the web app and running the server are two distinct phases. So I suppose we have to implement a server and then run it, but ok, let me try to use the default options and see if we will obtain the same results

antimora commented 7 months ago

Related: https://github.com/burn-rs/burn/issues/917

AlexErrant commented 7 months ago

Another idea is to do something similar to this PR to wasm_bindgen and use Python. Sure we could remove the Python dependency from examples/mnist-inference-web, but we can't remove it from burn-dataset or examples/guide. Should we move from a solid and popular dependency like Python to such a small library? Python is already incredibly popular in machine learning, and I'm not sure having people read/grok/debug xtask-wasm is a good use of time when they're just trying to follow an example. Related. Also, iteration times suffer since Python is a scripting language and WASM is not.

(Advanced apologies to Luni-4 for potentially nuking all your work 😬)

antimora commented 7 months ago

No, we need to stick with xtask because it is most portable. We have image classifier that does not use python. And I predict we might want to remove python dependency entirely in the future.

nathanielsimard commented 7 months ago

I think having minimal dependency for the build process is very valuable and will contribute to a simpler setup. We use Python in burn-dataset to download data from Huggingface, and it often causes problems, but we don't have any other viable alternative for now.

antimora commented 5 months ago

@Luni-4 has found this build tool we should look into: https://github.com/trunk-rs/trunk

Here is their vanilla js example: https://github.com/trunk-rs/trunk/tree/main/examples/vanilla

Luni-4 commented 5 months ago

The documentation and some information are here: https://trunkrs.dev/