ml5 file size - Githubissues

ml5js / ml5-next-gen

Repo for next generation of ml5.js: friendly machine learning for the web! 🤖

https://ml5js.org/

Other

54 stars 18 forks source link

ml5 file size #118

Open shiffman opened 5 months ago

shiffman commented 5 months ago

In preparing a ml5.js alpha build, we noticed the filze size is quite large.

It would be worth taking the time to research and look into why this is and if there are dependencies we can remove / scale down. I recall this used to be an issue due to the inclusion of magenta but i don't think we are anymore?

shiffman commented 5 months ago

So it looks like #111 may have increased the filesize, @ziyuan-linn will take a look.

lindapaiste commented 4 months ago

Seems like it's all TensorFlow? Screenshot of the webpack-bundle-analyzer. It does seem like we have both the webgl and cpu backends in the bundle. We may want to include only 1 backend, so that it works out of the box, and tell the user to include an additional <script> tag to register any other backends.

Edit: now I'm looking at the bundle and we're including source maps. We don't want those in a production build because they are huge.

We also have a comment block with the Google License that appears over 1,000 times even in the .min.js file. Seems excessive.

ziyuan-linn commented 4 months ago

@lindapaiste Thank you for looking into this. I set extractComments to true and it reduced the file size from 4.18 Mib to 3.43 Mib!

Though I am pretty sure the source map is built to a different file named ml5.min.js.map. I tried changing devtool to false and the bundle size did not change.

shiffman commented 4 months ago

This is probably a longer term discussion, but just thinking through some ideas for the future:

What if we don't include tensorflow.js at all in the bundle? What would be required for ml5.js end users, is it as simple as just including the tf.js script tag as well? Maybe this has no real benefit since the overall size of the webpage wouldn't change.
The cpu back-end is needed for the neuroevolution examples, but is otherwise rarely used. We could consider a build process that creates separate versions ml5.min.js, ml5.webgl.min.js, ml5.cpu.min.js? Or even build ml5.js in a modular way so that beginners can use the entire library, but if someone is building a project that uses only, say, facemesh, they could load ml5-facemesh.min.js?

This discussion may become a higher priority if/when we consider bundling other backends for supporting new models, like transformers.js.

3.48Mb feels "ok" to me for now so maybe we revisit this after 1.0?

MOQN commented 2 months ago

I agree with the idea of organizing the library in a modular way! During a workshop I recently offered at Tongji University in Shanghai, China, I found it challenging to demonstrate examples with multiple iterations because the library's large file size had to be downloaded via a VPN connection.

We can emphasize it on the website and consider adding a friendly error message like: "To use facemesh, please import ml5-facemesh.min.js. Visit the website for detailed information."

shiffman commented 1 week ago

Noting that a discussion here is happening with p5.js which may have some relevant and useful info. cc @sharellb and @gohai as I know you both may be investigating making ml5.js modular.

https://github.com/processing/p5.js/issues/6776