ml5js / ml5-next-gen

Repo for next generation of ml5.js: friendly machine learning for the web! 🤖
https://ml5js.org/
Other
95 stars 23 forks source link

Add offline module #200

Open ziyuan-linn opened 2 months ago

ziyuan-linn commented 2 months ago

The PR implements an offline version of the ml5.js library.

Webpack will now build ml5-offline.js and ml5-offline.min.js in addition to ml5.js and ml5.min.js. The file ml5-offline.js and ml5-offline.min.js is self-contained and will not make additional network requests. Once a user downloads ml5-offline.min.js, the ml5 library can run entirely without internet.

The entry point for the offline model is src/offline/index.js. It imports the ml5 object, injects a loadOfflineModel function into each ml5 model's prototype, and exports the ml5 object with the modified models. The approach keeps the offline implementation mostly separate from the main source code.

Changes:

To test this implementation, find an example sketch and change ml5.js to ml5-offline.js in index.html.

Note: This is WIP, ml5-offline.js only works for HandPose with modelType: "lite".

shiffman commented 2 months ago

This is exciting to see thank you @ziyuan-linn for working on this! One consideration is that running a stand-along node.js script for downloading the models may involves concepts/steps unfamiliar to beginners especially those used to working only in the p5.js web editor. I think it's ok for this to be the primary way to get the models since this is an "edge" use case, however, we should probably include some detailed documentation for total beginners. I could also record a video tutorial on this topic. Alternatively we could consider providing links to some model downloads directly but this might be hard to maintain.

ziyuan-linn commented 2 months ago

@shiffman Thank you for the review! The node.js script for downloading models is meant to be only called by the ml5.js maintainers. I could also set it up as a postinstall script that automatically runs after yarn so we never have to worry about it. The purpose of the script is to download the models so WebPack can bundle the model files into the ml5 library file itself.

As far as the users are concerned, they only need to change ml5.js to ml5-offline.js in the script tag. If they want to go fully offline, they can also download the p5.js and ml5-offline.js scripts and put them in the sketch folder. We can definitely have a quick guide/documentation for this!