Suggestions: smaller modules and NPM

[Bondifrench]

Hi there,

this seems to be a great platform in javascript for machine learning however it's lacking in popularity compared to other libraries like Synaptic which might not be as complete as yours. Questions:

• Would you be open to destructure your libraries in small modules? So you reduce the complexity of your files, the size, and improve readability? For instance Classifier.js could be separated into a CoreClassifier.ja then 'LDA.js,QDA.jsetc..or breakdown ofRegression.jsbetweenRidge,Lasso,KNN, separateLeast Squares,LeastAbsolute` etc.. A great example of this destructuring and modularity is this ml library, see the tools list

• Would you also consider putting all these separated modules on npm? Once your codebase is modular and on npm, they are easier to require with NodeJs and if we want to use in the browser, we can use browserify or Webpack (I personally have a preference for the first as it is much simpler to use).

I think your project deserves more attention and putting in small manageable modules and on npm, would increase the exposure of your project.

[lauerfab]

Hi,

Thanks for your interest and suggestions. I will probably consider splitting files into separate modules in the next release. Regarding node and npm, I am not so sure... If you really plan to use this on a server, you should probably consider other C/java/python alternatives and maybe just write a javascript wrapper around that. The whole point of MLweb is to do machine learning when you precisely cannot do that. There might be other reasons to use MLweb in nodejs that I fail to see, but I will not have the time to build npm packages for those in a near future.

[Bondifrench]

That's a shame, by putting it on NPM and within the NodeJs ecosystem, you will facilitate reuse of your code, server side or browser side, it doesn't matter. The functions that you wrote are usable on both sides almost as is. Yes most libraries for ML have been written in Python, C++, Lua etc...Why not, open your code to a wider community? it is still part of the Javascript community.

You can read here about the use of Common.js modules. The use of Common.js modules, coupled with Browserify really helps modularity and code reuse.

[r03ert0]

modular npm packages would be great indeed! Client-side development is much nicer using npm than wildly downloading libraries from websites... (npm also gives you information about the health of a project -- number of downloads, versions, etc)

[sorig]

@lauerfab I'm trying to use lalolib with Observable notebooks (see an example here). The way the library is currently packaged makes it a pain to use with Observable as it pollutes the window object with hundreds of functions rather than wrapping them in a single module object. It would be great if you could package this up in a nicer way.

Note that the argument about server-side vs client-side usage of the library doesn't apply here as Observable runs all code client-side in the browser. In fact, the philosophies behind MLWeb (that you outlined in MLweb: A toolkit for machine learning on the web) and Observable align really nicely!

[lauerfab]

OK, so I finally created some Modules. However, I did not go as far as NPM (mostly because I also want to keep things simple for people that are not javascript developers and know nothing about NPM ; but yes I know I could have done both...). So the modules can simply be used for instance with

and prefixing every functions with "mljs.". Four versions are available depending on what you need (LALOLib or ML.js, with or without linear programming via glpk).

Dividing into smaller modules does not seem to bring too much benefit since all functions would depend on the linear algebra module which already constitute most of the code (so loading a module for SVM would only save maybe 10%-20% of KB compared to loading all machine learning functions). Note also that, for most purposes, mljs-noglpk-module.js is sufficient and only 253 KB.

Of course, this does not solve the issue about development with large files instead of tiny modules, but this might help to use the library.

[sorig]

Amazing! Thanks a lot. I'm able to compile the module versions locally. However, the links in the main readme seem to be broken (currently returns 404)

[lauerfab]

Thanks for noticing. The links have been fixed.

[amaramth]

for those of us who npm is the easiest way to install anything, serverside or webside, it would be nice if you put this on npm