npm's like "global" mode

[joerg-krause]

Currently, there is no "global" mode for luvi/lit like npm has: https://docs.npmjs.com/misc/config#global.

Is this planned in the near future?

[creationix]

There are no plans for it. node regrets adding the feature and lit has no concept of prefix really other than relative to the bin folder containing lit/luvit.

[creationix]

That said, installing a library at $HOME/deps will be global for that user, or /deps is global for the system (though will make sysadmins cringe)

[joerg-krause]

Do you know why node regrets?

Buildroot uses npm's global mode to install Node.js packages in the global folder /usr/lib/node_modules. This simplifies installation of packages for embedded systems where you have no users, no home - just root.

npm_config_arch=arm \
npm_config_target_arch=arm \
npm_config_build_from_source=true \
npm_config_nodedir=/home/joerg/buildroot/output/build/nodejs-0.10.41 \
npm_config_prefix=/home/joerg/buildroot/output/target/usr \
/home/joerg/buildroot/output/host/usr/bin/npm install -g execSync

Running npm in global mode, downloads the whole package 'execSync' (from github?), runs npm install to fetch the dependencies and installs everything to /usr/lib/node_modules. This is quite handy!

I guess I could mimic this easily. Unfortunatly, I do not see a download url for most of the packages on https://luvit.io/lit.html.

EDIT: Installation in Buildroot means building a rootfs on a host target which is copied to the target, eg. on a sd card.

[creationix]

Where in the filesystem will your luvit programs live inside the rootfs?

How unkosher would it be to put them under /usr/* and then install global lit modules under /usr/deps?

Or if you need things deeper, programs could go under $LUVIT_PREFIX/bin with libraries under $LUVIT_PREFIX/deps and you can put $LUVIT_PREFIX wherever you want like /usr/lib/luvit.

[creationix]

If this is not an option, then I can see why you want a global mode so that each program doesn't need it's own copy of all dependencies. Space is often limited on such systems.

[joerg-krause]

I try to think in terms of luvi apps and bundles. My first idea was to use lit make to build lit and luvit, but then I realized I will have three times the binary of luvi installed on my target. Luvi itself, and luvi bundled both in lit and in luvit. Of course, this is not the end of the world, but luckely luvi can be used as a runtime for lit and luvit and other packages.

Where in the filesystem will your luvit programs live inside the rootfs?

For now, my approach is to install everything in a global folder $PREFIX/lib/luvi_modules :smile: , with $PREFIX=/usr for Buildroot.

So luvit (as a package) will be installed in /usr/lib/luvi_modules/luvit. I can use it with luvi /usr/lib/luvi_modules/luvit and bundle it together with other apps. This is useful for hacking on luvit, too.

To ease things, I guess I can put an executable script named luvit into /usr/bin (there is no /usr/local/bin in Buildroot) passing the bundles to luvi. But I have not tried yet.

To install an app depending on luvit, I would fetch the sources, run lit author/app on the host system, which will install luvit as a dependency in the deps subdirectory and install everything into /usr/lib/luvi_modules/app.

How unkosher would it be to put them under /usr/* and then install global lit modules under /usr/deps?

I think this is not a good idea as it is not in terms of the Linux Filesystem Hierarchy.

Or if you need things deeper, programs could go under $LUVIT_PREFIX/bin with libraries under $LUVIT_PREFIX/deps and you can put $LUVIT_PREFIX wherever you want like /usr/lib/luvit.

Let's say executable binaries go into $PREFIX/bin and the package into $PREFIX/lib/luvi_modules by default.

If this is not an option, then I can see why you want a global mode so that each program doesn't need it's own copy of all dependencies. Space is often limited on such systems.

Yes, this would save space, but can be difficult to manage. Even if two apps may depend on the same package or module, they might depend on different versions. And even if they depend on the same version initially, after updating the dependency (install globally), one package might still work, whereas the other is broken.

So, to simply the dependency management, every package should get their local copy of the dependency. As far as I understand, this is the same way npm goes and not let the user require a package installed globally (until you set NODE_PATH).

[creationix]

By the way, you can still have standalone luvi apps without copying the luvi binary into each app.

1. Install luvi to some shared place on your fs, perhaps $PREFIX/bin/luvi.
2. Create a zip file for your luvi app.
3. Prefix a unix shebang line to your zip file and mark executible.

#!/usr/bin/luvi --

Unix will append the path to the file which luvi will use at the bundle base.

[creationix]

And of course if you don't like keeping your luvi apps bundled in zips, you can have them as folders somewhere and just point the shebang line to the app.

1. Create luvi app somewhere.
2. Create single-line unix script with shebang and mark executable.

#!/usr/bin/luvi /path/to/app --

This was you can have an unpacked luvit and a global luvit command.

[creationix]

I did originally have a plan where luvit could require directly out of the lit object graph instead of having to have files on disk. This solves the global vs local problem quite nicely. In a git graph, there are never duplicate files stored on disk since the hash of the content is the key in the database. Also assuming there is a sane mapping from dependencies to hashes, multiple versions of the same library can be installed globally at the same time. This is the best solution disk-space-wise as all dependencies of all your apps can be in a single git pack file with all duplicates unified. (You could even technically store similar files as diffs of each-other, but I don't know of any tools for creating such packfiles, only the network protocols in git do this automatically.)

Let me know what you want.

[joerg-krause]

If I understand correctly, requiring from a lit object graph requires a network connection at runtime?

For now, if I use lit install luvit/lit I get lits dependencies installed in a directory named deps. Furthermore, this directory contains the package lit:

$ tree deps
deps
├── coro-channel.lua
├── coro-fs.lua
├── coro-http.lua
├── coro-net.lua
├── coro-spawn.lua
├── coro-split.lua
├── coro-tls.lua
├── coro-wrapper.lua
├── git
│   ├── core.lua
│   ├── db.lua
│   ├── init.lua
│   ├── package.lua
│   └── storage.lua
├── http-codec.lua
├── json.lua
├── lit
│   ├── commands
│   │   ├── add.lua
│   │   ├── auth.lua
│   │   ├── claim.lua
│   │   ├── config.lua
│   │   ├── down.lua
│   │   ├── get-luvi.lua
│   │   ├── help.lua
│   │   ├── init.lua
│   │   ├── install.lua
│   │   ├── ls.lua
│   │   ├── make.lua
│   │   ├── publish.lua
│   │   ├── README
│   │   ├── serve.lua
│   │   ├── share.lua
│   │   ├── sync.lua
│   │   ├── unclaim.lua
│   │   ├── update.lua
│   │   └── up.lua
│   ├── init.lua
│   ├── libs
│   │   ├── api.lua
│   │   ├── autoconfig.lua
│   │   ├── auto-updater.lua
│   │   ├── calculate-deps.lua
│   │   ├── codec.lua
│   │   ├── core.lua
│   │   ├── db-fs.lua
│   │   ├── db.lua
│   │   ├── exec.lua
│   │   ├── export.lua
│   │   ├── export-zip.lua
│   │   ├── get-installed.lua
│   │   ├── github-request.lua
│   │   ├── handlers.lua
│   │   ├── import.lua
│   │   ├── install-deps.lua
│   │   ├── log.lua
│   │   ├── pkg.lua
│   │   ├── rdb.lua
│   │   ├── rules.lua
│   │   ├── verify-signature.lua
│   │   └── vfs.lua
│   ├── main.lua
│   └── package.lua
├── pretty-print.lua
├── prompt.lua
├── readline.lua
├── require.lua
├── semver.lua
├── ssh-rsa.lua
└── websocket-codec.lua

I would like to use something like lit_config_prefix=/home/joerg/buildroot/output/target/usr lit install -g luvit/lit which installs all files from the package lit including its dependencies to /home/joerg/buildroot/output/target/usr/lib/luvi_modules:

$ tree lit
lit
├── commands
│   ├── add.lua
│   ├── auth.lua
│   ├── claim.lua
│   ├── config.lua
│   ├── down.lua
│   ├── get-luvi.lua
│   ├── help.lua
│   ├── init.lua
│   ├── install.lua
│   ├── ls.lua
│   ├── make.lua
│   ├── publish.lua
│   ├── README
│   ├── serve.lua
│   ├── share.lua
│   ├── sync.lua
│   ├── unclaim.lua
│   ├── update.lua
│   └── up.lua
├── deps
│   ├── coro-channel.lua
│   ├── coro-fs.lua
│   ├── coro-http.lua
│   ├── coro-net.lua
│   ├── coro-spawn.lua
│   ├── coro-split.lua
│   ├── coro-tls.lua
│   ├── coro-wrapper.lua
│   ├── git
│   │   ├── core.lua
│   │   ├── db.lua
│   │   ├── init.lua
│   │   ├── package.lua
│   │   └── storage.lua
│   ├── http-codec.lua
│   ├── json.lua
│   ├── pretty-print.lua
│   ├── prompt.lua
│   ├── readline.lua
│   ├── require.lua
│   ├── semver.lua
│   ├── ssh-rsa.lua
│   └── websocket-codec.lua
├── init.lua
├── libs
│   ├── api.lua
│   ├── autoconfig.lua
│   ├── auto-updater.lua
│   ├── calculate-deps.lua
│   ├── codec.lua
│   ├── core.lua
│   ├── db-fs.lua
│   ├── db.lua
│   ├── exec.lua
│   ├── export.lua
│   ├── export-zip.lua
│   ├── get-installed.lua
│   ├── github-request.lua
│   ├── handlers.lua
│   ├── import.lua
│   ├── install-deps.lua
│   ├── log.lua
│   ├── pkg.lua
│   ├── rdb.lua
│   ├── rules.lua
│   ├── verify-signature.lua
│   └── vfs.lua
├── main.lua
└── package.lua

Furthermore, it would be great if the single-line unix script named lit will be installed in $PREFIX/bin.

This way we could easily install lit packages similar to npm packages in Buildroot (and I guess in other package management systems, too).

[joerg-krause]

Closing as it is ancient.