Register multiple Jarfiles to merge and lock as a unit

[pangloss]

This allows me to define a library that depends on other libraries that also use lock_jar without needing clients to use any tools to prebuild a Jarfile.lock or manually combine the dependent libraries' Jarfiles beforehand.

Instead, each Jarfile can be registered using LockJar.register_jarfile(jarfile_path). Then all of the registered Jarfiles can be merged by calling LockJar.lock_registered_jarfiles. Finally, LockJar.load can be called as usual.

[mguymon]

@pangloss This looks great, I like how you DRY'd up the common method arguments as well. Could you update the lock_jar_spec to include coverage for .register_jarfile and .lock_registered_jarfiles?

I am going to ponder if the @@registered_jarfiles should be tracked in the LockJar::Registry, but that will not block your PR.

[pangloss]

Ok, I've added some test coverage.

I didn't notice the Registry, but that's probably a good idea.

Also, I've now tried this out in my own project and realized there is one slight snag, which I worked around: I need to be able to register the jarfiles very early, before loading lock_jar or any other dependencies. I'm actually registering them in the project/lib/version.rb file which by convention has no dependencies and is loaded by the gemspec. I do this because I need to be able to have the jar fully locked and loaded before I require the first library, which needs its jars to be loaded at boot time.

You can see what I did here https://github.com/pangloss/pacer/commit/fb27e41fb8423a7a7f3c791bd25de44ff901526b and here where I register a Jarfile to be loaded: https://github.com/pangloss/pacer-neo4j/commit/f421479f26dbf6e207e87214bdda0d6e88c143ff

[mguymon]

Ah, wiring it into the version, that is clever.

I tinkered around with auto loading Jarfile.lock from gems, but avoided having to resolve dependencies every time. If having to lock the merged Jarfile is a time sink, you could MD5 the merged Jarfiles and store a local cache the Jarfile.lock. Then you only need to lock if the MD5 changes.

[mguymon]

released with the 0.10.1 gem

[pangloss]

My initial attempt, registering the Jarfile in version.rb did not work. I've spent a lot of time over the last few days trying to make this process (loading multiple gems, each with their own jar dependencies) better, and have finally figured out something that works pretty well.

My goals are simple:

1) the gem works correctly if I just gem install pacer and then irb -r pacer 2) my bundled project automatically resolves all present Jarfiles correctly and automatically 3) projects that I don't control will also participate in both cases

After digging around I've discovered a few relevant things:

1) there is a Bundler module that only exists in a project if it is running in a bundler environment. 2) if not running under Bundler, all installed gems are present under Gem::Specification, but in a bundler environment, only gem specifications included in the bundle are loaded. 3) Gem specifications know the root directory for the gem on the file system, so I can search them for Jarfiles.

Given that, I do the following:

• If Bundler is not present, just lock pacer's Jarfile in isolation.
• Otherwise, if Bundler exists, Search all gem specifications for Jarflies, register them all and then use lock_registered_jarfiles. Then disable LockJar.lock and LockJar.load methods so that when other libraries are loaded, they do not resolve their dependencies in isolation.

Right now this functionality is in Pacer and is a bit hacky because it monkeypatches LockJar, but I think it could be brought into LockJar itself and would be extremely useful.

Here is the pull in Pacer where I made the changes: https://github.com/pangloss/pacer/pull/61. A library I use together with Pacer is pacer-neo4j. As you can see here https://github.com/pangloss/pacer-neo4j/pull/13, which you can see no longer needs anything special to participate in project-wide jar resolution.

[mguymon]

So I have spent a bunch of time mucking around with these ideas previously. I hacked at bundler some with https://github.com/mguymon/lock_jar_bundler in an attempt to generate a Jarfile.lock when bundler generates the Gemfile.lock. This removes the need to have the runtime figure out what to do with Jarfiles.

It is a shame the version.rb didnt work.

So it sounds like something alongs the lines of LockJar.start(freeze=true) is needed. This would scan, lock, and load all available Jarfiles with the option to freeze any additional lock n load. I think it makes sense to cache the resulting Jarfile.lock instead having to rebuilt it every time LockJar.start is called but the jar dependencies are the same.