search

whyrusleeping / gx

A package management tool
MIT License
1.88k stars 111 forks source link

Basic lockfile support #206

Closed travisperson closed 5 years ago

travisperson commented 5 years ago

This is basic support for a lock-file concept discussed between @Stebalien and @whyrusleeping this week.

See also: https://github.com/whyrusleeping/gx-go/pull/49

whyrusleeping commented 5 years ago

Added a version field here. Let's bikeshed a wee bit more, and then just pick something that works well enough for now.

whyrusleeping commented 5 years ago

Alright, now lock-install fetches all the dependencies to their gx paths inside the cache directory (which is currently just $PWD/vendor) and then symlinks everything into place. The files written into the cache directory are set read only. This works to build both ipfs and filecoin now!

travisperson commented 5 years ago

I updated this branch to use the recursive structure and implemented a rather crude worker pattern that can probably be updated a bit. It was the only way I could see doing it that wouldn't run into lock contention (though that might be able to be solved with extending the workers). Both the package install and lock install might be able to share some code if we supply a postFetch callback method as pointed out by @schomatis. I may look into this a bit further.

However, a few things immediately become apparent as I started to work on this code a bit.

  1. Making gx packages read-only breaks some hooks as they write a file to keep track I'm noting this because it was breaking some tests. However, this might not actually be an issue, and instead is just an artifact of not cleaning up tests. I believe only the post-install hook does this, which may not be an issue.

  2. Recursive / transient deps While we can keep track of the entire dependency tree in the lock-file, it currently has to be flattened entirely if we want to keep gx deps entirely read-only, as we wouldn't be able to use the dependencies "vendor" directory to store it's own / conflicting dependencies.

    In the go case I don't think this is an issue. However, if we want to support more languages we'd need to handle this case. This might be as simple as allowing the "vendor" directory to be writable. Doing this though breaks some ideas down the road, such as building directly from /ipfs / /ipns.

    Regardless I think this definitely shows that the language extensions of gx need to be more in the loop with regard to how the lock file is installed.

schomatis commented 5 years ago
  1. Making gx packages read-only breaks some hooks as they write a file to keep track However, this might not actually be an issue, and instead is just an artifact of not cleaning up tests.

Yes, gx drops a .gx/post-install file to mark the package as installed: https://github.com/whyrusleeping/gx/issues/208.

whyrusleeping commented 5 years ago

@Stebalien Are we happy with the lockfile format? I'd like to get things moving forward here.

If the lockfile format looks good, we can change the particulars of install (and lockfile generation) later.

Stebalien commented 5 years ago

I am. Let's merge this.

whyrusleeping commented 5 years ago

Thanks a bunch @travisperson ! This is exciting :)