Closed tegefaulkes closed 2 weeks ago
The reference discovery stage has been completed. I implemented it as AsyncGenerator
s which made it stream-able but also very clean logic wise. I opted for generators over webstreams
for this stage since composing streams together is a lot cleaner with generators than streams.
Generators are a higher level construct indeed. I think streams represent a lower level construct, and also streams are actually synchronous. Whereas generators represent coroutines in the javascript runtime. So if you're dealing at the object-level then async generator abstraction is better abstraction.
In the future, you should write up your own branch name.
New types and new classes need to be documented into the spec. Like this:
type X = ...
type Y = ...
Give a reason as why they exist now.
Expected to take the whole cycle.
This is a bug with the vault log - pull system. Needs to be fast forward merge.
Progress update: I've converted to using generators and swapped over to using them in the code. The test I've been using as a sanity check is passing as well. I'm going to go back through the code and clean it up a little now.
isometric-git
actually provides a lot of the functionality we needed in the git domain. I was actually able to replace most of the utils.ts
functions with it. I did some cursory checks and the things run just as fast, the whole git domain has been simplified immensely.
It seems odd that isometic-git
wasn't used to handle interacting with the git structure in the first place. Was the git domain created before we started using isometric-git
?
Originally isogit did not have server-side functionality. Only client side. You were able to replace it all? I thought there was git server functionality that was missing. Remember in the future we may want to expose an http API.
It is missing the server side functionality, but it does provide a lot of plumbing functionality for interacting with the git structure that the utils.ts
was replicating.
For example, Listing all the existing refs, resolving them to objectIds, then finding all of the dependent objectIds, and finally creating the packfile format that gets sent over can all be done with isogit. The only thing I had to do was generate the HTTP responses in the proper format.
Then yes it was a quick and dirty job.
Originally we even wanted to use libgit2 but it was easily adapted to work on a in-memory virtual filesystem. Too tied into posix fs semantics.
Mostly done now. Just need to fix some tests in the vaults domain now. Along with doing the benchmarks.
I think the benchmarks can be moved to a 2nd stage PR. I also think the whole vaults domain needs a general review and touch up, also in a separate PR.
Tests are fixed, I added a end-to-end integration tests of of cloning and pulling in the http.test.ts
tests. This does a git clone/pull using just the new http
code directly and the normal file system. So no efs
or networking. It should be a good basis of comparison between adding efs
and networking separately in the benchmarks.
I'm going to look into the clone/pull bug mentioned from before and then move on the merge prep. I'll start a new PR for adding the benchmarks.
1 test relating to discovery and 2 tests relating to MDNS are consistently failing. I'm not sure why and they're wait out of scope of this work so it shouldn't be affected.
This is ready for review. I'm going to start the benchmarks in a new PR for now.
I added some additional review comments with a first glance. Also this review comment previously should be addressed: https://github.com/MatrixAI/Polykey/pull/709#pullrequestreview-2039845891.
Otherwise is there specific things you want reviewed?
The final checklist should be ticked off.
Nothing specific needs to be reviewed, just need the extra pair of eyes for the due diligence.
I'm leaving the checklist along for when the review changes have been addressed.
Done and ready to merge. I'm going to rebase the related PRs before merging this.
Description
This PR addresses refactoring the
git
domain to be more clear and more in line with our latest practices.Issues Fixed
Tasks
http.ts
file.utils.ts
.Final checklist