search

sahib / brig

File synchronization on top of ipfs with git like interface & web based UI
https://brig.readthedocs.io
GNU Affero General Public License v3.0
567 stars 33 forks source link

Experience report by m-a-t #71

Open m-a-t opened 3 years ago

m-a-t commented 3 years ago

I invested a few hours evaluating brig for my usecase. Here is my experience report. I am happy to provide more details or to open issues for specific topics, if there is need for it.

Was it easy to get »brig« running?

Installation was easy for me because it was packaged in the nix package manager. I already had ipfs-desktop because conducting experiments with different ipfs tools.

I encountered an error on my first computer, when trying out the gateway browser interface. The upload button didn't work. This was the first occasion I gave up. Later I tried it on another computer, where it worked. Maybe this was due to an ad blocker, but I am not sure.

Was it easy to understand its concepts?

My first impression was: It's complicated, but seems well thought through. If possible, I would like to find a tool with simpler concepts. But so far I didn't find such a tool.

What is your intended usecase for it? Could you make it work?

I am looking for p2p/distributed way to share and store my files. It must allow me to use storage from my own server and not force to buy a subscription. So far I was quite happy with syncthing, but what I would like in addition is:

I have not come to a final decision yet, but so far brig seems to be the closest I found for my use case.

I did not test the FUSE part, didn't work on Mac out of the box, but I don't need it right now.

If no, what's missing in your opinion to make the usecase possible?

The biggest missing point for me is that the cool thing from ipfs, a stable link which identifies the file content as itself, is not used for sharing a file. brig offers a link like http://localhost:6001/get/README.md. However, I would like to provide a link which identifies the file exactly (like a ipfs hash), which even remains available if the brig repo contains a newer version of the file. For example http://localhost:6001/get/<hash>

Anything else that you feel free to share.

I really like the way how repos can be added as remotes with the fingerprint, this is what I found missing in git-annex.

The web GUI is a nice piece of work. I had problems with login after starting a second repo with gateway on the same computer (different port), but after configuring "brig cfg set gateway.cert.redirect.enabled false" the problems were gone, not sure whether this was the reason.

My gut feeling, without trying to understand it further, is that using an own version control could be problematic and hinder widespread adoption. I like the attempt to make a simpler version control than git. However, couldn't git itself be used for all the metadata, readable even without brig installed, and brig just implements a simpler interface how to modify this git repo? (This is was I found elegant in git-annex, it uses git for storing its metadata, instead of inventing some other stuff)

Simplification proposal: If the brig command is run while current working directory is inside a brig repository, this automatically could choose this repository.

Enhancement proposal: Allow publishing a file without encryption, such that it can be retrieved in ipfs without using the brig gateway. My personal preference would be an unencrypted repo, where the share link contains the ipfs hash.

Enhancement proposal: It is a little bit too easy to lose data. A way to ensure that n pins of a file still exist would be nice. I think you already have this on the agenda.

Enhancement proposal: Allow uploading a whole folder, not only files.

evgmik commented 3 years ago

Dear @m-a-t thanks a lot for sharing your experience.

I like your suggestions and some of them are already discussed though not yet planned for incoming version. @sahib will weight on this more, but we will work on web gateway in future versions. Our current focus is to squash bugs, with speed ups when it possible.

By the way, fuse layer probably would not work on Mac. The fuse module https://bazil.org/fuse seems to depreciate Mac support.

Simplification proposal: If the brig command is run while current working directory is inside a brig repository, this automatically could choose this repository.

I am not sure what do you mean by above, could you please elaborate.

Summary of enhancements with partial overlap of current enhancements

sahib commented 3 years ago

Cool, thank you @m-a-t!

My first impression was: It's complicated, but seems well thought through. If possible, I would like to find a tool with simpler concepts. But so far I didn't find such a tool.

While I'm happy that it seems thought through, I'm of course more interested in what seemed complicated? And what would you strip away from brig to make it a simpler tool?

I am looking for p2p/distributed way to share and store my files. It must allow me to use storage from my own server and not force to buy a subscription. So far I was quite happy with syncthing, but what I would like in addition is:

Possibility to share files with the public / users which do not have the software installed Use ipfs or another content-addressable file system Stronger version control Upload for users who do not have the software installed I have not come to a final decision yet, but so far brig seems to be the closest I found for my use case.

Yes, those are actually the selling points of brig over syncthing. It's just that syncthing really works well, while brig is still work-in-progress and IPFS performance is not convincing enough yet. But we'll get there slowly... hopefully.

I did not test the FUSE part, didn't work on Mac out of the box, but I don't need it right now.

As @evgmik mentioned, we never really tried it on macOS. We have to rely on community efforts to get proper support there. I have the same opinion as the FUSE author here, that I won't invest time in supporting platforms that make my life harder.

The biggest missing point for me is that the cool thing from ipfs, a stable link which identifies the file content as itself, is not used for sharing a file. brig offers a link like http://localhost:6001/get/README.md However, I would like to provide a link which identifies the file exactly (like a ipfs hash), which even remains available if the brig repo contains a newer version of the file. For example http://localhost:6001/get/<hash>

Good news then: I want to support something like <path>@<commit> syntax in all commands that handles paths (basically what git does). We could also do the <hash> variant. If I remember right there was preliminary support for it, but never got very far. But that would be a rather easy feature to add. We will consider it probably for version 0.7.0.

I really like the way how repos can be added as remotes with the fingerprint, this is what I found missing in git-annex.

Yes, it hides a lot of the complexity of reaching by delegating that to IPFS. Drawback is that debugging it is a bit harder (or less used to), when IPFS doesn't figure out how to find the repo.

The web GUI is a nice piece of work. I had problems with login after starting a second repo with gateway on the same computer (different port), but after configuring "brig cfg set gateway.cert.redirect.enabled false" the problems were gone, not sure whether this was the reason.

My gut feeling, without trying to understand it further, is that using an own version control could be problematic and hinder widespread adoption. I like the attempt to make a simpler version control than git. However, couldn't git itself be used for all the metadata, readable even without brig installed, and brig just implements a simpler interface how to modify this git repo? (This is was I found elegant in git-annex, it uses git for storing its metadata, instead of inventing some other stuff)

The version control system in brig is similar to and inspired by git, but not the same. There are no branches and the myriad of things that git supports. I probably could have implemented the current concept by using git, but that would have pulled in a dependency that I would have used 10% of. Also then we wouldn't have a way to track moved or removed files and would be stuck with sha1 for now.

And although git is not an exactly a good example of usability, I see your point. Users often want something they already know and having the version control as separate layer would make debugging easier. But brig is a file tracker, while git is a content tracker, which demands different concepts. Branching for example makes no real sense for us, but is at the heart of git. Pulling all that complexity in is what made git-annex so powerful, but also a bit awkward and complicated to use.

Simplification proposal: If the brig command is run while current working directory is inside a brig repository, this automatically could choose this repository.

Hmm, that used to work at some point. Will fix.

Enhancement proposal: Allow publishing a file without encryption, such that it can be retrieved in ipfs without using the brig gateway. My personal preference would be an unencrypted repo, where the share link contains the ipfs hash.

Yes, we have a ticket and implementation idea here: https://github.com/sahib/brig/issues/48

Enhancement proposal: It is a little bit too easy to lose data. A way to ensure that n pins of a file still exist would be nice. I think you already have this on the agenda.

Yes, that's already on the roadmap, but not feasible at the moment. Maybe 0.7.0 or beyond.

Enhancement proposal: Allow uploading a whole folder, not only files.

That already supported on the command-line (brig stage <dir>) but I guess you're referring to the gateway UI. Should be doable.

To add on the summary list of @evgmik:

m-a-t commented 3 years ago

Thanks for your responses and sorry for the long delay.

While I'm happy that it seems thought through, I'm of course more interested in what seemed complicated? And what would you strip away from brig to make it a simpler tool?

My concern is that the setup procedure will keep many potential users away which are not that technical. I do not propose to remove features and I do not have a simple path to success here. However, there should be a dead-simple way to run a brig node. (Maybe even from a web browser, without even installing something!) On the longterm, success of a tool depends on popularity, number of users.

Instead of asking what should be removed, lets approach the problem from the other end: What is the absolute minimum that a user needs to do such he/she can share files as a brig node?

Generally speaking, for every configuration degree of freedom, set sensible defaults if possible.

Example use case: Sharing a repo with a non-technical user, without usage of a gateway. Try to minimize the steps required for the non-technical user, such he/she can access files of the repo in the webbrowser.