search

gopasspw / gopass

The slightly more awesome standard unix password manager for teams
https://www.gopass.pw/
MIT License
5.83k stars 485 forks source link

RFC: Handling breaking changes #1365

Closed dominikschulz closed 3 years ago

dominikschulz commented 4 years ago

TL;DR: How should we handle truly breaking changes? With a major version bump or a fork?

Background

gopass has accumulated a lot of features in the past. Some of them were contributed, some never truly finished. Also supporting troublesome dependencies like GnuPG and git adds to the maintenance burden.

It has become clear that we don't have enough time to support all of these feature and dependencies and we prefer to have a stable, polished core feature set over a huge number of partly broken features.

We had some small breaking changes in the last minor release and we will have more small breaking changes while we continue to clean up the code base leading up to a major breaking change in the next major release.

Upcoming Changes

Going forward we consider to:

This will break most existing setups and will require careful planning and execution.

If you care about any of these features the only (!) way to keep them from being removed are sustained, high quality contributions, e.g. triaging issues, improving documentation, test coverage and finally refactoring the code to make them better maintainable. Unless contributors step up to work on these, the aforementioned features will be dropped. Probably before the end of this year.

Upcoming Changes

While our goal is clear, it's not so clear how we want to get there.

Obviously we want to keep supporting our existing users (but not necessarily on their current stack).

Right now I see two options moving forward.

Both have in common that we will make smaller (possibly) breaking changes in the minor releases leading up to the next major release. We will try to limit the impact but features which are already broken, rarely used or poorly implemented will either be changed or removed.

Also we will work to add support for the storage and crypto backends that will be used in the next major release.

Crypto Backend

In the next major release our age might become our default crypto backend.

I have a very strong preference for age, but this is only possible if the project receives sustained attention and provides answers for some open questions (e.g. Go library support, agent support).

Storage / RCS Backend

While git itself is muss less troublesome to support than GnuPG it still adds unnecessary overhead. If you think about how we use git you'll realize that we don't need most of it's features, e.g. we don't need merging (useless with encrypted files), branches, tags, or actually almost any other feature it has.

What do we really need in terms of storage / revision control?

Since we can't really merge or diff entries the versioning can just be a list ordered by the revision timestamp. Optionally we'd like to be able to hide the names of the keys (#673, #674).

These properties can trivially be implemented in Go.

Implementation cleanup

It's clear that in the next major release we'll reduce support for different backends, reducing the amount of code.

Eventually we'd like to get rid of complicated recipient key handling. If we're using age the recipient list will already include the whole public key with no need to additionally export or import keys to and from huge keyrings. We might even default to using GitHub username as the default mechanism (age can encrypt for ssh public keys which are publicly visible on GitHub).

Options

This section lists possible options how to reach a major reduction in tech debt with the next major release without loosing most of our users on the way.

Hard Fork

This would be the easiest option. We would implement all required changes as detailed above in the 1.9 branch and before we start to actually remove features we'd fork the gopass repo to e.g. github.com/gopasspw/gopass2 and keep both repos around: One for the 1.x branch and the other for the 2.x branch.

Obviously we could also do that in the gopass repo with a different branch, but I think that might be more confusing.

However this approach has two big disadvantages:

  1. We'd still have to maintain the old gopass 1.x repo. Since our time is very limited this will slowly rot up to a point where it's unuseable. For a security critical project this isn't a good idea.
  2. Many users won't make the switch, cf. the Perl5/6 or Python 2/3 situation.

Provide an automated migration path

The second option would be to (try to) automatically migrate all users to the new backends.

This would require us to implement the migration tooling that would automatically migrate the users stores to the new backends in the 1.x branch. If a user misses one of the releases with this migration tooling they would need to resort to the last 1.x release before the can use the 2.x releases.

The big advantage is that this avoid the disadvantages of the first option.

The disadvantage that this is a pretty disruptive change.

Users will need to replace their remote setup (git) with something else (which doesn't exist, yet). Also they will need to trust a different kind of crypto (which is likely much safer, but in some cases it might be perceived as a step back).

Request for Comments

Please let me know what you think. But please keep in mind that this is an open source project run by volunteers.

Feel free to share this proposal with any gopass users you know.

The question here is not if or when we'll make these breaking changes, only how! The only way to prevent a feature from being dropped is to make sustained high quality contributions!

Edit: Updated to better reflect to current plans.

fgarcia commented 4 years ago

I think the gopass --help shows many features that are not important and the features table of the README looks overwhelmingly ambitious (many of which I do not care)

My sympathy for the maintainers because pass and git trade offs hit a sweet spot not found in other programs at the expense of some controversial side effects.

Abandoning GPG sounds so wild and also a huge maintenance burden. I fear for the features I love like: having multiple recipients, an audit trail for my entries, being able to decode using a hardware token on my PC and on on my phone... and foremost, avoiding the tool lock-in because I can always gpg -d my-password.gpg

Even if you go ahead and drop pass compatibility, I think there are two steps due before that decision:

  1. Reduce the feature table of the readme
  2. define-discuss a plugin architecture (extensions)
XenGi commented 4 years ago

By removing GPG and Git support you basically remove the 2 features that made gopass a viable solution in the first place. This move makes absolutely no sense to me. I will probably go with a fork if there is one or fallback to pass and maybe write a small wrapper around it that gives me the 2 or 3 features that gopass has which where the reasons I switched from pass to it. Namely mounts and a better UX.

Sad to see such a good project go away.

nokernel commented 4 years ago

Personally, the main feature for me is OTP, GIT, GPG, Teams, and Multi repos. That is why I am using gopass :).

If I try to sum up my feelings here, removing those features would make gopass remove it's super powers and be like Keepass, so why use a stripped down version of gopass when there is already keepass that does this job?

And sadly I really can't help on the code side.

I agree that keeping OLD version for breaking changes might be a good idea. Depending how it is implemented, would the "old" fork be legacy and stop being improved?

avanier commented 4 years ago

I'll be siding the gentlemen above.

My use-cases really revolve around the smart-cards, in this case Yubikeys. Having a central state-store that allows easy conflict resolution like Git is also essential to me as I'll be pushing and pulling values from different devices asynchronously. OTP is a huge feature as well.

Teams and Multi-repos are features I use daily, but could live with seeing them scaled back for ease of maintenance.

@fgarcia hit it on the nail. Tool agnosticism and the general classic *nix approach is the appeal of this tool. From that perspective, a plugin system makes a lot of sense. It might also be a legitimate way to lessen the maintenance burden of the core of gopass. Focus on a clearly defined API, and provide the bulk of the UX?

On the other hand, GPG is a flaming mess, and age is very nice, modern, and sleek. I've seen your comments on age about agent-support for Yubikeys, but until that feature is reality, age is not a viable alternative for my use-cases. :confused:

addendum: I've been digging into this whole agent-support thing for age and the documentation is... vague. I'll need to try it. It's not clear what it does or doesn't. Also, swapping out GPG for PCKS#11 would be an option, but that's also not a feature yet.

dominikschulz commented 4 years ago

Thank you for the sensible comments to this proposal so far.

I'll try to address each concern separately, but first let me try to clarify.

@fgarcia Don't worry about either multiple recipients, audit trail or vendor lock in. These are not up for discussion. I'd like to have support for hardware tokens as well, but I'm not so sure if that is a strict requirement. Definitely a nice to have. The feature list, documentation and CLI subcommands are the first things that are being cleaned up right now. This will already help a bit.

I don't think I will invest time into a plugin architecture - that doesn't really fly with Go and a security critical tool. But we're definitely planning to offer a good API to allow using gopass as a library.

@XenGi Git is certainly harder to replace than GPG. But I don't get the point about GPG. If you consider forking already, why not invest the time to contribute to improve the current implementation? We have plenty of issues to get started with.

@nokernel Except for GPG and maybe Git none of these are up for discussion. If we decide to fork the old version would likely bit-rot. I wouldn't have much time to do any more than reviewing PRs - maybe not even that.

@avanier I don't think age is ready just yet. But I bet it might be soon. If it doesn't get some kind of agent and/or hardware token (read: yubikey) support, I won't propose it as an GPG replacement.

Does anyone want to comment why they feel GPG is so important? I think I get the point about Git and am trying to find a viable way to keep supporting that. But GPG really feels like a lost cause to me.

fgarcia commented 4 years ago

My 2 cents about why some people feel strongly about GPG

GpgCard owners (like Yubikey) might feel addicted to the safety of a hardware token that makes impossible stealing a password/keyfile, even if the host system is rooted. Not only that, one just has to memorize a pin code while still getting a really strong key.

GPG and X.500 tends to be the only tool available for Devops people to solve secret distribution, key trust, audit access, signing, revoke keys and allow others to encrypt without a password (asymetrical)

For me a big issue is that not all my secrets are in my password store. I do store secrets within different projects. For example, I do encrypt some of my accounting files and want the peace of mind that my backups are not saving confidential files unprotected.

With GPG I have a single encryption solution for every situation, not just password storage. It was a nightmare before trying to track passwords for every zip/file/disk... hell, I do not even know (or need to know) my own GPG password because I just use my gpg usb/card

martinhoefling commented 4 years ago

Just to add some of my thoughts here as well. First of all it is crucial to understand that the project is backed by very few people. In fact, if one omits smaller feature contributions and focuses on community moderation (aka. gh issues) and bugfixes, it's almost exclusively @dominikschulz working on the project.

GPG:

Personally, I was mainly involved in the gopass jsonapi and the tightly coupled gopassbridge browser plugin. Just looking at the bug reports for the browser plugin - numerous were directly related to gpg setups or gpg related issues (like agents + gpg hw token stealing focus). As such, interfacing with gpg takes up most of the maintenance time.

Git:

Regarding git, well - here the main point is, that we're not using many of the core features, like branching, merging etc. - we're using it primarily for storing a tree of K/V pairs.

Plugins / Library:

As @dominikschulz mentioned, plugin like systems contradict in many ways the golang single binary philosophy. However, defining a public API that allows using gopass as a library would open the ways to a more maintainable feature set of 'core gopass'. E.g. Binary Support, HIBP check, Browser API (jsonapi), OTP, YAML support, xkcd password generators and others ... could all be moved out and maintained independently. From my perspective, that would a) lower the maintenance efforts on gopass side b) lower the entrance barrier to contribute / maintain a feature for others

As @dominikschulz said, we're still in the discussion (here) on what key points are important for gopass to keep. For me, it's not a specific technology like git or gpg but a feature set. If the same can be achieved with a more maintainable stack, I am more than happy.

fgarcia commented 4 years ago

These are two examples of how I would imagine an extensible gopass. It would be very easy to offload git and otp related code from the maintainers.

gopass sync 

   call ~/pass-store/_bin/sync (bash script)
        git pull
        git push

gopass otp website/example.com

   call ~/pass-store/_bin/otp (bash script)
       otp-generator $(gopass kv --read 'otp' website.example.com)

Once again, I believe extensions and key-value per entry manipulation are the two big missing value/features

But not just inside plugin, also as an outside service plugin. For inspiration you can imagine poor man Vault implementation. The interface is not just the JsonAPI but the CLI too

XenGi commented 4 years ago

Adding another 2cents.

Git

Seems like the main concern about git is that you don't use enough of it. I don't really understand that. You use exactly what is needed. For me it's a convenience feature so I don't have to put in the git commands myself and gopass does it automatically. I also don't care if it is git or anything else. I would be totally happy with a hook dir under .config/gopass/hooks/ or better yet in the repo, where I can define commands to push and pull my password-store dir. Then everyone could put in their git, svn, dropbox or whatever calls. Implementing this in a gopass app would be a bit harder. You probably need to put in support for every backend that is needed. But only having git here is probably enough because most other sync backends like dropbox or syncthing do their thing automatically anyways.

GPG

I hate gpg like everyone and I don't want to use it anymore. The reason reason I want it in gopass is that it integrates with so much stuff. The main reason is having a pass compatible android app. If gpg support is going away and I think this would be a good thing someone needs to write a gopass compatible android app that works without gpg. I hate android development and I don't have the time to maintain something like that. I would probably fork the current pass app ask a friend to support me with android dev skills and then never touch the app again. This is not an app I would want. So if some android devs could build a gopass app all my concerns would be gone.

The other thing that doesn't concern me but some of my colleagues and friends is the hardware token support. I don't know enough about them but AFAIK they are all bound to gpg. If they can support something else then we could safely remove gpg support and replace it with some sane crypto and all be happy.

dominikschulz commented 4 years ago

@XenGi Thank you. These are all reasonable concerns. We'll try to keep these in mind as we move forward.

To understand my concerns with git (also applies to gpg), please try to understand my situation and goals. I'm maintaining gopass mostly on my own and have very little time besides $work and $family. But I do want to maintain gopass because I like to use it and so do a lot of people I care about. So in order to make this work, I need to reduce the maintenance burder as much as possible. This comes mostly from GnuPG and Git (and some of our past design choices). The problem with git is not that we only use a small part of it, but that the parts we don't need, have to be taken care of as well. Think of workdir vs. index vs. commits and other complications where all we really need is a list of revisions per secret. I'm really undecided about git, let's wait how the alternative experiments turn out.

Hardware tokens are something I'd really like to support (easily). Not necessarily with GPG.

For a mobile app I don't really see the use case (yet). Personally I wouldn't put my secrets on my phone. I'd be curious to better understand your usecases for having gopass on a phone.

marco-m commented 4 years ago

Hello all, I am very well familiar with open source maintainership. As such, the gopass team has all my empathy and support for them to do what they find it best for the project and their emotions.

I prefer a well-maintained limited-scope gopass than a kitchen sink, buggy gopass with burned-out or non-responsive maintainers.

To people reclaiming for "their" features, try remember that this is a free and volunteer-based project. We are not in a position to demand anything.

Minor comment about git: personally I don't have a need for git integration, but in case this can help, you might consider https://github.com/go-git/go-git ?

You have all my ❤️

worldofgeese commented 4 years ago

I use gopass for so much! Our team at MQS uses gopass together with summon to inject secrets as environment variables directly into our containers so there's no issue with secrets laying around in the open. I love that feature and hope it sticks around.

I use gopass in my personal life as this amazing veneer over pass on the command-line when I'm in a shell. I also use Emacs all the time and really appreciate being able to make use of the underlying pass with Emacs packages like ivy-pass and auth-password-store.

I use the Password Store app on my Android device, which integrates with my GPG key I've added in to another Android app, OpenKeychain, that my Android mail app, K-9 Mail, makes use of to encrypt and decrypt email.

GPG might be arcane but it is ubiquitous! I so love and appreciate being able to step down to the base pass package that powers apps on my phone.

dominikschulz commented 4 years ago

Thank you for that feedback.

Integrations like summon are something we definitely want to keep supporting, we're even trying to make it easier and less cumbersome to maintain these by providing a proper gopass API.

Also the cli surface of gopass should remain (mostly) unchanged, so it should continue to work as a drop-in replacement for pass (wrt. cli).

GPG is another story. It's so annoying ... but it is indeed ubiquitous. I don't want to keep it. Maybe we end up doing so, but it's not my favourite option.

XenGi commented 4 years ago

It's hard to believe you don't see the usecase for a password manager on the phone. I have like a hundred apps on mine and all of them have some kind of account or secret that is all stored in gopass. Also I use a mobile browser and I want to login to the same sites I do on the desktop. So a mobile app or some kind of integration is an absolute must for me. Since gopass is compatible with pass this is no problem atm because there is a pass app.

About the maintenance. I totally understand that. I also don't want to put too much time in my opensource projects. I can't imagine how hard it must be, having a family and also doing opensource. What about inviting some more people into the project to split the maintenance work? There are clearly a lot of people wanting to use gopass and it being a pretty technical thing I guess some of them are also go devs with some spare time. I could imagine helping myself. I just have no overview where work is needed atm.

worldofgeese commented 4 years ago

I've been thinking about this over the last few days: the "killer feature" of gopass is that it's built over ubiquitous tools. Once you take that away then it becomes another password manager flavor, of which there are dozens if not hundreds of implementations. In fact, if the intention is to move away from the closest thing we have to standards-compliance in the password manager world, would it not be better to put "more wood behind fewer arrows" like @sircmpwn's himitsu?

That of course wouldn't address the then missing Android and iOS apps.

XenGi commented 4 years ago

That's also why I use pass/gopass. If I just wanted a password manager that is open source and that I can host myself I would have used bitwarden or something similar. The cool thing about pass/gopass is that it is basically gpg+ssh+git+magic. That magic could totally be a simple shell script gluing everything together. Classic *NIX philosophy.

sudoforge commented 4 years ago

To echo what has been said in other comments:

I moved to gopass because it provided nice features on top of the feature list provided by zx2c4/password-store, such as mounts, native OTP support, and nicer Git integration. I used zx2c4/password-store because I want my PGP key used to encrypt and decrypt my secrets, just like I use it to manage authentication subkeys for SSH-based access, and just like I use it to encrypt sensitive content or messages where necessary. It's nice to have one solution for everything, and I think it is clear from these comments that removing support for these tools removes the main reason many people switched to gopass in the first place.

Personally, removing or demoting support for GPG and Git makes this project unviable for me, and based on the responses in this thread, would likely cause a hard fork in the community. I would be extremely happy to help alleviate the maintainence burden here; I will admit that I have had limited time outside of $WORK and $LIFE but will find a way to make it work if this is the only way to keep these features supported as first-class citizens in the project. Let me know if it would make more sense to talk offline about specific pain points and ways you're looking for help -- I don't see many open issues related to either of these features.

Some of the suggestions above that I didn't catch in my initial scanning include moving the support for Git to a general-purpose "pre/post hooks" system -- I think this is completely viable and helps remove the burden of Git integration (although the specific complaint that is talked about confuses me as it appears to be related to the design choice of the backend package rather than Git itself). I'm also not sure offhand why we couldn't use libgit2 or go-git instead of the current backend package.

tmccombs commented 4 years ago

I am not completely opposed to these changes, however, like others, above I use the the Password Store android app. I have things that I need to log in to on my phone, so would like to be able to access my password manager from my phone. Note that my phone is encrpyted, and protected by a password. And that my gpg key is also password protected, and I only unlock it when I need access to my password manager. If there was a decent app that supported whatever crypto gopass used, I'd be good with that. But unfortunately, I have very little experience with android development, and what I do have is probably way out of date. I don't think I could build and maintain such an app myself. I would be willing to help contribute to it, if someone else took the lead though.

Another nice thing about using gpg is that it's easy to have a gpg agent authenticate using your login credentials with pam on linux. One way you could get something similar is if you used the SecretService DBus api (or the mac keyring, not sure if there is an equivalent for windows) to get the private key.

Also, the git integration is nice to have, but I'd be ok, with just using git myself to handle syncing changes.

ckolumbus commented 4 years ago

Hi, with my view on things here I want to show appreciation for the great work and the hope that we can find a way to keep gopass up and alive as the great thing it is !

I use gopass mainly as a summon provider across platforms and to manage secrets within our distributed development teams. My MLF (most lovable features) are the use of standard tools, the distribution & multi-recipient capabilities and the compatibility with pass ecosystem.

Does anyone want to comment why they feel GPG is so important?

The agent support and the existing (GUI) key management solutions. Although awkward, they exist. An ecosystem around age needs to be build first. For some use case I see age as superior to the GPG (as far as I understand), but with dropping gpg completely we loose many tools that help with managing gpg keys.

I think I get the point about Git and am trying to find a viable way to keep supporting that.

Although I love and use git for distributing the secrets I'm not so bound to have this integrated into gopass. the hook-integration would be fully sufficient for me. And it follows more the unix tool concept. So if gopass keeps "the possibility" to use git for distribution it would be OK to remove it.

AnomalRoil commented 4 years ago

Wow, I almost missed this RFC!

I don't have much to add to the debate, but from what I can tell and from what I have touched in Gopass codebase, IMO the big problem with PGP is that it is relying on the GPG CLI app, which is a pain since it's heavily system-dependent. It would be nice to implement #1405 and have a full Go native app (which means replacing Git also with a Go library), but then I'm not sure how the hardware support would work. (From the comment on https://golang.org/src/crypto/rsa/rsa.go#L108 I'd say it might work with hardware modules, but I am yet to find an implementation using it.)

Now, why should we keep PGP?

On my side I would not use Gopass if it didn't support hardware tokens. I know that hardware token support is coming to age through its plugin system and/or through the ssh-agent, and I'm looking forward to it, but it doesn't seem to be quite there yet, although I haven't dug into it too much.

I don't want "yet another keepass" which is basically a black-box handling my passwords, I love the "file-centric", "git-backed", "PGP-compatible" Gopass where my passwords are PGP encrypted files that I can backup using git, Boxcryptor, DropBox, Google Drive, Mega or whatsoever, and decrypt using PGP on any device that has PGP. (Okay, granted, the git history thing becomes a bit useless past a certain date if you're using a PGP key with an expiry date set in 1 year as I do, but still.) (And yes, Keepass has a CLI that can work with dmenu or Rofi, so the CLI argument is not enough to sell me a Gopass without GPG and Git.)

Removing git would be fine for me, as long as we keep a file-centric design.

Now, I do agree that Gopass might profit from a feature removal, or from a clean-up. And cutting PGP and Git would be a good way to refactor and clean-up without it being too painful. But I'm sure we could also go through the whole codebase and find other ways to optimize it, removing some features or simplifying the design without resorting to such extreme measures.

And don't get me wrong: I love age and what they are doing, I hope it will take over and I'd be glad to have both support for GPG and Age within Gopass.

On my side I'd be happy to help more with Gopass, and I will try and look at the issues and contribute more. Or otherwise don't hesitate to tag me on anything encryption related, or easy stuff that you don't have time for :)

dominikschulz commented 4 years ago

Thanks for all the constructive feedback. The goal of this issue was to provide a clear vision of where I'd like to see this project move. Not something that will happen over night.

Of course age isn't ready, yet. Maybe it never will. But it's looking very promising. Promising enough that I'm betting a lot of hours (of my time) on it. Not more, but also not less.

I agree with the hardware token part. It's a must for me as well (but again: with PGP it's just terrible, IMHO).

Also I'm completely sold on the not-a-blackbox part. This is not a commercial product. I don't want to have any lock in effect. People need to be able to access their secrets even without gopass. But that doesn't mean this has to be super easy. However I'm strongly considering to move to a backend that somehow encrypts the secret names, as this can easily leak information (a prototype exists with the ondisk backend).

Currently I'm working on removing as many maintenance burdens as possible (i.e. features which are broken or too toilsome) without breaking (too many) existing use cases. Also I'm continuing to prototype possible implementations for a gopass without either GPG or git as well as designing and implementing a possible migration path.

I'm still very much in favor of dropping support for both GPG and git, but as I've said before: This is not set in stone and I have no idea when this might happen.

Any contributions to clean up the code base, fix or remove broken features or at least document the current features (e.g. how different command line flags should interact) are always very much appreciated.

Airblader commented 4 years ago

For a mobile app I don't really see the use case (yet). Personally I wouldn't put my secrets on my phone. I'd be curious to better understand your usecases for having gopass on a phone.

To somewhat add to what has been said: I don't use a password manager for the two important passwords at work. I can memorize those. I use it so that I can have strong, different passwords for each and every service I am signed up to. The fact that the world these days is more mobile than desktop should be well-known. A password manager which I cannot somehow make work on my phone would be 100% useless to me and an immediate reason to switch. It's already bothersome that with Android Store I can only have a single repository cloned, but one is better than none.

Most coworkers that I know don't synchronize important work passwords on their phone, but they do synchronize the dozens to hundreds of personal ones, be it GitHub, Facebook, Amazon or Aunty Emm's tiny web shop.

Edit 1: +1 to the idea of a plugin architecture. I think this could take over the responsibilities of user-facing git benefits provided by gopass.

Edit 2: It might be worth pinning this issue.

worldofgeese commented 4 years ago

Since what I see as overwhelming opposition to this transition has not swayed the project's leader (evidenced by the prerelease builds), is there someone who is up for maintaining a "classic" fork? I'd have time enough available to triage issues but not much more.

I don't mean this in an aggressive way, only that given no other option, it'd be good to have a smooth offramp for gopass users who favor a git plus gpg approach.

dominikschulz commented 4 years ago

@worldofgeese Did you actually try the PRs? They contain almost no breaking changes. At least nothing that touches the proposals discussed here. I'm still using gopass with gpg and git myself as I did ever since.

I don't see this thread as "overwhelming opposition", but instead as very valueable input, e.g. I still don't understand why you'd want your passwords on your phone, but I see that we need to figure out a way to enable that. My opinion on git has slightly changed. I still think we need to simplify the implementation, but maybe we should keep it. Just because it's so convenient. For gpg on the other hand not much has changed. If age gets proper hardware token support I wouldn't hesitate to drop gpg any moment.

sudoforge commented 4 years ago

I still don't understand why you'd want your passwords on your phone

I want my passwords on my phone because, at times, I need to enter passwords into applications. Reading a 64-128 character password off of one device (e.g. my workstation) and manually typing it in on another (e.g. my phone) is error prone and not feasible.

My opinion on git has slightly changed. I still think we need to simplify the implementation...

I mentioned this in an earlier comment, as did @marco-m -- what challenges do you see in using either of:

Airblader commented 4 years ago

I still don't understand why you'd want your passwords on your phone

Just curious, and I realize you said you see now it should be possible in some way, but why don't you think it's very common for people to log into a lot of different services on their phones? What would you recommend to do instead for these users? Maybe this is just an educational issue, and I'm happy to learn.

My opinion on git has slightly changed. I still think we need to simplify the implementation, but maybe we should keep it.

That is great news, and I appreciate your reflection on thinking about that. From my personal point of view git is just a really nice, convenient backup feature and if much of the git complexity wasn't supported (and left to the user to resolve outside of gopass should it raise issues) this would be totally fine for me as a user (much like a plug-in / hook architecture would be as well).

dominikschulz commented 4 years ago

@sudoforge Unfortunately both of these libararies don't work for us. git2go is a binding to a C library. But we can't use CGO without massively changing the release workflow. I don't think this would work with cross compiling to Windows. Not sure about the others. And I can't maintain a compile farm for building on all target platform. I did try go-git but it doesn't have merge support. And without that no realistic workflow is possible. It just wouldn't work for any team setup.

@Airblader We have discussed the mobile use case and in my personal experience whenever I set up a new phone there are usually very little apps that require me to enter a long password. But of course everyone has a different workflow.

The git implementation will definitely become a lot simpler. I was thinking of a rather brute force approach (e.g. always running git reset --hard HEAD before writing).

AnomalRoil commented 4 years ago

Notice that one of the really nice "feature" of using GPG and git and the same kind of file structure as pass is that it makes Gopass compatible with the "Password Store" app which is itself compatible with the OpenKeychain app to use hardware tokens on Android.

Notice that significantly changing Gopass might also impact other people relying on the somewhat pass retro-compatibility that we currently have. (For instance Password-Store is discussing supporting multiple stores because of Gopass)

Airblader commented 4 years ago

there are usually very little apps

Apps often use sign-in through other services, yes, but there's usually a lot of websites for me to sign into at least.

Thanks for the Slack invitation! I'm currently recovering from a hospital visit but will try to join soon and be happy to discuss this further.

dominikschulz commented 4 years ago

1530 adds a workaround if you rely on the main secret being in the first line of the secret.

It's somewhat of a hack, but it should work for users need this for mobile apps or interop.

sudoforge commented 4 years ago

@Airblader said:

Apps often use sign-in through other services ...

To add to this -- even if an application offers OAuth sign-in, I (and I'm sure others, as well) may not choose to use it. A good example is when an application's OAuth integration requires permissions which I see as excessive, e.g. access to my contacts or other properties of my IdP profile.

Brixy commented 4 years ago

Also from my side many thanks for maintaining gopass.

I am not a crypto expert—so these are my modest 2 cents.

I chose gopass (after using pass for years) because it seems to be very future proof.

Age is excellent (maybe it should/could replace GPG altogether). But we don’t know if that will be around for years to come—although it’s called age ;-) One thing to remember: Private age keys are currently not encrypted by default. Users would have to be very careful.

On the other hand age could be integrated closely into gopass: optionally generate age keys from within gopass, encrypt keys … A ‘dependency-free’ gopass with age in the background would make it easy to convince people using it.

Hosting a git repo anywhere—even on private servers—seems more ‘inclusive’ than relying on a single cloud service (like Dropbox, Sync, Box) for a whole team. This is a great plus over Keepass, Bitwarden etc. Moreover, a git log is a great security plus. If I accidentally delete a secret file I can recover it any time. Some cloud services provide a history that goes back only a few weeks. Rsync et al. don’t provide a history at all.

I do not need a mobile app, but I work with people who use an iPad-only workflow including pass for iOS who don’t know what a terminal is.

dominikschulz commented 4 years ago

Thanks for your feedback @Brixy.

You seem to be right in every regard and I really don't see us killing git support anytime soon. Although I'd love to get rid of the command line dependency somehow.

And before we can get rid of gpg support a lot has to happen. This issue really isn't about something that will change over night. More about the long term plans.

Your ideas about an age integration into gopass sound good. Actually we should have all that already. It's a bit hidden, but you can already use age with gopass as you describe it, i.e. encrypted private keys, dependency free (with the ondisk backend), .... The UX isn't great and the documentation isn't great either, but the code is there.

pbasov commented 4 years ago

I'd like to echo the comments about GPG + OpenKeychain + Ubikey support. It's very convenient and something that I would like to keep using.

I'd have to revert to GNU Pass if you decide to go forward with the implemenation change.

doronbehar commented 4 years ago

I also have a thing or two to say in favor of keep using GnuPG & Git support:

GnuPG

Although we all hate it, please note the fact that some of it's features are irreplaceable. I'm talking about the agent and the pinentry.

The pinentry is configurable from the bottom up: You can choose what pinentry flavor to use with pinentry-program and you can even let it act differently in different environments (using PINENTRY_USER_DATA). Example usages - 1 & 2.

Similarly, the cache timeout being configurable using default-cache-ttl 7200 is also irreplaceable.

Think how wonderful it is that all of these components - the browser, the agent and the gopass CLI play together (if only setup right).

Technically, if you'd implement these features I wouldn't complain. But please consider how tightly GnuPG is audited due to it being widely used. This argument proves the Unix philosophy is relevant once again, and that's the foundation upon the original passwordstore is implemented, righteously.

Git

I'm not sure I understand what's the downside of using Git but not using all of it's features. I guess I do understand that you just want to break free so I'm not left much to say. But once again:

In a way, Git has merging capabilities I doubt you'd be able to replicate. Perhaps not the most sophisticated algorithms are needed for gopass' purposes, but isn't it more then good? It seems to me you are "breaking free" at the price of the whole sync feature.

Perhaps you should consider using a libgit? However, I don't see how that would be less of a maintainence burden.

My personal suggestions

I don't know what will come up at the end of this discussion or at the 2.0.0 release - whether there will be a hard fork etc. But in the meantime, feel free to cc me in any issues related to GnuPG. I consider myself an expert now after this.

dominikschulz commented 4 years ago

@doronbehar Thank you for your feedback. As previously noted, maybe all of this will never happen. The purpose of this issues was to dicuss the "what-if" scenario.

I'm still not fully sold on keeping gpg. It definitely has some very nice advantages and is widely used, but interfacing programmatically is just so annoying and the myriad of configuration options makes it hard to support this. But I agree that we will need (a) some kind of agent support (for credential caching), (b) hardware token support and (c) mobile apps. We had (a) in gopass, but it was very naive/bad/insecure so I removed it. Unless we can implement all of this we won't remove gpg from gopass.

For git I have less technical concerns, but it feels a bit pointless of having to deal with merge conflicts et.al. when all we need are time-based revisions. We don't even need any kind of merging or conflict resolution. If two secrets are modified at exactly the same time we could just keep both revisions. The main reason why we'll likely keep supporting git is the remote end. Setting up and running a binary (or configuring a GCS/S3 storage bucket) is likely a major hurdle for most people. On the other hand most gopass users have an easy way of creating git repos already. So it's quite likely that we'll keep git either way.

I'm definitely happy to CC you on any gpg related issues. All I want is a maintainable project, and if we reach that goal by getting more people involved that'd be great!

doronbehar commented 4 years ago

I'm definitely happy to CC you on any gpg related issues. All I want is a maintainable project, and if we reach that goal by getting more people involved that'd be great!

Great :) I was thinking, for GnuPG's issues, perhaps we should highlight in issues' templates that certain common issues are likely to be caused by a faulty gpg-agent setup. Perhaps a link to Arch Wiki's GnuPG's "Troubleshooting" section or a like would be proper there.

I also have an idea that might improve the UX of gpg-agent's issues: When gopass fails to decrypt a password - when stderr of a gpg --decrypt is:

gpg: decryption failed: No secret key
Decryption failed: exit status 2

It might be nice to tell the user to check their gpg setup and use exclamation marks to indicate that the error is likely due to gpg's setup and not gopass itself.

euh2 commented 4 years ago

I admit, I only read around 40% of this discussion.

My immediate idea would be this: Why not join forces? I use @restic. Restic has a 'git-like' functioning, audited encryption and foremost a stable developer community. I think gopass would be a really cool sibling for that project - offloading git and encryption to restic.

(Personally I think we should see a lot more cooperation between large open-source projects)

Finkregh commented 4 years ago

first: thanks for this great project, the perfect path from just pass

I switched from pass because of gpg-mutiuser-foo and git as various others before. I'd +1 moving everything cumbersome to an external plugin that it is out of your way but still can be used by whomever might depend on it.

Using restic as a backend-plugin sounds like a nice idea @euh2

project0 commented 4 years ago

I totally agree, the code base growth a lot. Some parts are really hard to maintain (I just remember the different URI handling...) and needs at some point aggressive refactor.

@dominikschulz If i understood you correct, maintenance is basically the reason to remove functionality like git and gpg? I see the point to remove those. I am not a fan of calling other CLIs, always feels like workarounds and it can break pretty easy.

Just adding my 2 cent ;-)

Git

Personally i can live without direct git integration. Having some kind of script hook system would be enough for most use cases. Interesting that this feature tends to survive while most people just need an alternative way to trigger their git syncs...

GPG

At first glance loosing gpg encryption feels hard, as the origin of gopass is a handy and feature rich alternative for pass which promised compatibility. I understand that most people have problems with that. GPG is broad supported across systems, well standardized and bullet proofed. I also did a quick lookup on age. Interesting concept, but still in early state. I am not a crypto expert, but i am missing some kind of audit to be proofed to be secure and "support" reliability (which means proper RFC, multiple language integration, etc..).

However, as the discussions around gpg is getting personal and opinionated, i am missing hard facts. Can we have a separate RFC or ADR on that? What do we actually need to support? What is the goal? What are the reasons? Which alternatives exists? etc.. Maybe it turns out there is no real alternative and it needs to keep?

Maybe we can also find a way to support multiple (encryption) backends with some kind of extension/plugin system.

dominikschulz commented 4 years ago

@Project0 Thanks for your feedback.

This discussion has become quite long and sometimes emotional. I might have mislead some users that I want to kill git and gpg. That is not true. I was only trying to provide an outlook on what will happen if we don't get these into shape. age isn't fully ready for our usecase, yet. The point I was trying to make with age was that there are viable options to move away from gpg if we choose to.

We're currently doing some requirements engineering (see the docs/ folder) and try to define our core use cases. This is still very early, but I plan to write down all core use cases (aka critical user journeys) over time. This will include the encryption part. We should have done this before we started writing code, but we didn't. So I intent to fix this in the near term.

Multiple encryption backends, including age, are already supported. But this is another area where we could use better docs.

rbott commented 4 years ago

Since this discussion has mostly focused on (in my view) key features of gopass, namely git and gpg support: what about the rest? @dominikschulz also mentioned dropping parts of the config or unused CLI options. Would it be viable to start a poll with the community to figure out which features are more or less popular (e.g. pass compatibility, mounts, gpg backend, multiple recipients, library/API/3rd party integrations, OTP, rcs/git)? I find it always hard as an open source maintainer to guess which features of "my" software/project are acutally used and which can be safely deprecated. Maybe dropping unpopular features frees some time for widely used features.

In any ways, thanks everyone involved! I really like gopass for its simplicity and ease of use / integration with other standard tools. Kudos for making this possible :-)

dominikschulz commented 4 years ago

@rbott I'd certainly love to get more feedback from our users but this is hard to get right.

I don't want to bother users running gopass and we definitely won't collect telemetry or anything.

We had some good discussions in different github issues already and have adopted accordingly, e.g. I wanted to kill OTP until I was persuaded to keep it. Simliar for git.

If you have any good ideas how to run a good user survey let me know.

doronbehar commented 4 years ago

I use / want / need from gopass:

I don't use:

mordax7 commented 4 years ago

Also use Gopass on my Android device with Git and GPG.

daraul commented 4 years ago

Stealing @doronbehar's template so I can put in my 2 cents:

snuffop commented 4 years ago

Dropping GPG and Git support Drops this project. The point was to make a better more agle version of pass

AnomalRoil commented 4 years ago

@snuffop Yeah, that's what we've been discussing in this "RFC" as in "Request for comments" and the comments made clear that we should not drop GPG and Git support.

insoIite commented 4 years ago

Hello :) Gopass is intensively used either for personal stuff either within my company :) We are mainly using GPG + GIT in order to have several teams sharing the same password store ! (dozens of teams)

Don't know if a random comment can help you take your decision but this are the main features that make us using gopass !

Have a great day, and thanks again for the time you spent on it !

rvolosatovs commented 4 years ago

I would definitely not continue using the project if GPG and git support was dropped. In fact, I am currently intending to rollback to an earlier version due to the new password format introduced. It broke, for example, compatibility with https://github.com/docker/docker-credential-helpers for me, not to mention pretty much all my scripts in general (due to CLI interface changes) - I would not expect such changes without a major bump.