search

keepassxreboot / keepassxc

KeePassXC is a cross-platform community-driven port of the Windows application “Keepass Password Safe”.
https://keepassxc.org/
Other
21.23k stars 1.47k forks source link

Call okon-cli from keepassxc-cli for fast offline HIBP checks #5447

Closed stryku closed 4 years ago

stryku commented 4 years ago

First of, I'm new here, so I'd like to say hi to everyone \o

Second, I know that I've wiped out the issue template. I did this because it didn't consist of all sections/information that I wanted to share. I believe that below formatting and sections are clear too and hope that you'll forgive me not going with the default issue template. However, if you don't like this, please let me know in a comment and close the issue. I'll open a new one that goes with the issue template.

Abstract

KeePassXC supports passwords checks against HIBP passwords database. Online and offline. That's a really cool feature. As you probably know, HIBP passwords file is huge (>22G). Offline check for a single SHA1 hash takes a couple of minutes. That's not convenient for an end user. In this issue I'd like to propose a library integration that checks a hash in HIBP db very fast - and it does it offline.

okon

Some time ago I've created a library and a CLI tool with exactly one purpose in mind - fast offline checking for hashes in HIBP db. The tool is called okon, you can find it here: https://github.com/stryku/okon.

tl;dr introduction

okon works in two steps.

  1. Prepare downloaded HIBP passwords file. In this step okon creates a prepared file. This step takes a couple of minutes but it's done only once for every HIBP file.
  2. Use the prepared file to perform checks for SHA1 hashes. A single check takes ~3ms. Yes, miliseconds.

So, once you spent a couple of minutes on okon preparation, you can check for any hash in miliseconds.

Key advantages..

.. that I think matters in scope of integration into KeePassXC.

Library

okon provides a library with a simple C interface, so it can be integrated everywhere. You can find the interface and documentation in okon.h header file.

Readme

Much more information can be found in the repo itself. How much time preparation takes, how much time check takes, how okon works under the hood etc.

Showtime

Here's a quick gif that shows integrated okon and password checks while fooling aroung in 'Edit entry' dialog:

gif_for_issue

and here's a full video which shows a little more. Integrated okon preparation and playing with it in 'Edit entry' dialog: https://youtu.be/94U3ogRWlsI

Integration

As you can see I've integrated okon into KeePassXC codebase. Well, actually.. I did not.

More precise words would be that I hacked okon into the KeePassXC. It's more of a POC. It took around 2h and I did it only to create the gif and video. I wanted to give you the impression of how it would look. However, this shows that interation is not hard at all.

Code hacks

Before you go there, please remember that this code is very, very far from production ready. I did this in two hours of a Friday evening. It has two memory leaks, does not care about error handling, has hardcoded stuff etc etc. But, if you really, want to see how KeePassXC uses okon, here it is: https://github.com/stryku/keepassxc/commit/c1242dc4e1aa1eb9b6c0c21c455756e4831bbe42

I can be responsible for integration

As beeing said, I'm keen to do the integration in a proper way, go through the review process, fix issues and so on. The only condition is that you want me to do this.

What needs to be done to integrate

Here's a list off the top of my head that needs to be done:

What do you think

So, the last thing is to ask you what do you think about all that? Questions? Green light? No-go?

droidmonkey commented 4 years ago

Thank you for the discussion, however I have several problems with the proposal and the situation in general:

  1. This is a first-class feature (prominent in the GUI) but has a huge (literally) barrier to entry.
  2. The offline HIBP check is niche, at best, and we did not integrate it into the GUI for that reason. In general it is not a best practice to force users to download an excessively large file to find 1 specific hash entry. Let alone force them to download a huge file, then perform preprocessing of that file, then search for 1 specific hash value.
  3. This is why we integrated the online HIBP check in the reports section.
  4. We do not want to encourage users to make their own passwords that is what the password generator is for and why KeePassXC exists in the first place... you should only be using randomly generated passwords that would never appear in these hash lists.

You should really work with Troy Hunt to have him preprocess the offline hash file. If you can reduce the file by 50% in size then why the hell wouldn't he want to host and serve the post-processed version? As it stands now this is an untenable solution and will only get worse over time as more hashes are added.

With all this said, we will not be integrated offline HIBP checks of any kind into the GUI.

stryku commented 4 years ago

Thanks for your time checking the proposal! I see your points. Just want to clear some things up.

I didn't mean to integrate okon as an obligatory feature. I know that most of the users don't know about HIBP and certainly don't want to download such a huge file. I thought about it as a optional feature for users that really want that and know what all that means.

I didn't mean to force okon usage in any part of the program. You know best where it fits. It could be in 'Edit entry'. It could be in reports. It could be in CLI.

As you probably know from a long discussion in one of the issues, some users don't like online checking. I know that there are users who can live with downloading HIBP file, processing it and storing 11G on their disks, to be able to search for the hashes offline.

Here's why I actually created the proposal:

  1. KeepassXC already performs offline checking.
  2. To do this, users need to download and store 22G file.
  3. They need to wait a couple of minutes every time the check is performed. (Probably) Every time they add a new password to their db.

I've create the proposal because with okon KeepassXC would be able to to the same, but better. Users would store 11G, not 22G. They would wait a couple of minutes only once - for preparation. After that, any check would be performed instantly for any number of times.

I think offline checking is for users who already know what they are doing. If KeepassXC doesn't give them such a feature, they do this on their own, somehow. I just though it'd be a nice feature for this group of users.

Ok, that's all. Thanks again for your time!

droidmonkey commented 4 years ago

We could easily call an installed okon from our cli. Can you provide an example cli call sequence?

stryku commented 4 years ago

Sure, here are examples for preparing and checking: https://github.com/stryku/okon#command-line-interface

phoerious commented 4 years ago

That said, the main reason why we integrated checks into the GUI in the first place is due to popular demand. If you are using your password manager correctly, the only way one of your passwords will ever appear in the list is if the breached service has horrendous security practices by not hashing passwords. And even then the exposure will be limited to that service alone (and you should probably just go ahead and delete your account). A new auto-generated password will practically never appear in the list of breached passwords and thus it's a waste of time to check for it.