Open dcastro opened 2 years ago
This should definitely be done in core, so that there are effects describing this abstract "feature", which are than implemented by each backend. We can do it in two ways imo:
readSecretMany :: [Path] -> [Entry]
a very complicated type thing, which will probably utilize something similar to depedent types, so that one can construct [ readSecret ..., listSecrets ...]
and [writeSecret ..., deleteSecret ...]
but not [ writeSecret ..., readSecret ..., ]
. This would make for an API that's correct at the type level, but I'm not sure how to do it in Haskell (or in Agda for that matter, but I'm at least sure it can be done)
good idea bad idea? I'd love to implement option 2, I just have no clue how
Regarding (1):
This should definitely be done in core, so that there are effects describing this abstract "feature", which are than implemented by each backend
So, we have (will have) two backends, vault
and pass
.
I don't think it makes sense to generalize ReadSecret
into ReadSecretMany
because neither vault
nor pass
can retrieve more than one entry at once, right?
As far as I can tell, the same applies to WriteSecret
, ListSecrets
and DeleteSecret
.
If that's the case, then I think it makes sense to do this at a higher-level. I.e. in the implementation of the copy command in Command.hs
, call WriteSecret
multiple times concurrently (I think polysemy's sequenceConcurrently
would be handy here).
Regarding (2): I'm not sure I understood what you had in mind. Can you please explain it another way?
So what is the final task? To rewrite commands in Command.hs
using parallel requests? I also didn't understand what @MagicRB meant about dependent types, but if I understood correctly it is not needed, if we parallelize things in Command.hs
and not on lower level of readSecrets
.
@sancho20021 yeah let's try solving this at the Command.hs
level, seems like the simplest most straightforward solution, possibly using sequenceConcurrently
to parallelize where appropriate. I haven't tried it, but from a cursory glance, looks like it should work.
Most commands need to retrieve all entries descending from either the root directory or some other directory. When we have hundreds of entries stored in vault, this will mean doing hundreds of requests (1 per directory + 1 per entry).
Unfortunately, Vault's API does not support batch requests.
At the moment, these requests are done sequentially. We'd see a massive speedup if they were run concurrently.
Commands where this could be applied:
view
/find
: read directories/entries concurrentlyrename
/copy
: read / write entriesrename
/delete --recursive
: delete entries