Open CMCDragonkai opened 2 years ago
With regards to pk secrets cat
and "writing a file" to a secret vault. While the cat
command can be used to read out a secret, what's more difficult is putting back a secret that can be composed with other commands. Here I propose a null-command or a variant of pk secrets
that opens a readable file descriptor.
Here are some examples:
# using process redirection, as pk secrets v1:/a would need to take a FD
echo 'somesecret' > >(pk secrets v1:/a)
printf 'somesecret\n' > >(pk secrets v1:/a)
echo 'somesecret' | pk secrets v1:/a
# echo 'somesecret' > pk secrets v1:/a # <- this will not work as pk secrets is a comand, not a filepath
# compare with shell commands like:
echo 'somesecret' > >(cat > abc)
echo 'somesecret' | cat > abc
Note that pk secrets v1:/a
has no subcommand under secrets
. It would need to open a file descriptor by attempting to read from STDIN. This takes over from the shell command cat > abc
which runs cat
without a file path argument which causes it to read from STDIN. However is not usable in PK since we aren't running process redirection operators >
and <
from inside the PK command. That is PK is not a shell. Therefore we run something different:
# starts reading from STDIN immediately
pk secrets v1:/a
As well as the ability to use cat
subcommand to output to STDOUT.
# read v1:/abc and pipe it to cat, pretty simple
pk secrets cat v1:/abc | cat
At this point, one might argue that one should enable pk secrets v1:/a
to be a writable file descriptor as well. That is both STDIN and STDOUT may be usable.
cat <(pk secrets v1:/abc)
Which would take over from the cat
subcommand.
However I'm not sure I really like the usage of pk secrets v1:/abc
as a universal file descriptor. It's nice in one way, but it seems such a "default" command to be relatively hard to use and hard to learn.
Consider that we also need another command that one can just use to put a secret in without the need to wire up other commands.
pk secrets put v1:/abc 'abc'
pk secrets write v1:/abc 'abc'
pk secrets print v1:/abc 'abc'
And as we have talked about before, we need 3 ways:
I propose a special command for this:
# this will read from STDIN, and write to v1:/abc (like reading into a secret)
pk secrets read v1:/abc
# pipe into a file descriptor
echo 'secretdata' | pk secrets read v1:/abc
pk secrets read v1:/abc <<<'secretdata'
echo 'secretdata' > >(pk secrets read v1:/abc)
So then read
is the opposite of cat
.
But still one more command that allows one to write a secret in easily. Here are our 3 candidates.
pk secrets put v1:/abc 'abc'
pk secrets write v1:/abc 'abc'
pk secrets print v1:/abc 'abc'
From these 3, I quite like pk secrets write
. It's the opposite of read, it actually can function very similarly if pk secrets write v1:/abc
was allowed. That is with no arguments, it acquires from the STDIN. In that sense, there's no need for pk secrets read
at all then as it is just confusing. So write
is probably the correct keyword.
Some commands here due to usage over GRPC will involve client streaming or even duplex streaming. Deadlines will be important here. And if a command gets cut in the middle, the data written to EFS should be removed. That might be good idea to per-vault lock to do this. Until of course we can create a "vault"/efs snapshot. The beginnings of a vault/efs snapshot is already done in https://github.com/MatrixAI/js-encryptedfs/issues/49 but it would need to be exposed to the PK vaults system. Perhaps that can help with enabling COW for the vaults MatrixAI/Polykey#172 MatrixAI/Polykey#180.
Now that MatrixAI/Polykey#266 has fixed up vaults related commands, the next step is a proper review of all the secret commands, and hopefully finish up our pk secrets env
PR MatrixAI/Polykey#265 so it can be merged.
@CryptoTotalWar this is relevant for your review of PK CLI on Mac.
Leverage Unix-like command paradigms to streamline the process of creating secrets directly into the vault, bypassing the need to first create a file on the local filesystem. This would enhance the functionality of Polykey by allowing direct input of secrets via standard input (stdin), aligning with Unix principles of simplicity and flexibility.
Instead of the current method where a file must exist beforehand, we could introduce functionality where users can pipe data directly into a Polykey secret path. This approach avoids unnecessary steps and strengthens security by not requiring sensitive data to touch the filesystem.
echo 'your_secret_data' | polykey secrets write vaultName:/path/to/secret
This command would take the output from echo and directly write it to the specified secret path in the vault. It simplifies the workflow, especially in scripting and automation scenarios, where secrets need to be dynamically generated and stored.
I believe this enhancement would significantly improve the user experience by making secret management more seamless and integrated with standard command line operations. It encourages best practices in handling sensitive data.
Specification
Standard unix commands include things like
cp
,mv
andls
.This allows users to interact with secret vaults as if they were real filesystems. (And they sort of are).
Imagine (IN ORDER OF PRIORITY):
I believe some of these commands were already implemented before in the old PK codebase. You have things like
pk secrets create
andpk secrets delete
.I believe there's a problem with doing this. We are reinventing the wheel, and we'll never cover all the commands that Unix already has.
This has the benefit of reusing context that developers already know and remember when interacting with a Unix shell.
But how do we do this without having to rewrite all the code? Luckily it seems someone has already done this.
See: https://github.com/shelljs/shelljs
It has implemented all the major Unix shell commands in raw JS.
The only problem that their command source code currently directly imports the native Node fs:
https://github.com/shelljs/shelljs/blob/79ae14d30d7ce4064de05d41c7889885326b6754/src/ls.js#L2
If we would want to use the
shelljs
library, we would need to globally mock the fs as described here: https://github.com/shelljs/shelljs/issues/747#issuecomment-316301410However that may be dangerous if that leaks into other places of the FS.
There is another alternative: https://github.com/dthree/cash. The difference between the 2 are: https://github.com/dthree/cash#doesnt-shelljs-do-this However I think it's even less integratable compared to
shelljs
.So it seems we would need to "extract" the command code from
shelljs
and place it into PK directly and thus enable us to change thefs
object to our Vault EFS.Note that we do not need all shell commands, just the major ones that relate to files, and doesn't change cwd context since we don't use that. Nothing that changes permissions is relevant to us. Process control is also not relevant.
One major difference is that our commands will have to traverse both vault filesystems and the real filesystem. For example
pk secrets mv vault1:/a ./a
which has to take a file from a vault to the real fs. The functionality to achieve this will also end up being used to do commands between vaults. Because theefs.mv
won't work between EFS instances anyway.Additional context
pk secrets ed
commandpk secrets env
command is similar concept here replicating the Unix commandenv
pk secrets cat
command and composition of PK to other commandsTasks