Closed aryanjassal closed 3 weeks ago
I'm not sure how to handle checking if the file exists or not.
My first idea was to add a check to make an empty file if it doesn't already exist in all RPC handlers in Polykey, but that would break things and the behaviour would feel strange.
My second idea was to use the new ls
command on Polykey CLI to see if the file exists, as on UNIX, if you ls
a file, it would return only that file. However, simplicity and speed was prioritised, and this feature wasn't included in the current PR for the secrets ls
command, so this can't be used either.
My third idea was to make an RPC handler on Polykey which checks if a file exists or not, but that seems like a lot of work for doing something seemingly simple.
All these ideas have their pros and cons, and I'm not sure which one to follow, or if I'm missing an easier solution.
Part of the process is getting the file data to edit. You have two options here.
ENOENT
error and work from there.ENOENT
error for the most part. Permission related errors would be possible but shouldn't happen for our usage since we don't use the file permissions in the vaults.Stat the file first. If the file doesn't exist then you'd get an ENOENT error and work from there.
I thought error-driven workflow wasn't ideal as it takes a lot of resources to throw an error.
Perhaps alternatively, we can return something like null
or undefined
if the file doesn't exist at the time of vaultsSecretsGet
? Or maybe extend its functionality to include a parameter which would create the file if it doesn't exist?
Usually yeah, but for FS related stuff it's just how the API works. So in this case we just have to deal with it.
I know it's a contradiction of what I said earlier. But we can't really force our best practice on a library that's following an API pattern based on exit codes in a terminal.
You should copy over the editor detection and error if not found spec to the spec above, it's core to this issue, it shouldn't be just additional context.
Also some links for later cross platform testing:
Since they haven't been updated, it's likely the existing user accounts on those platforms is still the case. I have to update access for @brynblack.
How can we automate orchestration of the Mac and Windows machines? That's something to figure out later - new issue needed @brynblack. But for now, manual access for @brynblack acting as the Orchestrator is sufficient.
@brynblack I noticed that there's no mention of remmina for the 2 platforms. They are required for it.
@brynblack https://gitlab.com/MatrixAI/Employees/matrix-orientation/-/issues/37 - this is missing from the docs.
Related cross platform testing for this: https://github.com/MatrixAI/Epsilon-Base/issues/3.
The behaviour of secrets edit
command shouldn't be what is simplest. It should be what makes the most sense.
For example, instead of making a blank secrets file if it doesn't exist, ignore the part where the secret contents are fetched instead. Point the editor, say, vim
to the temporary file, but don't make the file yet. If the editor exists without making the file, the action was cancelled. Do nothing. No need to make a file in the vault. If, on the other hand, a file which wasn't there before was created, then the user wants to make a new file with that contents. Then, make a new file with the contents that the user just entered. Otherwise, it should behave as it already has been.
The file should be in the temp directory under a random subdirectory. For example, a file I'm editing could look like /tmp/somedir-1234/file
. The file name could be a constant, that doesn't matter. The directory it is under should be randomised. This appears to be the current default behaviour, so no changes are required for this.
Finally, consider concurrency. An agent can run multiple RPC commands concurrently. What happens when a file is edited at the same time another file is being edited? What if the same file is being edited in two separate agent commands? This would need to look into PCC and OCC for file locking. However, this has already been implemented in Polykey side, where it only allows a single operation to modify a file at a time.
Implement the changes to how the file creation is handled, and verify that the implemented behaviour for file randomisation and file locking works.
The behaviour of
secrets edit
command shouldn't be what is simplest. It should be what makes the most sense.For example, instead of making a blank secrets file if it doesn't exist, ignore the part where the secret contents are fetched instead. Point the editor, say,
vim
to the temporary file, but don't make the file yet. If the editor exists without making the file, the action was cancelled. Do nothing. No need to make a file in the vault. If, on the other hand, a file which wasn't there before was created, then the user wants to make a new file with that contents. Then, make a new file with the contents that the user just entered. Otherwise, it should behave as it already has been.The file should be in the temp directory under a random subdirectory. For example, a file I'm editing could look like
/tmp/somedir-1234/file
. The file name could be a constant, that doesn't matter. The directory it is under should be randomised. This appears to be the current default behaviour, so no changes are required for this.Finally, consider concurrency. An agent can run multiple RPC commands concurrently. What happens when a file is edited at the same time another file is being edited? What if the same file is being edited in two separate agent commands? This would need to look into PCC and OCC for file locking. However, this has already been implemented in Polykey side, where it only allows a single operation to modify a file at a time.
Implement the changes to how the file creation is handled, and verify that the implemented behaviour for file randomisation and file locking works.
That should be part of the spec.
I have implemented the creation of a file if it doesn't already exist in the vault. The remaining tasks for this issue will be tracked in https://github.com/MatrixAI/Polykey-CLI/issues/277 (ENG-405).
You should copy over the editor detection and error if not found spec to the spec above, it's core to this issue, it shouldn't be just additional context.
Also some links for later cross platform testing:
- https://gitlab.com/MatrixAI/Engineering/Infrastructure/Platforms/matrix-mac-1/-/issues
- https://gitlab.com/MatrixAI/Engineering/Infrastructure/Platforms/intel-nuc7i7bnh/-/issues/
- https://graph.matrix.ai/infrastructure/epsilon/epsilon-dc/intel-nuc7i7bnh
- https://graph.matrix.ai/infrastructure/epsilon/epsilon-dc/matrix-mac-1
Since they haven't been updated, it's likely the existing user accounts on those platforms is still the case. I have to update access for @brynblack.
How can we automate orchestration of the Mac and Windows machines? That's something to figure out later - new issue needed @brynblack. But for now, manual access for @brynblack acting as the Orchestrator is sufficient.
Has the cross platform testing been done?
PROBABLY not! Because you need access to the mac and windows machines, and I haven't delegated due to lack of store-and-forward convenience. That's for manual sanity checks.
Specification
If a secret doesn't exist when the
edit
command is invoked, then currently, the command just fails. Instead, the user needs to runpolykey secrets create
to copy a file from their file system and make that a secret, then they are able to usepolykey secrets edit
command. This is not great for UI/UX.Moreover, each platform has their own preferences for file editor. As such, identify and launch the corresponding file editor to edit the secret. Windows would have
notepad.exe
, and MacOS and Linux editors would require to be inferred from environment. To ensure everything is working smoothly, integration tests need to be done on all the given platforms.The behaviour of
secrets edit
command shouldn't be what is simplest. It should be what makes the most sense.For example, instead of making a blank secrets file if it doesn't exist, ignore the part where the secret contents are fetched instead. Point the editor, say,
vim
to the temporary file, but don't make the file yet. If the editor exists without making the file, the action was cancelled. Do nothing. No need to make a file in the vault. If, on the other hand, a file which wasn't there before was created, then the user wants to make a new file with that contents. Then, make a new file with the contents that the user just entered. Otherwise, it should behave as it already has been.The file should be in the temp directory under a random subdirectory. For example, a file I'm editing could look like
/tmp/somedir-1234/file
. The file name could be a constant, that doesn't matter. The directory it is under should be randomised. This appears to be the current default behaviour, so no changes are required for this.Finally, consider concurrency. An agent can run multiple RPC commands concurrently. What happens when a file is edited at the same time another file is being edited? What if the same file is being edited in two separate agent commands? This would need to look into PCC and OCC for file locking. However, this has already been implemented in Polykey side, where it only allows a single operation to modify a file at a time.
The implementation should consider the changes to how the file creation is handled, and verify that the implemented behaviour for file randomisation and file locking is functional as intended.
Additional context
Tasks