search

windmill-labs / windmill

Open-source developer platform to turn scripts into workflows and UIs. Fastest workflow engine (5x vs Airflow). Open-source alternative to Airplane and Retool.
https://windmill.dev
Other
9.21k stars 410 forks source link

bug: Unclear error message on empty Secret restored from Git Backup #2828

Open johnnycube opened 7 months ago

johnnycube commented 7 months ago

Describe the bug

Hi,

I just recently had to move my selfhosted instances between clusters and since I use git to backup my workplaces anyway, I used it to re-instantiate the workspaces in the new cluster.

However, actions and flows using secrets did not function properly. A rather confusing error message ist sent: 

ExecutionErr: ExitCode: 1, last log lines:
write cached binary: /tmp/windmill/cache/gobin/8fffe58e7f86d3fe7a5de1786cd848c22f49eef43b2967f6c259b3762dfa5282
Internal: BlockModeError

The problem was an empty variable of type "secret" that wasnt synced correclty.

To reproduce

  1. Backup a workspace to git using the provided scripts from the demo repo: https://github.com/windmill-labs/windmill-sync-example
  2. The workspace has to include a variable of type "secret"
  3. Initialize a new workspace and sync the backup to the workspace using the provided demo repo: https://github.com/windmill-labs/windmill-sync-example
  4. Execute an action/flow reading a variable of type "secret"
  5. See error (attached as image)

Expected behavior

The secret is empty which is expected - however - the error message is quite unexpected. Tracking down the error I tried to understand the rust implementation that references writing of cached files etc.

I would have expected some message highlighting that the secret was empty/not usable. Maybe I have missed something in the documentation - the frontend and worker - however - are pretty unclear on the exact problem.

Screenshots

image

Browser information

Firefox 120.0.1 (64-bit)

Application version

CE v1.224.0

Additional Context

No response

rubenfiszel commented 7 months ago

Noted on the feedback we will try to wrap better that error.

What happens is that for this secret to actually be a secret, it need to be encrypted with a key that's not easily guessable. Otherwise it would be sufficient for someone to import your secret into their instance to decrypt them, defeating the purpose of secrets.

To that end, secrets are encrypted by a random key that is workspace specific. To move secret from one instance/workspace to another, you need to copy the workspace key to the target workspace (in table workspace_settings) or you need to pull your secrets in plain and then push them in plain. The CLI has the option to do so for both push and pull (see --help on either command)

johnnycube commented 7 months ago

That explains why a google search lands with AES Rust handling - makes perfect sense that the keys are the reason and I thought that it has to be something like this. But without the knowledge of the inner workings of storing the secrets I just recreated the secrets in the new instance and the problem was solved (after some hours of head scratching ;) )

Thank you for your prompt feedback.

In order to make it easer for the end-user (e.g. me) I would have expected some error or info during the import that the secret cannot be properly decrypted. Also the same message during an action/flow run would help.

Thanks for the great product and the quick feedback