does not update secrets

[chiefjester]

Hi, Thanks for this tool. 👋🏻

So I was testing this out and here are the couple of bugs I've noticed.

1. If there's a secret already set on a particular repo, it doesn't update the secret. You have to delete the secret first and re-add it, to get an updated secret.
2. If you add a space in the secret dont tell mum the space becomes \n.

Here's a sample error (I tried curling the secret to my PC just to verify if the data is successfully decrypted). The first letter before space gets passed, but succeeding words 'tell' and 'mum' gets appended with \n.

[mpdude]

Great to hear you tried it!

Regarding the first issue, could you try to switch to this commit? (If you have checked out a submodule as described in the README, you might need to cd into the bin directory, do a git pull, then cd .. and git commit?)

[mpdude]

To debug 2), do you still have the secret/private key for the config file at hand?

If so, could you please – from your config repo – run SPREAD_PRIVATE_KEY=the-private-key-goes-here bin/debug?

Make sure it's either not a production secret, or that the command does not end up in your shell's history ⚠️ .

[chiefjester]

Hi, @mpdude thanks for a quick reply. I think the link is broken though. Is that commit the latest?

for number 2, I'll just add a /debug? That's so cool if it is 🙌🏻

[chiefjester]

okay for number 1, switching to that particular commit via git checkout commit does solve the issue, it's now updating.

For number 2, When I try to run debug it does show me the correct values without \n

FOO: 'foo bar baz'

I think I've figured it out for no. 2, I just need to quote my secrets

      --data "${{ secrets.FOO }}"

[mpdude]

I've released a 0.1.1 and updated the README with it.

So we're done here?

[mpdude]

I meant this commit:

https://github.com/webfactory/secret-spreader/commit/5f2ee07382f4dbaf8f0c457600ff644c5f9aab61

... which also gives the reason for the change.

[chiefjester]

hey sorry for the late reply @mpdude, confirmed, switching to that commit solves the issue. Is there documentation pointing for the rate limits of the secrets API?

As of right now, if you have 500 repos to update with secrets, this tool would update those 500 repos? Or is it checking against a value that actually changed in your secrets-manager repo? (eg. if a file was changed in the repo, then it'll update that secret)?

[mpdude]

Right now, it will update all of them. I am not aware of any documentation regarding rate limits.

The challenge with getting selective updates right is that we'd need to track on the per-repo/per-secret level when a particular value was assigned. That, in turn, means we cannot have a simple config file that you update manually, but that we‘d need to have tooling (additional commands) to configure (add/change/delete) assigned secrets.

I‘m not saying that isn’t going to happen, but right now I’d like to avoid that added complexity. Also, personally I don’t that that use case right now.

[chiefjester]

@mpdude, what if we maintain a hash table for all the secrets in the spread-secrets file? That way you just check against it, if there's a discrepancy, update that secret, if not, then just skip?

[mpdude]

I‘m sorry, I do not really understand what this hashtable would look like. Could you explain a bit more?

[chiefjester]

@mpdude so say we have a hash.json file that contains all the secrets in that file like this, the key would be the file name and the value would be md5 hash:

{
  "gcf-secret": "d9457d420278a2b7e8d68bee9a58d54e",
  "something-secret": "7f16d4345751d062fe97c39b2706c5c0"
}

then in GitHub actions, it will just check all the files and generate md5 hash, if it's the same as the hash table, then we can update the secret and the hash table (hash.json). If it's not, it's noop/ skips the update for secrets.

[mpdude]

That means we‘d need to update (and commit) the file when the action runs?

[chiefjester]

@mpdude yes, it's actually a good thing, because you'll know when secrets are updated by simply checking which commits we're done by Github Actions.

[chiefjester]

@mpdude I just want to give you more feedback while I used this tool. This just bit me, I had a couple of CI builds failed, I thought it was a fluke, but I confirm, this tool deletes the secrets you added manually. So in order to use this effectively, you should have all the secrets managed by this tool.

The deletion is intermittent, sometimes it just leaves the manually set secrets, sometimes it clears it. 🤷🏻‍♂️

[mpdude]

Are you running it manually or from a config repository and an action as described in the README?

If the latter is the case, you should have logs for the action run where it deleted the secrets.

Could you post those?

[mpdude]

Oh, and to avoid misunderstandings: The secrets that you set manually are separate ones, so their names are not listed in the config file, right?

[chiefjester]

Are you running it manually or from a config repository and an action as described in the README?

@mpdude I'm running from a config repository. I'm not seeing a log that's deleting a secret; I never did use the ~ as described in the README. What I'm trying to say is suppose I have this in config repository.

owner/repo:
   FOO: foo
   BAR: bar

In the GUI of owner/repo, I've added a secret called BAZ. When I ran the config repository, it deletes or causes BAZ to get removed. The deletion seems to be intermittent. While I don't have logs that the config repository deletes the secret from one of my repositories. I do have records of that repo, that suddenly failed, because of a missing SECRET. Like all builds pass, and suddenly failed because a SECRET was missing (got deleted).

This deletion might be a bug in Github internally, but I assure you it happened multiple times now, I can confirm that the SECRETS gets deleted if it's not on the list of the config repository.

Oh, and to avoid misunderstandings: The secrets that you set manually are separate ones, so their names are not listed in the config file, right?

Exactly!

[mpdude]

Please see this:

https://github.com/webfactory/secret-spreader/blob/master/spread#L30-L31

We should never delete a secret without at least printing the information that we do so.

So, if the secret has really been deleted (is it gone in the repo settings page?) and the action run that updated the secrets did not print that it deleted it, this might be a bug in the API. The API is still beta, but I cannot really imagine how that could happen. Either way, you might need to contact GH support then and figure out if they can trace what happened.

[chiefjester]

So, if the secret has really been deleted (is it gone in the repo settings page?)

Yes, it just vanished, and this secret is not even listed on the config repository. At first I thought as this was a fluke, user error, but it happened a couple of times, I'm convinced it's a bug.

P.S. I've contacted Github Support. I hope this gets resolved.

@mpdude have you considered the hashmap solution I proposed? I have a bunch of repositories using this config manager, not sure if having multiple calls to the secrets API is triggering the deletion of SECRETS.

[mpdude]

At least for now, I’d like to avoid having to commit status changes from within the action and push those back to the repo. Things are much more simple if we can keep things "flowing" in one direction only.

If API rate limits become an issue anytime soon, having a command to update the config and keep a timestamp for all changes seems to be the way to go.

[mpdude]

Do you have any new information/idea regarding deleted secrets?

[chiefjester]

hey, @mpdude I haven't received any feedback yet, I'll follow up today.

[chiefjester]

@mpdude So update with GitHub support, they haven't found any entry of deletion either. I've sent some screenshots to them, I'll let you know how it goes.

[chiefjester]

Hey, @mpdude latest update from the Github Actions team, seems like all the deletion came from UI and not from an API call. It seems to be pointing to User error (in my case ~ me).

[mpdude]

Ok, that’s good news!

Is there anything left here for now, or can we close this for the time being?

[chiefjester]

@mpdude I guess we can close this for now, and just reopen it if we hit the API limit. I have about 20+ repos using this, and I have yet to hit any rate limit.

Are you using this internally while there's no org level secrets support yet?

[mpdude]

Yes, I am.