💬 RFC: Support for `SecretCollection` in the Scaffolder

[benjdlambert]

🔖 Need

In order to collect secrets from a user to run in a job, you have the useTemplateSecrets hook which enables you to set secrets inside CustomFieldExtensions. For GITHUB_TOKENs we've added the ability to configure the FieldExtension through the use of the requestUserCredentials options like so:

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: v1beta3-demo
  title: Test Action template
  description: scaffolder v1beta3 template demo
spec:
  owner: backstage/techdocs-core
  type: service

  parameters:
    ...

    - title: Choose a location
      required:
        - repoUrl
      properties:
        repoUrl:
          title: Repository Location
          type: string
          ui:field: RepoUrlPicker
          ui:options:
            # Here's the option you can pass to the RepoUrlPicker
            requestUserCredentials:
              secretsKey: USER_OAUTH_TOKEN
              additionalScopes:
                github:
                  - workflow
            allowedHosts:
              - github.com
    ...

  steps:
    ...

    - id: publish
      name: Publish
      action: publish:github
      input:
        allowedHosts: ['github.com']
        description: This is ${{ parameters.name }}
        repoUrl: ${{ parameters.repoUrl }}
        # here's where the secret can be used
        token: ${{ secrets.USER_OAUTH_TOKEN }}

    ...

But let's say that I don't have the need for a RepoPicker. How do I get the current GITHUB_TOKEN from the user.

The current solution

The way that we suggest people do this right now is by creating their own FieldExtensions and then having a field in the JSONSchema that doesn't actually provide any value to the template in order to collect that secret and then set it using useTemplateSecrets.

This is a bit of a hack, and pretty clunky because you also need to hide the input field from the the JSONSchema, and it actually really isn't even a field extension at all at that point.

🎉 Proposal

I'm suggesting an API that we can pass in functions that can be deployed with the frontend which run on the client.

Not sure if it makes sense to make these components or not yet, that just always get magically rendered in the Form or not, thinking that you could provide some feedback to the user that isn't presented when calling the implementation of the API.

Proposal

When writing up the Scaffolder plugin in App.tsx you could have the ability to pass in hooks that are going to be called when the TemplateEntity has been loaded in the client. This will allow you to use existing hooks today, to be able to grab information from the user on load, and update the secrets context.

// define the hook
const useGithubAuth = createScaffolderFormHook({ id: 'githubAuth', hook: ({ template, input }) => {
  const githubAuth = useApi(githubAuthApiRef);
  const { setSecret } = useTemplateSecrets();

  // can do something with the template variable too if needed?

  return useAsync(async () => {
     const { token } = await githubAuth.getAccessToken();
     setSecret(input.secretKey, token);
  })
}});

// pass into the scaffolder plugin
<ScaffolderPlugin formHooks={[useGithubAuth]}>
    <ScaffolderFieldExtensions>
     ...
    </ScaffolderFieldExtensions>
</ScaffolderPlugin>

And then you can apply which hooks to collect in the template.yaml

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: v1beta3-demo
  title: Test Action template
  description: scaffolder v1beta3 template demo
spec:
  owner: backstage/techdocs-core
  type: service

  parameters:
    ...

    - title: Choose a location
    ...

  formHooks:
    - id: githubAuth
      input: 
         secretKey: GITHUB_TOKEN
  steps:
    ...

    - id: publish
      name: Publish
      action: publish:github
      input:
        allowedHosts: ['github.com']
        description: This is ${{ parameters.name }}
        repoUrl: ${{ parameters.repoUrl }}
        # here's where the secret can be used
        token: ${{ secrets.GITHUB_TOKEN }}
    ...

〽️ Alternatives

There's a few different approaches that we could use that I think both have positives and negatives.

hooks vs pure async

A problem I can see with the hook approach, is that the useAsync thing is a little awkward, as you wouldn't be able to use async/await inside the main hook function just like React components. But if we were to enable that, we would need some way of being able to get access to the apis that are registered in Backstage, and the way you can do that today in the validators is accessing the apiHolder directly as something that's passed in:

const setGithubAuth = createScaffolderFormHook({
  id: 'githubAuth',
  hook: ({ template, input, apiHolder }) => {
    const githubAuth = await apiHolder.get(githubAuthApiRef);
    const scaffolderApi = await apiHolder.get(scaffolderApiRef);
    // we also don't have any scaffolder api to interact with the secrets, but something like this:

    await scaffolderApi.setSecret({
      key: input.secretKey,
      value: await githubAuth.getAccessToken(),
    });
  },
});

There's also another option here, which maybe would be good to move over the validator functions to, is that we follow a pattern similar to the apiFactories of both the frontend and backend and use our own DI framework to do it. I came up with this approach just writing this doc, so I'm still stewing on this, but quite like it!

const setGithubAuth = createScaffolderFormHook({
  id: 'githubAuth',
  deps: { githubAuth: githubAuthApiRef, scaffolderApi: scaffolderApiRef },
  fn: ({ template, input}, { githubAuth, scaffolderApi }) => {
    await scaffolderApi.setSecret({
      key: input.secretKey,
      value: await githubAuth.getAccessToken(),
    });
  },
});

Extension code or no?

One other thing that popped up would be how we get these things into the ScaffolderPlugin. We have the <ScaffolderFieldExtension> wrapper component, and the extensions are passed in as JSX elements. I thought about doing that here but think that it leads to confusion if they're hooks or not. Not sure:

// pass into the scaffolder plugin
<ScaffolderPlugin formHooks={[useGithubAuth]}>
    <ScaffolderFieldExtensions>
     ...
    </ScaffolderFieldExtensions>
    <ScaffolderFormHooks>
      <UseGithubAuth />
    <ScaffolderFormHooks>
</ScaffolderPlugin>

❌ Risks

No response

👀 Have you spent some time to check if this RFC has been raised before?

• [X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• [X] I have read the Code of Conduct

[UsainBloot]

Thanks for raising this RFC. Personally I am in favour of the pattern similar to apiFactories. Therefore with this approach, they should be passed in as an array to a hooks prop belonging to <ScaffolderPlugin />, rather than a JSX element that doesn't render any visible components

[Rugvip]

@UsainBloot +1, I do prefer the option of implementing this pattern though Utility APIs. It feels like the best tool for the job.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.