feature request: attach custom tags to the ec2

[pharindoko]

Hey @kichik,

I need the possiblity to attach custom labels to an ec2. Background: I want to be able to track the costs in cost explorer based on certain labels.

1. I imagine to have an optional step in the stepfunction in between to match labels for certain runners based on the data provided. Basically a lambda that I can provide to return back labels in a certain format.
2. These labels will be attached additionally to the new created ec2.

br,

@pharindoko

[kichik]

Do you mean tags?

[pharindoko]

Do you mean tags?

Yes I mean tags btw 😄

[kichik]

I believe it should already be copying the provider tags to the EC2 instance. So you should be able to use cdk.Tags.of(provider).add('tagName', 'tagValue'). Does that not work? Or do you need it even more dynamic than that?

[pharindoko]

I believe it should already be copying the provider tags to the EC2 instance. So you should be able to use cdk.Tags.of(provider).add('tagName', 'tagValue'). Does that not work? Or do you need it even more dynamic than that?

I would need it more dynamic. Having a lambda function or some typescript function in between, that I can overwrite, would be nice. I need to do some mapping and match values.

[kichik]

What data would this Lambda need? Requested labels? The entire webhook?

[pharindoko]

What data would this Lambda need? Requested labels? The entire webhook?

The entire webhook to get things like org, repo, job run, step and the labels.

[kichik]

I'll have to think about the best way to enable such a feature request.

For now you can use the environment variables already supplied by the actions runner code. GITHUB_REPOSITORY, GITHUB_RUN_ID, etc. are already there. Check out the integration test. It runs export to show them all. You can then give your instances permission to tag themselves. If you want to be real fancy, you can have a runner hook automatically take care of it. One of the discussion threads has a usage example.

[pharindoko]

Will do. Thanks for the hint @kichik.

[pharindoko]

I'll have to think about the best way to enable such a feature request.

For now you can use the environment variables already supplied by the actions runner code. GITHUB_REPOSITORY, GITHUB_RUN_ID, etc. are already there. Check out the integration test. It runs export to show them all. You can then give your instances permission to tag themselves. If you want to be real fancy, you can have a runner hook automatically take care of it. One of the discussion threads has a usage example.

I had some time to really understand the solution you propose. Yes this definitely will work but I fear that this could be hard to debug, monitor and to modify.

Could it be another option to provide some kind of pre-deployment step in the stepfunction that can be overwritten to select and add tags that can be attached e.g. to the ec2 or other components ?

[kichik]

I still don't have a good idea of what something like that would look like. For example, one of the big problems is that it can't be generic. Settings tags is only supported natively on EC2 and through some hacks on ECS (RunTask doesn't have an option for tags).

Maybe some sort of an escape hatch is in place. Even that I'm not sure how to do elegantly as task objects don't have any option to override until they're rendered and then you have to deal with the entire step function JSON. We would have to take out all of the parameters into an object, pass it to a custom lambda, and then use the results as input for the actual step. But then when this is not used, we just skip the pass and directly pass the parameters?

[pharindoko]

I still don't have a good idea of what something like that would look like. For example, one of the big problems is that it can't be generic. Settings tags is only supported natively on EC2 and through some hacks on ECS (RunTask doesn't have an option for tags).

Maybe some sort of an escape hatch is in place. Even that I'm not sure how to do elegantly as task objects don't have any option to override until they're rendered and then you have to deal with the entire step function JSON. We would have to take out all of the parameters into an object, pass it to a custom lambda, and then use the results as input for the actual step. But then when this is not used, we just skip the pass and directly pass the parameters?

1. Hmm got it. Yes so to do it in a generic manner doesn`t make sense.

But to do it specifically for the ec2 provider could be an option. e.g. here: https://github.com/CloudSnorkel/cdk-github-runners/blob/f64d5a4c0f23abbe98cfc4dc6532c107b33d56f6/src/providers/ec2.ts#L415

• The lambda for the step function in that case has to be injected as Lambda.function.

  
  /**
  * Determine custom tags to assign to new runners.
  *
  * @default null
  */
  readonly customTagsFunction?: lambda.Function;

}

- optional step will be added

```typescript
getStepFunctionTask(parameters: RunnerRuntimeParameters): stepfunctions.IChainable {
    // we need to build user data in two steps because passing the template as the first parameter to stepfunctions.JsonPath.format fails on syntax

    const params = [
      stepfunctions.JsonPath.taskToken,
      this.logGroup.logGroupName,
      parameters.runnerNamePath,
      parameters.githubDomainPath,
      parameters.ownerPath,
      parameters.repoPath,
      parameters.runnerTokenPath,
      this.labels.join(','),
      parameters.registrationUrl,
    ];

    if (this.customTagsFunction) {
      const customTags = new stepfunctions_tasks.LambdaInvoke(
        this,
        'Get custom tags',
        {
          lambdaFunction: this.customTagsFunction,
          payloadResponseOnly: true,
          resultPath: '$.tags',
          payload: stepfunctions.TaskInput.fromObject({
            runnerName: stepfunctions.JsonPath.stringAt('$$.Execution.Name'),
            owner: stepfunctions.JsonPath.stringAt('$.owner'),
            repo: stepfunctions.JsonPath.stringAt('$.repo'),
            installationId: stepfunctions.JsonPath.numberAt('$.installationId'),
            error: stepfunctions.JsonPath.objectAt('$.error'),
          }),
        },
      );
    };

    const passUserData = new stepfunctions.Pass(this, `${this.labels.join(', ')} data`, {
      parameters: {
        userdataTemplate: this.ami.os.is(Os.WINDOWS) ? windowsUserDataTemplate : linuxUserDataTemplate,
      },
      resultPath: stepfunctions.JsonPath.stringAt('$.ec2'),
    });

[kichik]

It just occurred to me that assigning the tags before a job is assigned to the runner may result in unexpected results. Especially if you're using organization level registration. There is no guarantee the runner will pick up the job you think it will if the labels are shared.

If the labels are not shared, each provider can use its own static tags. This won't work right now as the tags come from the launch template which is generated by the image builder. I think that part might be resolved by fleets (#518). My current plan is to move EC2 provider to using fleets with SSM document. It would require the provider to create its own launch template. Once it's the provider creating the launch template, using cdk.Tags.of(provider).add(...) would work as expected.

I still can't get behind a function specifically for tags specifically for EC2 provider. If we could think of a way to make a generic escape hatch where you have a way to edit the step function after being generated, I think that would make more sense. You can technically already do it by overriding the entire JSON. But that's not a realistic method as you'd have to recreate everything.

[pharindoko]

I haven`t thought about this - but auto tagging should help me ... https://medium.com/@mithun.kadyada/lambda-function-for-auto-tagging-ec2-instances-cff38c30bdc8

One thing to mention: it would be nice if you could add organization and repository statically to each ec2 machine created by the ec2-provider. Is this possible ? @kichik

[kichik]

How would the auto-tagging Lambda have access to the webhook info needed to properly tag the instance?

it would be nice if you could add organization and repository statically to each ec2 machine created by the ec2-provider. Is this possible ?

Not sure what you mean by that. Are you asking to have access to the repository or organization where the instance was registered? As in some tag that the Lambda function could later read?

[pharindoko]

How would the auto-tagging Lambda have access to the webhook info needed to properly tag the instance?

it would be nice if you could add organization and repository statically to each ec2 machine created by the ec2-provider. Is this possible ?

Not sure what you mean by that. Are you asking to have access to the repository or organization where the instance was registered? As in some tag that the Lambda function could later read?

Yes I would like to have a simple way to read out the relation between the runner and the github org + repo when the runner is started.

[kichik]

Might this information be available in the state machine log?

[pharindoko]

Yes that's true but hard to query for each machine I would assume.

I would try to get the information when a new instance is started. And I assume it's easier to read the information from the ec2's instance tags.