[FEATURE] Unable to fetch build phases when using Reserved Instance Fleets

[a13zen]

Describe the feature When configuring the codeseeder Codebuild project to use a CodeBuild Fleet with Reserved Capacity, seedfarmer fails looking up the phases. It could be because of a change in behaviour of the boto3 API between On-Demand and Reserved Capacity jobs.

To Reproduce Steps to reproduce the behavior:

1. AWS Console > Developer Tools > Codebuild > Compute Fleets
2. Configure a New Compute Fleet with:
   • Capacity 1 / Linux / Overflow: On-Demand
3. Wait for Capacity to be 1/1
4. AWS Console > Developer Tools > Codebuild > Build Projects > Codeseeder project
5. Edit Project
6. Change Provisioning model to Reserved Capacity
7. Choose your fleet
8. [Optional] Select custom image. But Ubuntu Standard should be sufficient
9. Trigger seedfarmer apply
10. Module deployment kicks off (and succeeds)
11. seedfarmer fails with error below

[2024-06-11 17:18:18,245 | INFO | _stack_commands.py:365 | Deploy-0-eks-module ] Delaying module eks-module deployment to allow IAM Roles and Policies to take effect
[2024-06-11 17:18:29,887 | INFO | codeseeder.py:224 | Deploy-0-eks-module ] Seedkit Configuration Complete
[2024-06-11 17:18:30,082 | INFO | codeseeder.py:303 | Deploy-0-eks-module ] Beginning Remote Execution: seedfarmer.commands._module_commands:_execute_module_commands
...
Traceback (most recent call last):
  File "/.pyenv/versions/3.8.11/bin/seedfarmer", line 8, in <module>
    sys.exit(main())
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/__main__.py", line 295, in main
    cli()
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/__main__.py", line 151, in apply
    commands.apply(
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/commands/_deployment_commands.py", line 700, in apply
    deploy_deployment(
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/commands/_deployment_commands.py", line 559, in deploy_deployment
    _deploy_validated_deployment(
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/commands/_deployment_commands.py", line 241, in _deploy_validated_deployment
    deploy_response = list(workers.map(_exec_deploy, mdos))
  File "/.pyenv/versions/3.8.11/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
    yield fs.pop().result()
  File "/.pyenv/versions/3.8.11/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/.pyenv/versions/3.8.11/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/.pyenv/versions/3.8.11/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/commands/_deployment_commands.py", line 229, in _exec_deploy
    return _execute_deploy(mdo)
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/commands/_deployment_commands.py", line 136, in _execute_deploy
    return commands.deploy_module(mdo)
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/commands/_module_commands.py", line 154, in deploy_module
    resp_dict_str, dict_metadata = _execute_module_commands(
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/seedfarmer/commands/_module_commands.py", line 380, in _execute_module_commands
    _execute_module_commands(
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/aws_codeseeder/codeseeder.py", line 391, in wrapper
    build_info = _remote.run(
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/aws_codeseeder/_remote.py", line 109, in run
    build_info = _execute_codebuild(
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/aws_codeseeder/_remote.py", line 84, in _execute_codebuild
    return _wait_execution(
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/aws_codeseeder/_remote.py", line 46, in _wait_execution
    for status in codebuild.wait(build_id=build_id, session=session):
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/aws_codeseeder/services/codebuild.py", line 252, in wait
    build = fetch_build_info(build_id=build_id, session=session)
  File "/.pyenv/versions/3.8.11/lib/python3.8/site-packages/aws_codeseeder/services/codebuild.py", line 217, in fetch_build_info
    for p in build["phases"]
KeyError: 'phases'
make: *** [deploy] Error 1

Expected behaviour:

• Seedfarmer should work using reserved or on-demand capacity provider

[dgraeber]

Thanks for this. Currently, support for a fleet-managed compute resource in CodeBuild is not supported, but we will look at adding support. I have changed this request to a Feature.

This support will have to be added to AWS CodeSeeder - but we can leave this request here.

[srinivasreddych]

I was able to reproduce the error. I agree w/ Derek, this is a feature that needs to be added to aws codeseeder.

When i created a dedicated compute fleet and integrated with codebuild project, the build launched by codeseeder fails.

[dgraeber]

See https://github.com/awslabs/aws-codeseeder/issues/193