Request: Orb usage example (and addJob compatibility)

[chabou]

Is there an existing issue that is already proposing this?

• [X] I have searched the existing issues

Is your feature request related to a problem? Please describe it

First of all, thank you for this awesome project. I was frustrated to write a complex configuration with YAML (monorepo with conditional CI "path"). I was looking for a template engine to write a yaml file. And I found this! Mind blowing.

My configuration currently uses 6 orbs (jobs and command). And I am trying hard to use them with the sdk. The only documentation is your test suite.

And the only way I found to use this orb job is:

const config = new Config()
const queueOrb = new orb.OrbImport('queue', 'eddiewebb', 'queue', 'volatile', '', {
  commands: {},
  jobs: {
    block_workflow: new parameters.CustomParametersList([
      new parameters.CustomParameter('time', 'string'),
      new parameters.CustomParameter('only-on-branch', 'string'),
    ]),
  },
  executors: {},
})

config.importOrb(queueOrb)

const job = new workflow.WorkflowJob(queueOrb.jobs['block_workflow'], {
  time: '15',
  'only-on-branch': 'master',
})

const myWorkflow = new Workflow('build', [job])

config.addWorkflow(myWorkflow)

This is not possible to use this (because of job parameter type):

myWorkflow.addJob(job)

Describe the solution you'd like

It would be awesome to have an example of how to use an Orb (job, command, executor).

And it would be great to use addJob like other jobs. This is not ideal to declare the workflow at the end, and using a temporary (workflow.WorkflowJobAbstract | Job)[] array to store the jobs.

Teachability, documentation, adoption, migration strategy

This feature (documentation and addJob compatibility) will unlock some power usage, needed for the audience that needs this sdk

What is the motivation / use case for changing the behavior?

Using an orb job/command/executor as easier as a regular job/command/executor

[KyleTryon]

Hey @chabou. We will likely be looking to improve this, but here is what we have come up with for now: https://github.com/CircleCI-Public/circleci-config-sdk-ts/wiki/Use-Orbs

[chabou]

Hi @KyleTryon! The documentation is great!

I don't understand why you need to use typing. This is because you are not using a manifest in OrbImport? For me, it works great:

import * as CircleCI from '@circleci/circleci-config-sdk'

export const orbSlack = new CircleCI.orb.OrbImport('slack', 'circleci', 'slack', '4.3.0', '', {
  jobs: {},
  commands: {
    notify: new CircleCI.parameters.CustomParametersList([
      new CircleCI.parameters.CustomParameter('branch_pattern', 'string'),
      new CircleCI.parameters.CustomParameter('event', 'string'),
      new CircleCI.parameters.CustomParameter('template', 'string'),
    ]),
  },
  executors: {},
})

export const slackNotifyFailCommand = new CircleCI.reusable.ReusedCommand(orbSlack.commands['notify'], {
  branch_pattern: 'master',
  event: 'fail',
  template: 'basic_fail_1',
})

orbSlack.commands['notify'] has this type: CircleCI.parameters.CustomParametersList<"string">. It would automatically be inferred as CircleCI.parameters.CustomParametersList<"string" | "boolean"> if one of the 3 parameters was a boolean for example. Isn't it better than CircleCI.parameters.CustomParametersList<CircleCI.types.parameter.literals.CommandParameterLiteral>?

Do you plan to introduce a parseOrbManifest like @circleci/circleci-config-parser?

It would be awesome to have an orb job example.

[KyleTryon]

@chabou Exactly, In that example I was avoiding creating the manifest and instead instantiated the object directly, but both are valid. What we are considering maybe is an npx CLI tool perhaps to automatically fetch/generate the orb manifest and add it to your project as a local dependency. Taking some inspiration from how npx prisma generate works.

Isn't it better than CircleCI.parameters.CustomParametersList?

Yes i do think you are right!

Do you plan to introduce a parseOrbManifest like @circleci/circleci-config-parser?

See above on potentially creating an npx command. Any thoughts?

[chabou]

I am a huge fan of prisma (since 2018). But I think this is more like a npm/yarn dependency usage. I had this idea: Adding a orbDependencies key in the package.json and create a npx circleci orb add/remove command to add/remove the name+(autofetched)version key in this orbDependencies record, and maintain a generated orbs.d.ts (fetch orb, parse it and generate type definition). But maybe this is not the right choice to colocate these orbs dependencies in the package.json.

[Xavientois]

@chabou In case you want to import orbs more easily, I've published a package that allows you to dynamically fetch the OrbImport at runtime.

https://www.npmjs.com/package/@voiceflow/circleci-config-sdk-orb-import

@KyleTryon An npx command would be awesome, imo! One of the limitations with my package is that it only catches errors at runtime rather than at compile-time. Being able to generate type definitions from the orb manifests would be really handy.

[chabou]

@Xavientois thanks! For now I prefer to add them manually with a crafted manifest to have some types. But your work is a good base to do the same with a CLI command instead of doing this at runtime 👍

[KyleTryon]

Getting back from vacation, awesome amazing work @Xavientois! And @chabou I LOVE your idea of using the package.json file. Thank you both, this is a very productive thread <3.

[worace]

@KyleTryon thanks for your recent work on this in https://github.com/CircleCI-Public/circleci-config-sdk-ts/commit/d6f56bacad745364651bd4bae8c22a89ab9c3ea9. That helped a lot to understand how to re-use dynamic commands from an orb.

I'm still struggling to understand how to use a Parameterized Job from an orb for the same type divergence reasons that @chabou mentioned in his original post. Is such a thing possible via the TS SDK?

We have a lot of cases in our pipeline where we use the build-and-push-image job from the AWS ECR Orb: https://circleci.com/developer/orbs/orb/circleci/aws-ecr#jobs-build-and-push-image

so in our Yaml config this looks like:

workflows:
  myworkflow:
    jobs:
      - aws-ecr/build-and-push-image:
          dockerfile: docker/app/Dockerfile
          foo: bar/etc

I haven't found an example of how to reproduce something like this in the TS SDK yet. Luckily in this case that orb also exports a command that does the same thing as that job, so I guess I can rewrite my workflow to use that command instead of the job. But would be great to understand what the best equivalence between this portion of the Yaml config and the TS sdk is.