emmeowzing / dynamic-continuation-orb

CircleCI orb for directory-targeted dynamically continued pipelines
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation
BSD 3-Clause "New" or "Revised" License
22 stars 12 forks source link

config validate command shows a warning #88

Open emmeowzing opened 1 year ago

emmeowzing commented 1 year ago

https://app.circleci.com/pipelines/github/emmeowzing/test-repo-2.0/5/workflows/ce575e25-ab88-4bbc-aba8-d994797c746c/jobs/7?invite=true#step-108-0_59

Could not fetch a valid org-id from collaborators endpoint.
Check if you have access to this org by hitting https://circleci.com/api/v2/me/collaborations
Continuing on - private orb resolution will not work as intended
Config file at .circleci/continue-config.yml is valid.
szeitlin commented 1 year ago

I'm getting an error that seems related to this:

> Could not fetch a valid org-id from collaborators endpoint.
Check if you have access to this org by hitting https://circleci.com/api/v2/me/collaborations
Continuing on - private orb resolution will not work as intended
Error: config compilation contains errors: config compilation contains errors:
        - ERROR IN CONFIG FILE:
        - [#/workflows/xxx-workflow] only 1 subschema matches out of 2
        - 1. [#/workflows/xxx-workflow/jobs/1] 0 subschemas matched instead of one
        - |   1. [#/workflows/xxx-workflow/jobs/1] expected type: Mapping, found: Null
        - |   |   SCHEMA:
        - |   |     type: object
        - |   |   INPUT:
        - |   |     null
        - |   2. [#/workflows/xxx-workflow/jobs/1] expected type: String, found: Null
        - |   |   SCHEMA:
        - |   |     type: string
        - |   |   INPUT:
        - |   |     null

Exited with code exit status 255

Here's my (sanitized) config in case that helps, since it's not obvious to me what it's complaining about (circleci config validate run locally succeeds on both files??):

version: 2.1

setup: true

executors:
  docker-executor:
    docker:
      - image: cimg/python:3.8.4
    resource_class: small

orbs:
  dynamic-continuation: bjd2385/dynamic-continuation@3.7.0

workflows:
  on-commit:
    jobs:
      - dynamic-continuation/continue:
          context: circleci
          default-branch: main
          modules: |
            /one
            /two
            /three
            /four

And here's the continuation it's complaining about. I'm assuming this is user error? I added a second workflow but since I'm not sure how it knows what to trigger, I'm not sure if that's the right way to use this orb.

version: 2.1

commands:
  setup-config:
    description: "install credentials and requirements"
    steps:
      - python/install-packages:
          pkg-manager: pip

orbs:
  python: circleci/python@2.1.1

executors:
  docker-executor:
    docker:
      - image: cimg/python:3.8.4
    resource_class: small

jobs:
  xxx-tests:
    executor: docker-executor
    steps:
      - checkout
      - setup-config
      - run: echo $(python --version)
      - run: echo $(pip freeze)
      - run: python test_job.py

  v2-tests:
    executor: docker-executor
    steps:
      - checkout
      - setup-config
      - run: echo $(python --version)
      - run: echo $(pip freeze)
      - run: python v2_job.py

workflows:
  xxx-workflow:
    jobs:
      - xxx-tests
  v2-workflow:
    jobs:
      - v2-tests
emmeowzing commented 1 year ago

@szeitlin I've tested your configs over in this test repo that I've created just now ~

https://github.com/emmeowzing/test

and it looks like I'm able to get the jobs to spawn correctly, i.e. I don't think there are any changes I made to your configs to get it running.

Screenshot from 2023-09-05 13-30-12

I also created a quick PR so the orb would compare changes against the default branch to test ~

https://github.com/emmeowzing/test/pull/4

Question: do you have other configs with similar job names and / or workflow-names in the other configs? A common error I myself have come across is, I'll have cross-pipeline workflows with the same name, so the Reduce-step will only result in one of them showing up in the continuation config.

===

How it works is, by default the orb will have auto-detect enabled, so the modules-block isn't required. So

workflows:
  on-commit:
    jobs:
      - dynamic-continuation/continue:
          context: circleci
          default-branch: main
          modules: |
            /one
            /two
            /three
            /four

is equivalent to

workflows:
  on-commit:
    jobs:
      - dynamic-continuation/continue:
          context: circleci
          default-branch: main

I.e. they're just ignored by default.

On my project, the detected modules on that PR are

INFO: auto-detected the following modules:

one
scripts

WARN: creating default ignore file for ".circleci/one.yml": .circleci/one.ignore
INFO: including "/one" corresponding workflow ".circleci/one.yml"
WARN: creating default ignore file for ".circleci/scripts.yml": .circleci/scripts.ignore

(You'll see this output in the Filter-job ~ https://app.circleci.com/pipelines/github/emmeowzing/test/7/workflows/65869517-ca9c-42bc-908f-a031d0a3b8aa/jobs/12)

And then it'll take those detected configs from the Filter-step and use them in the Reduce-step, so it'll merge all the configs into one and make a continuation call with that. Merging these configs is a little tricky in that, if there are workflows with the same name cross-pipeline (i.e. in two separate files under .circleci/), the orb currently won't error, it will just result in a single workflow or job showing up in the resulting continued pipeline.