search

openSUSE / open-build-service

Build and distribute Linux packages from sources in an automatic, consistent and reproducible way #obs
https://openbuildservice.org
GNU General Public License v2.0
936 stars 440 forks source link

Inform users about wrong YAML indentation #12199

Open ashaduri opened 2 years ago

ashaduri commented 2 years ago

Issue Description

I followed the manual on using OBS as a CI and tried using on my gsmartcontrol repository:

GitHub repository: https://github.com/ashaduri/gsmartcontrol OBS repository: https://build.opensuse.org/project/show/home:alex_sh:gsmartcontrol:github_ci:main

The GitHub webhook is set up to trigger on push and pull_request. The response from OBS is 500 Internal Server Error, with some HTML contents saying that OBS crashed.

This is my `.obs/workflows.yml` file contents (click to open/close) ```YAML # This file describes OBS workflows: # https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.scm_ci_workflow_integration.html # Workflow for pull requests #build_pull_request_workflow: # steps: # - branch_package: # source_project: home:alex_sh:gsmartcontrol:github_ci:main # source_package: gsmartcontrol # target_project: home:alex_sh:gsmartcontrol:github_ci:main # # - configure_repositories: # project: home:alex_sh:gsmartcontrol:github_ci:main # repositories: # # - name: openSUSE_Tumbleweed # paths: # - target_project: openSUSE:Factory # target_repository: snapshot # architectures: # - x86_64 # - i586 # # - set_flags: # flags: # - type: publish # status: disable # project: home:alex_sh:gsmartcontrol:github_ci:main # # filters: # event: pull_request # branches: # only: # - main # Workflow for pushes build_main_workflow: steps: - rebuild_package: project: home:alex_sh:gsmartcontrol:github_ci:main package: gsmartcontrol - configure_repositories: project: home:alex_sh:gsmartcontrol:github_ci:main repositories: - name: openSUSE_Tumbleweed paths: - target_project: openSUSE:Factory target_repository: snapshot architectures: - x86_64 - i586 - name: openSUSE_Leap_15.4 paths: - target_project: openSUSE:Leap:15.4 target_repository: standard architectures: - x86_64 - name: openSUSE_Leap_15.3 paths: - target_project: openSUSE:Leap:15.3 target_repository: standard architectures: - x86_64 - name: openSUSE_Factory_PowerPC paths: - target_project: openSUSE:Factory:PowerPC target_repository: standard architectures: - ppc64 - ppc64le - name: openSUSE_Factory_ARM paths: - target_project: openSUSE:Factory:ARM target_repository: standard architectures: - aarch64 - armv7l - name: SUSE_SLE-15-SP4 paths: - target_project: SUSE:SLE-15-SP4:GA target_repository: standard architectures: - x86_64 - name: SUSE_SLE-12-SP5 paths: - target_project: SUSE:SLE-12-SP5:GA target_repository: standard architectures: - x86_64 - name: xUbuntu_21.10 paths: - target_project: openSUSE:Tools target_repository: xUbuntu_21.10 - target_project: Ubuntu:21.10 target_repository: universe architectures: - x86_64 - name: xUbuntu_20.04 paths: - target_project: openSUSE:Tools target_repository: xUbuntu_20.04 - target_project: Ubuntu:20.04 target_repository: universe architectures: - x86_64 - name: Fedora_Rawhide paths: - target_project: openSUSE:Tools target_repository: Fedora_Rawhide - target_project: Fedora:Rawhide target_repository: standard architectures: - x86_64 - name: Fedora_35 paths: - target_project: openSUSE:Tools target_repository: Fedora_35 - target_project: Fedora:35 target_repository: standard architectures: - x86_64 - aarch64 - armv7l - ppc64le - name: Fedora_34 paths: - target_project: openSUSE:Tools target_repository: Fedora_34 - target_project: Fedora:34 target_repository: standard architectures: - x86_64 - name: Debian_Unstable paths: - target_project: openSUSE:Tools target_repository: Debian_Unstable - target_project: Debian:Next target_repository: standard architectures: - x86_64 - i586 - name: Debian_Testing paths: - target_project: openSUSE:Tools target_repository: Debian_Testing - target_project: Debian:Testing target_repository: update architectures: - x86_64 - i586 - name: Debian_11 paths: - target_project: openSUSE:Tools target_repository: Debian_11 - target_project: Debian:11 target_repository: standard architectures: - x86_64 - i586 - name: Debian_10 paths: - target_project: openSUSE:Tools target_repository: Debian_10 - target_project: Debian:10 target_repository: standard architectures: - x86_64 - i586 - name: CentOS_8_Stream paths: - target_project: openSUSE:Tools target_repository: CentOS_8_Stream - target_project: Fedora:EPEL:8 target_repository: CentOS - target_project: CentOS:CentOS-8:Stream target_repository: standard architectures: - x86_64 - name: CentOS_8 paths: - target_project: openSUSE:Tools target_repository: CentOS_8 - target_project: Fedora:EPEL:8 target_repository: CentOS - target_project: CentOS:CentOS-8 target_repository: standard architectures: - x86_64 - set_flags: flags: - type: publish status: enable project: home:alex_sh:gsmartcontrol:github_ci:main - type: debuginfo status: enable project: home:alex_sh:gsmartcontrol:github_ci:main filters: event: push branches: only: - main ```
GitHub sent headers (click to open/close) ``` Request URL: https://build.opensuse.org/trigger/workflow?id=5904 Request method: POST Accept: */* content-type: application/json User-Agent: GitHub-Hookshot/e76a539 X-GitHub-Delivery: 68db3d6a-8d89-11ec-9e7e-f85327a21ced X-GitHub-Event: push X-GitHub-Hook-ID: 343177367 X-GitHub-Hook-Installation-Target-ID: 341614595 X-GitHub-Hook-Installation-Target-Type: repository X-Hub-Signature: sha1=a4553283b658582d5bafaaa29a37df05bbe2c90e X-Hub-Signature-256: sha256=4014a2a42881f68fe4bc900e265ac5a3ecfba86e1ef4120be645c5d394b02527 ```

Expected Result

No crash, successful builds.

Further Information

Thanks!

dmarcoux commented 2 years ago

Hey @ashaduri,

the alignment of some YAML keys have to be corrected in the YAML you provided. While we should definitely better inform you about this, we simply crash right now... Sorry for that! We're going to improve this for sure. I'm tracking this now in our backlog.

The keys under each step are misaligned. I ran the YAML through a validator like http://www.yamllint.com/ and checked the result to see how each step looks like. Be sure to refer to the user documentation if you aren't sure about how a certain step should be defined.

For example, this YAML:

build_main_workflow:
  steps:
    - rebuild_package:
      project: home:alex_sh:gsmartcontrol:github_ci:main
      package: gsmartcontrol

should instead be like this:

build_main_workflow:
  steps:
    - rebuild_package:
        project: home:alex_sh:gsmartcontrol:github_ci:main
        package: gsmartcontrol

Once this is addressed, it should work :slightly_smiling_face:. Let me know how it goes.

dmarcoux commented 2 years ago

Exception on Errbit.

dmarcoux commented 2 years ago

Validate .obs/workflows.yml with a schema validation

ashaduri commented 2 years ago

Thanks!

I tried matching the yaml indents (I really hope a better syntax is used here). Using yamllint, the examples and my file seem to give similar output. However, now I get HTTP 400:

<status code="400" origin="backend">
  <summary>no repository defined</summary>
</status>

even though I have more than enough repositories defined in workflows.yaml.

I understand this is a different issue, so if needed, I can open another report.

The new `workflows.yaml` (click to open/close) ```YAML # This file describes OBS workflows: # https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.scm_ci_workflow_integration.html # Workflow for pull requests build_pull_request_workflow: steps: - branch_package: source_project: home:alex_sh:gsmartcontrol:github_ci:main source_package: gsmartcontrol target_project: home:alex_sh:gsmartcontrol:github_ci:main - configure_repositories: project: home:alex_sh:gsmartcontrol:github_ci:main repositories: - name: openSUSE_Tumbleweed paths: - target_project: openSUSE:Factory target_repository: snapshot architectures: - x86_64 - i586 - set_flags: flags: - type: publish status: disable project: home:alex_sh:gsmartcontrol:github_ci:main filters: event: pull_request branches: only: - main # Workflow for pushes build_main_workflow: steps: - rebuild_package: project: home:alex_sh:gsmartcontrol:github_ci:main package: gsmartcontrol - configure_repositories: project: home:alex_sh:gsmartcontrol:github_ci:main repositories: - name: openSUSE_Tumbleweed paths: - target_project: openSUSE:Factory target_repository: snapshot architectures: - x86_64 - i586 - name: openSUSE_Leap_15.4 paths: - target_project: openSUSE:Leap:15.4 target_repository: standard architectures: - x86_64 - name: openSUSE_Leap_15.3 paths: - target_project: openSUSE:Leap:15.3 target_repository: standard architectures: - x86_64 - name: openSUSE_Factory_PowerPC paths: - target_project: openSUSE:Factory:PowerPC target_repository: standard architectures: - ppc64 - ppc64le - name: openSUSE_Factory_ARM paths: - target_project: openSUSE:Factory:ARM target_repository: standard architectures: - aarch64 - armv7l - name: SUSE_SLE-15-SP4 paths: - target_project: SUSE:SLE-15-SP4:GA target_repository: standard architectures: - x86_64 - name: SUSE_SLE-12-SP5 paths: - target_project: SUSE:SLE-12-SP5:GA target_repository: standard architectures: - x86_64 - name: xUbuntu_21.10 paths: - target_project: openSUSE:Tools target_repository: xUbuntu_21.10 - target_project: Ubuntu:21.10 target_repository: universe architectures: - x86_64 - name: xUbuntu_20.04 paths: - target_project: openSUSE:Tools target_repository: xUbuntu_20.04 - target_project: Ubuntu:20.04 target_repository: universe architectures: - x86_64 - name: Fedora_Rawhide paths: - target_project: openSUSE:Tools target_repository: Fedora_Rawhide - target_project: Fedora:Rawhide target_repository: standard architectures: - x86_64 - name: Fedora_35 paths: - target_project: openSUSE:Tools target_repository: Fedora_35 - target_project: Fedora:35 target_repository: standard architectures: - x86_64 - aarch64 - armv7l - ppc64le - name: Fedora_34 paths: - target_project: openSUSE:Tools target_repository: Fedora_34 - target_project: Fedora:34 target_repository: standard architectures: - x86_64 - name: Debian_Unstable paths: - target_project: openSUSE:Tools target_repository: Debian_Unstable - target_project: Debian:Next target_repository: standard architectures: - x86_64 - i586 - name: Debian_Testing paths: - target_project: openSUSE:Tools target_repository: Debian_Testing - target_project: Debian:Testing target_repository: update architectures: - x86_64 - i586 - name: Debian_11 paths: - target_project: openSUSE:Tools target_repository: Debian_11 - target_project: Debian:11 target_repository: standard architectures: - x86_64 - i586 - name: Debian_10 paths: - target_project: openSUSE:Tools target_repository: Debian_10 - target_project: Debian:10 target_repository: standard architectures: - x86_64 - i586 - name: CentOS_8_Stream paths: - target_project: openSUSE:Tools target_repository: CentOS_8_Stream - target_project: Fedora:EPEL:8 target_repository: CentOS - target_project: CentOS:CentOS-8:Stream target_repository: standard architectures: - x86_64 - name: CentOS_8 paths: - target_project: openSUSE:Tools target_repository: CentOS_8 - target_project: Fedora:EPEL:8 target_repository: CentOS - target_project: CentOS:CentOS-8 target_repository: standard architectures: - x86_64 - set_flags: flags: - type: publish status: enable project: home:alex_sh:gsmartcontrol:github_ci:main - type: debuginfo status: enable project: home:alex_sh:gsmartcontrol:github_ci:main filters: event: push branches: only: - main ```

Thanks!

dmarcoux commented 2 years ago

This error happens when a package cannot build due to not having repositories defined. Is this happening in build_main_workflow? If so, I would first configure the repositories. However, configuring repositories over and over again in a workflow for push events doesn't provide much value. I would suggest configuring those repositories in the project itself, through the web UI. Do it once, and it doesn't need to run in a workflow. Rebuilding the package would work afterwards since repositories are defined in the package's project.

ashaduri commented 2 years ago

Hello,

Yes, it happened on build_main_workflow (triggered via push event). I fixed it by moving the rebuild_package step to be the last step, which kinda makes sense now.

As for setting up the repositories, I do prefer to have all the workflow configuration in one place, that's why I have it in the workflow file. Plus, it's quicker and easier than the web UI. Though I do understand that the configure_repositories step makes more sense for merge requests.

I don't suppose there is an (undocumented) "remove existing repositories" step or an option to clear the repository list before adding the new ones via configure_repositories?

Thanks!

dmarcoux commented 2 years ago

I don't suppose there is an (undocumented) "remove existing repositories" step or an option to clear the repository list before adding the new ones via configure_repositories?

Removing/clearing repositories isn't possible through the OBS workflows right now. We don't have plans right now to implement this.