sclorg / testing-farm-as-github-action

GitHub Action to execute tests by Testing Farm and update Pull Request status
MIT License
13 stars 11 forks source link

v2 can't pass context to tmt #136

Closed henrywang closed 5 months ago

henrywang commented 6 months ago

Configured context in the github action, but it can't be passed to tmt. Configuration in github action:

Run sclorg/testing-farm-as-github-action@v2
  with:
    compose: CentOS-Stream-9
    api_key: ***
    git_url: https://github.com/henrywang/bootc-workflow-test
    git_ref: osbuild_fix
    arch: x86_64
    tmt_context: arch=x86_64
    update_pull_request_status: true
    pull_request_status_name: bootc-cs9-x86_64-bib-qcow2
    tmt_plan_regex: qcow2
    tf_scope: private
    secrets: QUAY_USERNAME=***;QUAY_PASSWORD=***
    variables: TEST_OS=centos-stream-9;PLATFORM=libvirt;ARCH=x86_64;IMAGE_TYPE=qcow2;AWS_REGION=***;GOVC_INSECURE=1
    api_url: https://api.dev.testing-farm.io/v0.1
    github_token: ***
    create_issue_comment: false
    create_github_summary: true
    copr: epel-7-x86_64
    environment_settings: {}
    timeout: 480

Testing farm log:

[32m[09:38:08] [+] [testing-farm-request] Connected to Testing Farm Service 'https://internal.api.dev.testing-farm.io'
[09:38:09] [+] [testing-farm-request] Initialized with 184f6b60-bd1f-4940-9a81-51f6b54b69d1:
{
    "environments_requested": [
        {
            "arch": "x86_64",
            "artifacts": [],
            "compose": "CentOS-Stream-9",
            "secrets": "******",
            "settings": {
                "pipeline": null,
                "provisioning": null
            },
            "snapshots": false,
            "tmt": {
                "context": null,
                "environment": null
            },
            "variables": {
                "ARCH": "x86_64",
                "AWS_REGION": "***",
                "GOVC_INSECURE": "1",
                "IMAGE_TYPE": "qcow2",
                "PLATFORM": "libvirt",
                "TESTING_FARM_GIT_REF": "osbuild_fix",
                "TESTING_FARM_GIT_URL": "https://github.com/henrywang/bootc-workflow-test",
                "TESTING_FARM_REQUEST_ID": "184f6b60-bd1f-4940-9a81-51f6b54b69d1",
                "TESTING_FARM_TEST_TYPE": "fmf",
                "TEST_OS": "centos-stream-9"
            }
        }
    ],
    "plan": "qcow2",
    "plan_filter": "<not applicable>",
    "ref": "osbuild_fix",
    "test_filter": "<not applicable>",
    "test_name": "<not applicable>",
    "type": "fmf",
    "url": "https://github.com/henrywang/bootc-workflow-test",
    "webhook_url": "<no webhook specified>"
}
jamacku commented 6 months ago

~I don't see any issue based on the logs you have provided, That context seems to be passed correctly to TF.~

~tmt_context: arch=x86_64 ~ "variables": {"ARCH": "x86_64",...}~

EDIT: Actually, you are right. There is a bug. We expect that all context variables are passed, which is probably a wrong assumption.

Corresponding code:

https://github.com/sclorg/testing-farm-as-github-action/blob/a7d9cca1648be8b432f99f68dd5f55c106cf1d01/src/action.ts#L74-L80

https://github.com/sclorg/testing-farm-as-github-action/blob/a7d9cca1648be8b432f99f68dd5f55c106cf1d01/src/schema/input.ts#L50-L58

henrywang commented 6 months ago

I don't see any issue based on the logs you have provided, That context seems to be passed correctly to TF.

tmt_context: arch=x86_64 ~ "variables": {"ARCH": "x86_64",...}

Corresponding code:

https://github.com/sclorg/testing-farm-as-github-action/blob/a7d9cca1648be8b432f99f68dd5f55c106cf1d01/src/action.ts#L74-L80

https://github.com/sclorg/testing-farm-as-github-action/blob/a7d9cca1648be8b432f99f68dd5f55c106cf1d01/src/schema/input.ts#L50-L58

The ARCH is variable I defined as environment variable. But the arch is the context I defined and it's null now.

tmt": {
                "context": null,
                "environment": null
            },

The right one should look like:

"tmt": {
                "context": {
                    "arch": "x86_64"
                },
                "environment": null
            },
jamacku commented 6 months ago

You are right I have just updated my comment above