trcli fails to upload JUnit report with KeyError: 'custom_automation_id'

[arielelkin]

TestRail CLI Version

1.5.0

CLI Environment

macOS 13.4.1

TestRail Version

TestRail v8.0.1 Default (1030)

TestRail Instance Type

Professional Cloud

Current behavior

trcli \
    --host "https://instance.testrail.io/" \
    --username "email@email.com" \
    --password "password" \
    --project "My App" \
    parse_junit \
    --title "Testing Test Run One" \
    --run-description "My CI Build" \
    -f "test-reports/junit-report.xml"

TestRail CLI v1.5.0
Copyright 2021 Gurock Software GmbH - www.gurock.com
Parse JUnit Execution Parameters
> Report file: test-reports/clarity-junit-report.xml
> Config file: None
> TestRail instance: https://iris2.testrail.io/ (user: carmen@shapedbyiris.com)
> Project: Mac App
> Run title: Testing Test Run One
> Update run: No
> Add to milestone: No
> Auto-create entities: None
Parsing JUnit report.
Processing suite - ClarityUITests.xctest
Processed 11 test cases in 4 sections.
Checking project. Done.
Traceback (most recent call last):
  File "/usr/local/bin/trcli", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/trcli/commands/cmd_parse_junit.py", line 92, in cli
    result_uploader.upload_results()
  File "/usr/local/lib/python3.10/site-packages/trcli/api/results_uploader.py", line 67, in upload_results
    missing_test_cases, error_message = self.api_request_handler.check_missing_test_cases_ids(
  File "/usr/local/lib/python3.10/site-packages/trcli/api/api_request_handler.py", line 291, in check_missing_test_cases_ids
    aut_case_id = case["custom_automation_id"]
KeyError: 'custom_automation_id'

Here's a sample report file: junit-report.xml.zip

Desired behavior

No response

More Details

No response

[Testinator-X]

Did you set up the custom field "automation_id" in your TestRail instance as described here or here? Is it assigned to the TestRail project „My App“ and is it displayed in the web UI when creating a TestCase?

[arielelkin]

Hello @Testinator-X , thanks for your response.

Nope I haven't set up that custom field. In any case, this is a bug in trcli as I'd expect it to output an error message to that effect, rather than crash as it currently does.

By the way, the installation instructions don't mention that setting up the automation_id custom field is necessary for trcli to run, so consider modifying the docs accordingly.

[mookiang]

I set the automation_id. The first one succeeded in uploading, but the above error occurred from the second one. T_T

[d-rede]

Hello @arielelkin

I'm glad you were able to ultimately figure it out.

Setting up the automation_id is not a requirement to use the CLI itself. It is a requirement for the Code-first workflow, where we explain how to set up the field. We do have a check in place in the CLI code to warn users if they haven't configured the field, unfortunately we can't detect all situations where it may be misconfigured so, in very particular occasions, this error may occur.

Since this is no longer an active blocker, I'm closing this issue and we will follow-up on https://github.com/gurock/trcli/issues/144.