search

reportportal / agent-Python-RobotFramework

Robot Framework integration for Report Portal
Apache License 2.0
59 stars 34 forks source link

No launch in reportportal after execution pabot tests #153

Closed gajdaj2 closed 1 year ago

gajdaj2 commented 1 year ago

I have this issue when I execute my tests with pabot. Everything going ok but I don't see launch in reportportal my version of reportportal: 5.6.1 , pabot: 2.7.0, robotframework 5.0.1 robot-framerwork-reportportal 5.2.0

I'm executing my tests like this: pabot --verbose --processes 6 --listener robotframework_reportportal.listener --variable RP_LAUNCH_UUID:xxxxxxx --variable RP_UUID:xxxxxx --variable RP_ENDPOINT:https://reportportal_url --variable RP_LAUNCH:SANITY --variable RP_PROJECT:project_name --include healthcheck -v env:dev02 -v Release:APiv2 tests

in output I get this:

Storing .pabotsuitenames file Suite names resolved in 3.032909870147705 seconds 2022-10-06 09:40:13.377252 [PID:13540] [0] [ID:0] EXECUTING PARALLEL Tests.Services Health Check with command: robot --suite "Tests.Services Health Check" --include healthcheck --variable RP_LAUNCH_UUID:xxxxx --variable RP_UUID:xxxxx--variable RP_ENDPOINT:https://reportportal)url --variable RP_LAUNCH:SANITY --variable RP_PROJECT:project_name --variable env:dev01 --variable Release:APIv2 --variable CALLER_ID:91d2082319f444c48d4beb921dd9e9cd --variable PABOTLIBURI:127.0.0.1:8270 --variable PABOTEXECUTIONPO OLID:0 --variable PABOTISLASTEXECUTIONINPOOL:1 --variable PABOTNUMBEROFPROCESSES:6 --variable PABOTQUEUEINDEX:0 --variable PABOTLASTLEVEL: --listener robotframework_reportportal.listener --log NONE --report NONE --xunit NONE --outpu tdir ".\pabot_results\0" --consolecolors off --consolemarkers off tests 2022-10-06 09:40:23.337080 [PID:13540] [0] [ID:0] PASSED Tests.Services Health Check in 9.1 seconds

Tests

Tests.Services Health Check

Obh Services Health check | PASS |

Down Stream Services Health Check | PASS |

Tests.Services Health Check | PASS | 2 tests, 2 passed, 0 failed

Tests | PASS | 2 tests, 2 passed, 0 failed

Output: C:\git_repos\SIT\openbanking-automation-sit\pabot_results\0\output.xml

2 tests, 2 passed, 0 failed, 0 skipped.

Output: C:\\output.xml Log: C:\\log.html Report: C:\\report.html Total testing: 9.10 seconds Elapsed time: 13.13 seconds

sovap commented 1 year ago

HI all, did a similar test with pabot with the same results. Nothing was imported to reportportal.

Here are the versions: reportportal: 5.7.2 robotframework-pabot: 2.8.0 robotframework 5.0.1 robot-framerwork-reportportal 5.3.0

Here is the command: pabot --pabotlib --processes 10 --testlevelsplit --include report_test -d robot/results --loglevel TRACE -v NO_CACHE:1 --listener robotframework_reportportal.listener --variable RP_UUID:464fa3f5-3b0d-4be7-a0d4-395f8abae5ad --variable RP_ENDPOINT:http://localhost:8080 --variable RP_LAUNCH:Regression --variable RP_PROJECT:vbt -V robot/config/variables/variables.py:en:orbit-vbt-refact-at robot/tests

Log from robotframework: 20221118 13:13:46.938 ERROR Calling method 'start_suite' of listener 'robotframework_reportportal.listener' failed: RobotServiceException: Pabot library is used but RP_LAUNCH_UUID was not provided. Please, initialize listener with the RP_LAUNCH_UUID argument. 20221118 13:13:47.113 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:47.126 WARN Attempt to start item for non-existent parent item
20221118 13:13:48.145 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:48.148 WARN Attempt to start item for non-existent parent item
20221118 13:13:48.211 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:48.212 WARN Attempt to log to non-existent item 20221118 13:13:48.212 WARN Attempt to start item for non-existent parent item
20221118 13:13:48.221 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:48.222 WARN Attempt to log to non-existent item 20221118 13:13:48.261 WARN Attempt to log to non-existent item 20221118 13:13:48.592 WARN Attempt to log to non-existent item 20221118 13:13:48.593 WARN Attempt to log to non-existent item 20221118 13:13:48.860 WARN Attempt to log to non-existent item 20221118 13:13:48.860 WARN Attempt to log to non-existent item 20221118 13:13:48.861 WARN Attempt to finish non-existent item 20221118 13:13:48.861 WARN Attempt to finish non-existent item 20221118 13:13:48.862 WARN Attempt to start item for non-existent parent item
20221118 13:13:48.872 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:48.872 WARN Attempt to finish non-existent item 20221118 13:13:48.873 WARN Attempt to finish non-existent item 20221118 13:13:48.875 WARN Attempt to start item for non-existent parent item
20221118 13:13:48.932 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:48.932 WARN Attempt to finish non-existent item 20221118 13:13:48.933 WARN Attempt to finish non-existent item 20221118 13:13:48.934 WARN Attempt to log to non-existent item 20221118 13:13:48.935 WARN Attempt to finish non-existent item 20221118 13:13:48.946 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:48.946 WARN Attempt to log to non-existent item 20221118 13:13:48.947 WARN Attempt to log to non-existent item 20221118 13:13:48.947 WARN Attempt to log to non-existent item 20221118 13:13:49.170 WARN Attempt to log to non-existent item 20221118 13:13:49.171 WARN Attempt to log to non-existent item 20221118 13:13:49.174 WARN Attempt to finish non-existent item 20221118 13:13:49.245 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:49.246 WARN Attempt to log to non-existent item 20221118 13:13:49.247 WARN Attempt to start item for non-existent parent item
20221118 13:13:49.301 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:49.302 WARN Attempt to log to non-existent item 20221118 13:13:49.303 WARN Attempt to log to non-existent item 20221118 13:13:49.304 WARN Attempt to finish non-existent item 20221118 13:13:49.305 WARN Attempt to finish non-existent item 20221118 13:13:49.307 WARN Attempt to start item for non-existent parent item
20221118 13:13:49.322 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:49.323 WARN Attempt to finish non-existent item 20221118 13:13:49.324 WARN Attempt to finish non-existent item 20221118 13:13:49.325 WARN Attempt to start item for non-existent parent item
20221118 13:13:49.339 WARN start_test_item - invalid response: {'errorCode': 4001, 'message': "Incorrect Request. [Field 'launchUuid' should not be null.] "}
20221118 13:13:49.339 WARN Attempt to finish non-existent item 20221118 13:13:49.340 WARN Attempt to finish non-existent item 20221118 13:13:49.341 WARN Attempt to log to non-existent item 20221118 13:13:49.342 WARN Attempt to finish non-existent item 20221118 13:13:49.343 WARN Attempt to finish non-existent item 20221118 13:13:49.345 WARN Attempt to finish non-existent item 20221118 13:13:49.347 WARN Attempt to finish non-existent item 20221118 13:13:49.349 WARN Attempt to finish non-existent item 20221118 13:13:49.351 ERROR Calling method 'end_suite' of listener 'robotframework_reportportal.listener' failed: IndexError: pop from empty list

Here is the log from console (nothing interesting here i guess):

Storing .pabotsuitenames file
2022-11-18 13:13:42.152724 [PID:33280] [0] [ID:0] EXECUTING Tests.Common.Keywords.Sftp Tests.Sftp test 2 - create a directory on the sftp server
2022-11-18 13:13:49.853390 [PID:33280] [0] [ID:0] PASSED Tests.Common.Keywords.Sftp Tests.Sftp test 2 - create a directory on the sftp server in 6.9 seconds
1 tests, 1 passed, 0 failed, 0 skipped.
===================================================
Output:  C:\Users\sovap\Projects\payments-parent\Testing-refactoring\robot\results\output.xml
Log:     C:\Users\sovap\Projects\payments-parent\Testing-refactoring\robot\results\log.html
Report:  C:\Users\sovap\Projects\payments-parent\Testing-refactoring\robot\results\report.html
Stopping PabotLib process
Robot Framework remote server at 127.0.0.1:8270 started.
Robot Framework remote server at 127.0.0.1:8270 stopped.
PabotLib process stopped
Total testing: 6.90 seconds
Elapsed time:  9.39 seconds

Process finished with exit code 0

Note: without pabot everything is working as expected.

HardNorth commented 1 year ago

@gajdaj2 @sovap It's not possible for us to create a launch with pabot before it starts running test suites. Pabot does not provide any instruments to do something before test suites are distributed across processes.

So you should use RP_LAUNCH_UUID property and provide there a UUID of existing launch. See our documentation: https://github.com/reportportal/agent-Python-RobotFramework#properties

Also you can create a new launch in a shell script with curl command, like that:

curl --location --request POST 'http://localhost:8080/api/v1/default_personal/launch' \
--header 'Authorization: Bearer your_api_token_here' \
--header 'Content-Type: application/json' \
--data-raw '{
  "attributes": [
    {
      "key": "key",
      "value": "value"
    }
  ],
  "description": "Launch description",
  "mode": "DEFAULT",
  "name": "Launch name",
  "startTime": "2021-05-28T08:03:05.063Z"
}'

And extract Launch UUID from the response:

{
    "id": "08657127-80f6-4b98-9507-9030c9da40c4",
    "number": 1
}

You can use jq command for it, E.G.:

| jq -r '.id'

The result will look like that: 08657127-80f6-4b98-9507-9030c9da40c4

theeraweej commented 1 year ago

@HardNorth

I am working on robot framework and pabot. It works properly as I can see the test result on ReportPortal is correct. Here is what I have done

Next step, I want to send attachment to reportportal, so I I follow the instruction from https://github.com/reportportal/client-Python#send-attachement-screenshots by calling 'logger.info'

However, it does not work as expected. Attachment is not sent to reportportal and there is no error. In order to identify the root cause, I capture the message and I can see that launch id is NULL (it is NOT NULL when sending test result).

I am not sure which step I missed. please advise.

HardNorth commented 1 year ago

@theeraweej there is a separate issue for this: #160