Fix not download files because of an early throw error.

[espekkaya]

When the test was finished with an error, I realized that the customer artifact file have not been downloaded. So I put throw section after the downloaded artifacts.

[nirinchev]

@cla-bot check

[cla-bot[bot]]

Realm welcomes all contributions! The only requirement we have is that, like many other projects, we need to have a Contributor License Agreement (CLA) in place before we can accept any external code. Our own CLA is a modified version of the Apache Software Foundation’s CLA. Our records show that CLA has not been signed by @enespekkaya-carbon. Please submit your CLA electronically using our Google form so we can accept your submissions. After signing the CLA you can recheck this PR with a @cla-bot check comment. The GitHub usernames you file there will need to match that of your Pull Requests. If you have any questions or cannot file the CLA electronically, make a comment here and we will be happy to help you out.

[cla-bot[bot]]

The cla-bot has been summoned, and re-checked this pull request!

[nirinchev]

Hey @espekkaya thank you so much for this! Would you mind signing our digital CLA? @bwachter this change looks reasonable to me, but would you mind verifying it as you have much more context than me.

[espekkaya]

You are welcome @nirinchev. Yeah, I've filled out the google form for this pr.

Some of our test cases have failed, but when I unzipped the Customer Artifacts.zip file, there is no existing file which I got. So I carry throw error section to the bottom of the method.

[nirinchev]

@cla-bot check

[cla-bot[bot]]

Realm welcomes all contributions! The only requirement we have is that, like many other projects, we need to have a Contributor License Agreement (CLA) in place before we can accept any external code. Our own CLA is a modified version of the Apache Software Foundation’s CLA. Our records show that CLA has not been signed by @enespekkaya-carbon. Please submit your CLA electronically using our Google form so we can accept your submissions. After signing the CLA you can recheck this PR with a @cla-bot check comment. The GitHub usernames you file there will need to match that of your Pull Requests. If you have any questions or cannot file the CLA electronically, make a comment here and we will be happy to help you out.

[cla-bot[bot]]

The cla-bot has been summoned, and re-checked this pull request!

[espekkaya]

@cla-bot check

[cla-bot[bot]]

Realm welcomes all contributions! The only requirement we have is that, like many other projects, we need to have a Contributor License Agreement (CLA) in place before we can accept any external code. Our own CLA is a modified version of the Apache Software Foundation’s CLA. Our records show that CLA has not been signed by @enespekkaya-carbon. Please submit your CLA electronically using our Google form so we can accept your submissions. After signing the CLA you can recheck this PR with a @cla-bot check comment. The GitHub usernames you file there will need to match that of your Pull Requests. If you have any questions or cannot file the CLA electronically, make a comment here and we will be happy to help you out.

[cla-bot[bot]]

The cla-bot has been summoned, and re-checked this pull request!

[nirinchev]

@cla-bot check

[cla-bot[bot]]

The cla-bot has been summoned, and re-checked this pull request!

[bwachter]

You are welcome @nirinchev. Yeah, I've filled out the google form for this pr.

Some of our test cases have failed, but when I unzipped the Customer Artifacts.zip file, there is no existing file which I got. So I carry throw error section to the bottom of the method.

@espekkaya could you share sections of the build log for the failed job? I'd like to understand a bit better what is going on.

That artifact download clause should only get triggered if a test run has passed, so moving it should not change what you're observing - so we might need to catch whatever is happening in a different way.

[espekkaya]

Hi @bwachter , We use webdriverio framework for our E2E. If one of the test cases is failed, it throws an error. So aws device farm understand that there is a wrong something.

[0-0] FAILED in safari - /__tests__/specs/provider/login.spec.ts
Report successfully generated to /private/tmp/scratchY7Qecz.scratch/test-package_NtNbN/report/allure-report

Spec Files:  0 passed, 1 failed, 1 total (100% completed) in 00:02:08 

you can see the ss;

[bwachter]

@espekkaya there are a few cases where things can fail either before or after the test run itself - which also would show up as error like this in AWS console. You could see if there are useful logs about what exactly was the error in the AWS console, and check if you can share them with me (after cleanup) - though Amazons logging is quite bad there, so it's possible it's either not there, or not easy to find.

As a workaround I have relatively verbose output inside of the action itself - so please check the output of the github runner for the job on github to see if there is relevant log output (ideally also shareable).

[espekkaya]

@bwachter , I understand what you mean, but there were no useful logs when I looked at them.

When all of test cases were passed, everything worked as a charm.

[espekkaya]

Hi, Is there any progress in that pr? or do you have any concerns about it? :)

Let me tell you one more thing. In case of success or failure, anyone wants to download logs to analyze it.

[nirinchev]

@bwachter is it a problem to try and download the artifacts even if the test run fails? As far as I can tell, even for a failed run, it's possible there are valuable details in the logs that we wouldn't be getting otherwise.

[bwachter]

@espekkaya ah, I think I misunderstood the problem you were trying to solve. Just to confirm, you did have downloadable logs for those failures? This is currently structured this way as in my aborted test runs I never seen any useful downloadable logs. If there are logs it makes sense to download them, though we might need to guard the log downloading separately in case we don't have logs (but we probably should observe it failing first, so if you can confirm there were logs I think I can merge it)

[espekkaya]

Hi @bwachter, We put our logs and also report results into Customer Artifacts.zip. Yeah, we did have downloadable logs for those failures to save them in the Aws s3 bucket.

Such as, we run our test cases over appium, and we need to serve the log on our screens.

You can find the following little logs about appium.

Thank you in advance.

[HTTP] --> POST /session
[HTTP] {"capabilities":{"alwaysMatch":{"platformName":"Android","browserName":"chrome","appium:deviceName":"R3CR10AXW5A","appium:platformVersion":"12","appium:chromedriverExecutable":"/opt/chromedriver/linux/65/chromedriver","appium:orientation":"PORTRAIT","appium:automationName":"UiAutomator2","appium:newCommandTimeout":240,"goog:chromeOptions":{"w3c":false}},"firstMatch":[{}]},"desiredCapabilities":{"platformName":"Android","browserName":"chrome","appium:deviceName":"R3CR10AXW5A","appium:platformVersion":"12","appium:chromedriverExecutable":"/opt/chromedriver/linux/65/chromedriver","appium:orientation":"PORTRAIT","appium:automationName":"UiAutomator2","appium:newCommandTimeout":240,"goog:chromeOptions":{"w3c":false}}}
[debug] [W3C] Calling AppiumDriver.createSession() with args: [{"platformName":"Android","browserName":"chrome","appium:deviceName":"R3CR10AXW5A","appium:platformVersion":"12","appium:chromedriverExecutable":"/opt/chromedriver/linux/65/chromedriver","appium:orientation":"PORTRAIT","appium:automationName":"UiAutomator2","appium:newCommandTimeout":240,"goog:chromeOptions":{"w3c":false}},null,{"alwaysMatch":{"platformName":"Android","browserName":"chrome","appium:deviceName":"R3CR10AXW5A","appium:platformVersion":"12","appium:chromedriverExecutable":"/opt/chromedriver/linux/65/chromedriver","appium:orientation":"PORTRAIT","appium:automationName":"UiAutomator2","appium:newCommandTimeout":240,"goog:chromeOptions":{"w3c":false}},"firstMatch":[{}]}]
[debug] [BaseDriver] Event 'newSessionRequested' logged at 1668425214419 (11:26:54 GMT+0000 (Coordinated Universal Time))
[Appium] Appium v1.22.3 creating new AndroidUiautomator2Driver (v1.75.0) session
[Appium] Applying relaxed security to 'AndroidUiautomator2Driver' as per server command line argument. All insecure features will be enabled unless explicitly disabled by --deny-insecure
[debug] [BaseDriver] W3C capabilities and MJSONWP desired capabilities were provided
[debug] [BaseDriver] Creating session with W3C capabilities: {