Invalid Http Response Code Not Properly Handled

[Still34]

Expected Behavior

When performing a scrape run, if the server responds with an unexpected status code (e.g. 500), the script should handle such response gracefully, as opposed to exiting the task. This is particularly true if the module was invoked with the -Recurse flag, where more than one input files are expected. Perhaps offering a finite amount of retries per task or showing a warning would be a potential solution. While setting $ErrorActionPreference to a non-default value that allows for the script to continue (e.g. Continue,) this solution is not ideal, as the problematic files would not be logged.

Current Behavior

The following exception occurs while executing the script if the server does not respond a 200 OK:

❯ Javinizer -Recurse
Get-RunspaceData: <$env:userprofile>\Documents\PowerShell\Modules\Javinizer\2.0.2\Private\Invoke-Parallel.ps1:532
Line |
 532 |                      Get-RunspaceData
     |                      ~~~~~~~~~~~~~~~~
     | Exception calling "EndInvoke" with "1" argument(s): "The running command stopped because the
     | preference variable "ErrorActionPreference" or common parameter is set to Stop: [ABCD-001]
     | [Get-JVData] Error occured during scraper jobs: [ABCD-001] [Get-JavlibraryUrl] Error occured on [GET]
     | on URL [http://www.javlibrary.com/en/vl_searchbyid.php?keyword=ABCD-001] with Session: [] and
     | UserAgent: []: Response status code does not indicate success: 500 (Internal Server Error)."

Your Environment

• Module version used: v2.2.0
• Operating System and PowerShell version: Windows 10 x64 v7.0.3

[jvlflame]

I agree with your points. I've personally run into the 500 error on the Javlibrary scraper as well, albeit very randomly. I'll do some testing to see what the cleanest approach is to handling that this interaction.