javinizer / Javinizer

(NSFW) Organize your local Japanese Adult Video (JAV) library
MIT License
606 stars 63 forks source link

Invalid Http Response Code Not Properly Handled #109

Closed Still34 closed 4 years ago

Still34 commented 4 years ago

Expected Behavior

When performing a scrape run, if the server responds with an unexpected status code (e.g. 500), the script should handle such response gracefully, as opposed to exiting the task. This is particularly true if the module was invoked with the -Recurse flag, where more than one input files are expected. Perhaps offering a finite amount of retries per task or showing a warning would be a potential solution. While setting $ErrorActionPreference to a non-default value that allows for the script to continue (e.g. Continue,) this solution is not ideal, as the problematic files would not be logged.

Current Behavior

The following exception occurs while executing the script if the server does not respond a 200 OK:

❯ Javinizer -Recurse
Get-RunspaceData: <$env:userprofile>\Documents\PowerShell\Modules\Javinizer\2.0.2\Private\Invoke-Parallel.ps1:532
Line |
 532 |                      Get-RunspaceData
     |                      ~~~~~~~~~~~~~~~~
     | Exception calling "EndInvoke" with "1" argument(s): "The running command stopped because the
     | preference variable "ErrorActionPreference" or common parameter is set to Stop: [ABCD-001]
     | [Get-JVData] Error occured during scraper jobs: [ABCD-001] [Get-JavlibraryUrl] Error occured on [GET]
     | on URL [http://www.javlibrary.com/en/vl_searchbyid.php?keyword=ABCD-001] with Session: [] and
     | UserAgent: []: Response status code does not indicate success: 500 (Internal Server Error)."

Your Environment

jvlflame commented 4 years ago

I agree with your points. I've personally run into the 500 error on the Javlibrary scraper as well, albeit very randomly. I'll do some testing to see what the cleanest approach is to handling that this interaction.