Closed Burby500 closed 2 years ago
nothing wrong with the parser at all, the site and servers are just shit and are under constant heavy load. nothing we can about it. its up to them to make their servers respond faster and download faster. 64KB download speed for files is not normal at all. but it is for them.
It looks like requests for objects e.g. https://kemono.party/attachments/xxxxxx/xxxxxx/xxxxxx.jpg
receive a 302 and get balanced onto one of several datax.kemono.party
subdomains. If you remove the data
subdomain from the parser's downloadable/pursuable urls you can let the server do its redirect and get better results.
you might actually be right. i did not make the full parser as a whole and overlooked that regex replacement part there. no idea why it would replace the url and prepend the site like that. need to test a bit. hmm might help a bit for sure if it splits shit up across the servers.
testing the parser without those regex changes now and it seemingly work. will have to test a bit more then ill push a new release for it. it should speed up the file downloads for sure... in theory
Cool, although I should say it was throwing the same error for attachments/xxx...xxx.gif as well.
Full errors: Ran out of reattempts on this error: 502: … (Copy note to see full error) Traceback (most recent call last): File "hydrus\client\networking\ClientNetworkingJobs.py", line 1257, in Start raise e hydrus.core.HydrusExceptions.ShouldReattemptNetworkException: 502:
502 Bad Gateway
502 Bad Gateway
502 Bad Gateway
502 Bad Gateway
502 Bad Gateway
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "hydrus\client\importing\ClientImportFileSeeds.py", line 1144, in WorkOnURL self.DownloadAndImportRawFile( file_url, file_import_options, network_job_factory, network_job_presentation_context_factory, status_hook ) File "hydrus\client\importing\ClientImportFileSeeds.py", line 317, in DownloadAndImportRawFile network_job.WaitUntilDone() File "hydrus\client\networking\ClientNetworkingJobs.py", line 1533, in WaitUntilDone raise self._error_exception File "hydrus\client\networking\ClientNetworkingJobs.py", line 1287, in Start raise HydrusExceptions.NetworkInfrastructureException( 'Ran out of reattempts on this error: ' + str( e ) ) hydrus.core.HydrusExceptions.NetworkInfrastructureException: Ran out of reattempts on this error: 502:
502 Bad Gateway
502 Bad Gateway
502 Bad Gateway
502 Bad Gateway
502 Bad Gateway