Closed metaprime closed 10 years ago
This seems ripper-specific (not all sites have .php
files).
Can you provide links to albums that sometimes rip .php
files? Or that have ripped .php files in the past? Which ripper was it?
Sadly it was the E-Hentai ripper I just implemented, but I never wrote anything about .php files in it, so I'm not even sure how that's happening unless it's coming from the code which calls it.
Usually it's just one or two of them for every couple dozen photos downloaded. It seems non-deterministic like a race condition or something. One album that was downloaded particularly badly (about 25% of files weren't named properly) was: http://g.e-hentai.org/g/288442/34e34ff52d/
Here's the directory listing post-download.
000_09.12.05A.jpg 025_image.php 001_DSC05023.jpg 026_DSC05143.jpg 002_DSC05026.jpg 027_DSC05146.jpg 003_DSC05029.jpg 028_DSC05148.jpg 004_DSC05035.jpg 029_DSC05153.jpg 005_DSC05039.jpg 030_DSC05155.jpg 006_DSC05041.jpg 031_image.php 007_DSC05043.jpg 032_DSC05158.jpg 008_image.php 033_DSC05163.jpg 009_DSC05046.jpg 034_DSC05166.jpg 010_DSC05052.jpg 035_DSC05167.jpg 011_DSC05059.jpg 036_image.php 012_DSC05061.jpg 037_DSC05170.jpg 013_DSC05063.jpg 038_DSC05176.jpg 014_DSC05066.jpg 039_DSC05194.jpg 015_DSC05087.jpg 040_image.php 016_image.php 041_DSC05206.jpg 017_DSC05097.jpg 042_DSC05208.jpg 018_DSC05100.jpg 043_image.php 019_image.php 044_DSC05218.jpg 020_DSC05118.jpg 045_DSC05222.jpg 021_DSC05121.jpg 046_image.php 022_DSC05124.jpg 047_DSC05230.jpg 023_image.php 048_image.php 024_image.php [LiTU100] ?????? Lan Yi #1 (2010.01.18) - E-Hentai Galleries.url
All of the files have the correct ordinal prefix, which is being set inside of the ripper class as far as I can tell, but anything which is .php comes out as "%03d_image.php"
By the way those .php files are actually jpegs. Renaming them to file extension .jpg works correctly and everything is properly downloaded.
Here's another album that downloaded with a few files names incorrectly.
http://g.e-hentai.org/g/293294/af8f042c87/
It may simply be a race condition that is somehow not code-specific but rather site-specific? I have no idea, I'm just shooting in the dark.
I know that running rename *.php *.jpg
following the download fixes all the files in a hurry, so maybe something similar could be done here, following the download?
The overloaded addURLToDownload()
method will try to guess the filename when given a URL & prefix: https://github.com/4pr0n/ripme/blob/master/src/main/java/com/rarchives/ripme/ripper/AbstractRipper.java#L94-L110
Where it's invoked from Ehentai: https://github.com/4pr0n/ripme/commit/f9e8dd33060167a10c572495dc9d0ec8886d06ad#diff-3ed19e2df23533ecd444859cc81df81bR96
Most of the album will be correctly downloaded, but a couple of images are named like 013_image.php (which seems an odd choice for a default extension for downloads anyway). Changing extension to .jpg shows that the picture was properly downloaded, just not renamed.
Possibly a thread locking or concurrency issue?
I'm on Windows 7 btw