Closed GoogleCodeExporter closed 9 years ago
Yeah, I thinking about this. I think I'll just check for placeholder, if you
getting placeholder then application automatically will stop downloading. I
don't really know how to "detect" if it safe to continue. Maybe resume after ~
2min is enough for it. I also think maybe you can use ~15 second delay between
pictures.
Original comment by catgirlfighter
on 9 Jan 2014 at 8:48
That sounds excellent.
Original comment by tehvesu...@gmail.com
on 9 Jan 2014 at 7:40
Буржуй дело говорит, я уже писал ранее что
ошибка 504 идет при скачке больших галерей
примерно после 100-150 картинки, было бы
хорошо добавить в настройки пункт,
временной интервал настраиваемый и после
какой картинки его делать, ну или как вы уже
писали ранее выявление ошибки и после
пауза на минуты две (но я думаю второй
вариант программно довольно сложно
реализовать).
З.Ы.
Присоеденяюсь к буржую, Огромная
благодарность вам за ваш труд и этот проект.
Original comment by server26...@gmail.com
on 10 Jan 2014 at 7:19
That feature, or something like it, would surely be helpful.
Also, I've noticed that if you use $fname$, it doesn't actually use the
filename. Instead, it uses the last part of the Web Address of the post showing
the image.
Could this be fixed as well? (Or does not using $fname$ make it work in this
case?
Original comment by JarieSui...@gmail.com
on 12 Jan 2014 at 9:17
You should use %original_name% key to get "real file's name", $fname$ designed
to be unique and represent only one possible for this name image on the same
resource.
Original comment by catgirlfighter
on 13 Jan 2014 at 8:21
I added 509 check in e/exhentai script, and when it will be encountered, script
will stop downloading (for all resources, not onlt e/exhentai). And I added an
option in Threads Settings (Stop Signal Timer) where you can set an interval in
seconds to restart downloading.
Original comment by catgirlfighter
on 2 Apr 2014 at 2:04
Doesn't appear to be working. I enabled stop signal timer with a 121 second
delay and it refuses to start downloading the first image in the entire list,
even though I'm well under image limit cap.
Good to see you return, I thought you'd gone on holiday or taken a break :P
Original comment by tehvesu...@gmail.com
on 3 Apr 2014 at 2:49
In addition to my last post, I can't seem to download at all now, even after
changing stop signal timer back to 0.
Original comment by tehvesu...@gmail.com
on 3 Apr 2014 at 3:15
Nah, it's was just a deadline on my job, just got no time for grabber.
Can you provide keywords and name format to reproduce it?
Original comment by catgirlfighter
on 3 Apr 2014 at 8:14
I just got a problem with reproduction, I did download around 1500 pic in one
session yesterday and didn't get 509 message even once, so I just made changes
by intuition :)
Original comment by catgirlfighter
on 3 Apr 2014 at 8:19
Found it. Should work now.
Original comment by catgirlfighter
on 3 Apr 2014 at 8:39
Sorry to say, no it doesn't work, at least not yet (for me.)
Grabber is quite happy to keep trying to download pictures in regular or full
size well over 200 images past my limit. The only way I can trigger the image
limit detection is to try re-downloading the images after they fail.
Original comment by tehvesu...@gmail.com
on 3 Apr 2014 at 3:50
Maybe they use not only one placeholder for 509 error. When you will get an
error next time, please leave urls for that pictures (ploceholder urls, not
actual pictures).
As I told, I failed to reproduce a bandwidth exceed (it's just not happens to
me for some reason), so I should ask people, who actually encounter it.
Original comment by catgirlfighter
on 4 Apr 2014 at 8:36
Hmm this is odd. Keep in mind I'm downloading full images by default.
When I go over the limit Grabber reports the exhentai.org/fullimg.php etc image
url with ": can not get file extension" on the end.
I can go to the "url" link in my browser and I get the picture attached to this
post, so there's no 509 image to download it's just text.
I can open the "post_url" link and the picture loads fine, no 509.
I can't load "alt_url", that DOES come up with 509.
So I guess image limit only seems to increment when Grabber requests either:
1. A normal sized image that isn't downloaded through the hentai@home network
and needs to be pulled from the server (default URL or ALT URL.)
2. A full sized image.
You're right that there isn't only one placeholder for 509 errors, it's just
that the 509 error for a full size image isn't being detected by Grabber since
it's a text page (again, see attached picture.)
Original comment by tehvesu...@gmail.com
on 4 Apr 2014 at 3:14
Attachments:
I see now, I'll fix it (but now only next week).
Original comment by catgirlfighter
on 4 Apr 2014 at 3:21
Is going nowhere :/ I need a source of page with an error, to know, where I
should look for a message. Trying to get a limit again. 440 pics done for now.
Original comment by catgirlfighter
on 10 Apr 2014 at 11:15
How many pictures you download in one session, or it's just me? almost 3k
pictures without limits.
I need you to upload a source of page with an error message (file with
html-text in it) when you encounter it next time.
Original comment by catgirlfighter
on 10 Apr 2014 at 2:31
Attachments:
Sent you an email.
Original comment by tehvesu...@gmail.com
on 15 Apr 2014 at 4:09
Wow, this is weird. Didn't see it in my mail client, checked it through gmail
web-interface.
Original comment by catgirlfighter
on 16 Apr 2014 at 8:19
I see now, maybe is because you're downloading from exhentai, and I from
g.e-hentai. Testing now.
Original comment by catgirlfighter
on 16 Apr 2014 at 8:38
No, not because of it, is because of "Full image" flag in settings. There is
limit for downloading full images.
>Image Limits
>You are currently at 204 towards a limit of 180. This regenerates at a rate of
1 per minute.
>Note that image views only count towards the limit if they cannot be loaded
through H@H.
>Also note that the limit is not constant - it varies with server load and time
of day.
So you actually can download only 1 pic per minute to be out of limit range. Or
you can reset limit for 1GP per limit point you exceeded. And you should reset
it every 180 pictures. Annoying feature.
Original comment by catgirlfighter
on 16 Apr 2014 at 9:07
Bingo :)
I don't mind paying the GP to reset the images, but if I leave Grabber running
unattended and don't pay attention to the image limit it can reach around
1000/200 because Grabber isn't detecting a 509 error for full sized images.
That means paying an extra 800 credits because for some reason their system
keeps counting image requests to your total even if you don't get an image.
Original comment by tehvesu...@gmail.com
on 16 Apr 2014 at 10:57
Ah, why you didn't say it to me from the start?
Currenlty working on it. Will add it with some another changes.
Original comment by catgirlfighter
on 16 Apr 2014 at 11:36
Updated.
Later I maybe will add an option to reset limit automatically. Not sure, how to
do it right for now.
Original comment by catgirlfighter
on 16 Apr 2014 at 11:47
It's working.
Thank you very much.
Original comment by tehvesu...@gmail.com
on 18 Apr 2014 at 3:14
Added an option to reset limits automatically as well.
Original comment by catgirlfighter
on 25 Apr 2014 at 3:43
12:44:30 Stop signal sent with a reason: Can not reset Full Size Limit
12:44:30 exhentai.org: Can not reset Full Size Limit
Limit reset doesn't work and it stops grabber completely instead of using the
stop signal.
Original comment by tehvesu...@gmail.com
on 26 Apr 2014 at 6:07
Hm. Didn't test it on exhetai, thought it should be the same as g.e-hentai.
>Can not reset Full Size Limit
means script didn't get confirmation about success. It means an error accured,
and I'm not sure if progress should be able continue.
Original comment by catgirlfighter
on 28 Apr 2014 at 8:44
Found it out. To reset exhentai's limits you should use g.ehentai's home page.
Original comment by catgirlfighter
on 28 Apr 2014 at 9:10
Updated.
Original comment by catgirlfighter
on 28 Apr 2014 at 9:17
Aces.
Original comment by tehvesu...@gmail.com
on 30 Apr 2014 at 5:53
Original issue reported on code.google.com by
tehvesu...@gmail.com
on 8 Jan 2014 at 7:11