Restore support for past-supported sites

[GoogleCodeExporter]

Graber version 1 supported Pixiv.net and DeviantArt.com (though I couldn't get 
them to work how I wanted). I would hope for this to be restored.

Particularly, if the Pixiv galleries could be downloaded more like the Firefox 
addon 'Ank Pixiv Downloader', that would be awesome. In particular, I have it 
set like this:
'?memoized-name?'[?pixiv-id?](?member-id?)\'?memoized-name?' - ?title? 
(?illust-id?)
This way it is pre-organized upon downloading.
It would be nice if there was a way to have R-18 rated pictures automatically 
store into a sub-folder... but simply getting the galleries is good enough.

As for DeviantArt, I can't even make it download in the older version. However, 
it would be very nice to be able to download their Gallery and Scraps (I don't 
really care about their Favorites).

I hope this isn't a bother, and that it's a feasible change. Thank you for your 
time!

Original issue reported on code.google.com by JarieSui...@gmail.com on 14 Jan 2013 at 2:22

[GoogleCodeExporter]

Right now i'm working on adding pixiv and deviant to 2.0 version. I hope it 
will be done this or next week.

>'?memoized-name?'[?pixiv-id?](?member-id?)\'?memoized-name?' - ?title? 
(?illust-id?)

It will be possible in 2.0.

>R-18

There is no way to recognise what pics is r-18 on list page, details parsing 
(i.e. page with pic tags) making process too slow. You can use r-18 tag to 
separate results, and download with two staps (first with "-r-18", means 
"exclude r-18", second with "r-18").

Somewhat changed in deviantart template and it's ruined graber's work. Will fix 
when add it to 2.0.

Original comment by catgirlfighter on 15 Jan 2013 at 12:58

• Changed state: Accepted

[GoogleCodeExporter]

Yay! That's great to hear! So excited!

But, since the system often uses tags, it couldn't be able to automatically 
store anything with an R-18 tag in a sub-folder (or just append "R-18" to them)?

Original comment by JarieSui...@gmail.com on 16 Jan 2013 at 2:05

[GoogleCodeExporter]

Ya, nekopaw uses tags, but only where it possible. 
I think I can do something to get tags for pictures, as extra feature 
(disableable because it's resource expensive).

Original comment by catgirlfighter on 16 Jan 2013 at 8:47

[GoogleCodeExporter]

Thanks for considering it! I really only care about the R-18 tag being used 
though (to have them pre-sorted out). ^_^;

Original comment by JarieSui...@gmail.com on 16 Jan 2013 at 7:31

[GoogleCodeExporter]

Done with pixiv. How i can see it's bugged some (can throw some critical 
errors), but anyway it's working. Right now it download only from search page. 
Will do user-works and user-favorites tomorrow.

Use flag "get tags" to get tags and r-18 rating for pictures.

Pixiv have field "rating", and you can use it in name format. Something like 
this:

$rootdir$\pics\$rname$\$tag$\<%rating%\><%album_id%\>

Right now wiki not so friendly for non-russians. Think to translate it 
someday...
I'll fix all bugs as soon as possible. Working on going to betta, need to fix 
alot :)

Original comment by catgirlfighter on 17 Jan 2013 at 6:11

[GoogleCodeExporter]

Added user-works and user-favorites modes.

Will try to release deviantart next week, but it's not so easy. Need to get 
something about categories selection to avoid 10000 pic limit.

Original comment by catgirlfighter on 18 Jan 2013 at 2:52

[GoogleCodeExporter]

Well, Pixiv works if you search only by a tag, but I can't figure out how to 
have it target a specific Artist. I've copied the artist page URL (ex: 
http://www.pixiv.net/member.php?id=1877631) and their Gallery URL (ex: 
http://www.pixiv.net/member_illust.php?id=1877631), but neither works.

Something I just noticed: the changelog opens as gibberish. The English text is 
fine, but the rest is scrambled (opened either via the program or Notepad).

Original comment by JarieSui...@gmail.com on 22 Jan 2013 at 9:02

[GoogleCodeExporter]

Oh, also, if it is a 'manga'-style entry, it only downloads the very first 
image.
ex: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=32518542

Original comment by JarieSui...@gmail.com on 22 Jan 2013 at 9:08

[GoogleCodeExporter]

I think you're not noticed "+" near row?

Original comment by catgirlfighter on 23 Jan 2013 at 5:04

[GoogleCodeExporter]

Catched a bug, will fix it soon.

Original comment by catgirlfighter on 23 Jan 2013 at 7:03

[GoogleCodeExporter]

>>7
>Well, Pixiv works if you search only by a tag
It works like in old version, maybe you're not seen it. Select pixiv in list 
and set "Use user ID" checked, then put user id and select source (works or 
favorites).

>Something I just noticed: the changelog opens as gibberish.
Haha, this is it for you. It uses ANSI codepage (WIN-1251) and uses another 
language. Oh, I think I just will start translating from versionlog.txt.

Original comment by catgirlfighter on 23 Jan 2013 at 9:06

[GoogleCodeExporter]

>Catched
>translating
Ok, maybe I have to improve my English.

Original comment by catgirlfighter on 23 Jan 2013 at 9:16

[GoogleCodeExporter]

... -_-; Wow. I totally didn't notice that I could click on the Pixiv button 
there to bring up a more advanced search...
Ok, so I can now get an artist's gallery. >_<

However, the manga-pages don't seem to work right. I can't get the Expand (+) 
option to appear until I've downloaded the image, but after that, it has the 
sub-images each marked with 'Skip', and won't change that or download them. If 
I delete the downloaded image, then select only one of the sub images, it will 
then be able to download that one, but only with the same name as the first 
image (it won't save any others of that page, and doesn't save them as (ex: 
32518542_big_p0.png), but as the default, as though it were the only image 
there.
Can this be fixed so it saves the bunch as a folder that gets the 
custom-naming, with the images inside just named 01, 02, etc? That's how Ank 
Pixiv Downloader does it, and I think it is the most efficient method...

This also came up, don't know why (doesn't bother me, unless it actually 
becomes a usage issue; I'm just curious. I haven't seen it since the first 
time).
Error: Could not convert variant of type (UnicodeString) into type (Boolean)

Original comment by JarieSui...@gmail.com on 25 Jan 2013 at 12:18

[GoogleCodeExporter]

Right now I'm too busy, but on this weekend I'll fix a bug what I caught, hope 
it solve your problem.

Can you write your name format string? Maybe there is an error in it, and I can 
give some advice while the wiki not translated.

>images inside just named 01, 02, etc? 
For it you can use $fnn$ key. To format it like 00001 use a formating like 
$fn:.5u$.
What it's means:
[.Precision]Type
.4 - count of numbers (zeros) in string, for flaotpoint it's a zeros AFTER 
point, for another is a LEADING zeros.
u - is UNSIGNED type, number type without a point and negative sign.

I recommended to use string like this in your situation:
YOUR PATH\$tag$\<%rating%\><%album_id%\>$fnn:.5u$

<> symbols means isolation of a strings with a keys that should not be used if 
empty.

Original comment by catgirlfighter on 25 Jan 2013 at 10:04

[GoogleCodeExporter]

Ok, I've ASLMOST got it how I want...

LOCATION\'%author%'[%author_login%](%author_id%)\<%rating%\>'%author%' - 
%title% (%id%)<(%album_id%)\>$fnn:.5u$

(I separate parts both for organization, as well as because it works better/at 
all with Paint Shop Pro V, my preferred image program; yes it's ancient, but 
even new Paint Shop doesn't like Unicode...) Anyway.

The problem left: I can't get it to treat an album and a non-album page 
differently. I want 

I tried putting <%album_id%\$fnn:.5u$> at the end, but that just made even 
non-albums become a folder with "%album\00001" (one image) on it, as well as 
still having the double ID for Albums.

Is it possible to make an if/then situation, where if there is an Album ID it 
places it instead of ID and then puts the images as a numbered set inside the 
sub-folder. kind of like this (though I know it doesn't work that way):

...\{<(%id%)>OR<(%album_id%)\$fnn:.5u$>}
 (putting the {} just makes it not have an error and not do anything.)

Original comment by JarieSui...@gmail.com on 25 Jan 2013 at 6:20

[GoogleCodeExporter]

Yeah, right now one key can't replace another key if it (not) exists. But will 
be good idea to add it. Added to "to do" list.

Original comment by catgirlfighter on 27 Jan 2013 at 5:05

[GoogleCodeExporter]

^_^ Yay! I'll have to wait for my bulk-downloading to start until some way of 
it working comes forth, because I don't want to have to re-download so many 
files, and then manage to re-merge them...

(I'm going to keep hoping for Albums to be auto-folder using the file-name 
layout, like Ank Pixiv Downloader... ^_^ )

Hm, have you used that before? If not, maybe you could get a couple easy fixes 
from how it works? (Sorry, I don't know much about programming)

Original comment by JarieSui...@gmail.com on 27 Jan 2013 at 11:24

[GoogleCodeExporter]

Pixiv changed the searchpage layout. Fixed it, maybe will be more changes soon, 
as I know the pixiv devs.

Cheated a bit with a name format. I changed rules for $$ in <>, now it become 
visible only if %% in <> is not empty. And I removed %id% and %title% from 
childs (in albums). Now you can use this without troubles:

LOCATION\'%author%'[%author_login%](%author_id%)\<%rating%\>'%author%' - 
<%title% (%id%)><%album% (%album_id%)\>$fnn:.5u$

And fixed bug with downloading many albums in the same time (1 picture from 
album and repetitive erros).

Original comment by catgirlfighter on 28 Jan 2013 at 2:53

[GoogleCodeExporter]

>>18
Oh, i mean

LOCATION\'%author%'[%author_login%](%author_id%)\<%rating%\>'%author%' - 
<%title% (%id%)><%album% (%album_id%)\$fnn:.5u$>

of course.
BTW, you can use %page:.5u% instead of $fnn:.5u$.

Original comment by catgirlfighter on 28 Jan 2013 at 2:56

[GoogleCodeExporter]

It works, as far as I've tested! >_<

However, Author and Title are now coming out blank.
I assume, that is from the Pixiv changes? I know Ank Pixiv DL has to update 
every so often to work with them as well. (I wonder how it changes to make that 
happen?)

Original comment by JarieSui...@gmail.com on 28 Jan 2013 at 8:14

[GoogleCodeExporter]

Not sure what happen, but new pixiv script just not added to update. Fixed it.

Original comment by catgirlfighter on 28 Jan 2013 at 8:53

[GoogleCodeExporter]

Hm. Didn't work, still blank. They don't even show in the program.

Original comment by JarieSui...@gmail.com on 28 Jan 2013 at 8:59

[GoogleCodeExporter]

And now fixed author + title from the userworks page.

Keep looking on pixiv layout updates.

Original comment by catgirlfighter on 28 Jan 2013 at 9:09

[GoogleCodeExporter]

I think i'm broken updater in this version. Soooo bad x)

Original comment by catgirlfighter on 28 Jan 2013 at 9:24

[GoogleCodeExporter]

So, temporary you can just redownload updater from Download page, but you need 
do it again after that horrible bug will be fixed. Can't do it now because all 
source in another place :(

Original comment by catgirlfighter on 28 Jan 2013 at 9:34

[GoogleCodeExporter]

^_^; I think I'll wait for your next update then. Looking forward to it! >_<

Original comment by JarieSui...@gmail.com on 31 Jan 2013 at 1:34

[GoogleCodeExporter]

Oh, next update released next day after I found a bug. Sorry about I'm not 
notified you.

Original comment by catgirlfighter on 31 Jan 2013 at 8:32

[GoogleCodeExporter]

Oh, wow! >_< It works perfectly! (As far as I can see)

Oh, and has there been any advancement on DeviantArt? I see it's still not 
listed in the program.

Original comment by JarieSui...@gmail.com on 31 Jan 2013 at 8:49

[GoogleCodeExporter]

Yeah. I'm just working on it. Free time is never enough :)

Original comment by catgirlfighter on 1 Feb 2013 at 10:42

[GoogleCodeExporter]

I know what you mean. Life gets in the way of fun. ^_^;

Cool! I'll wait for the next update and check it out.

Original comment by JarieSui...@gmail.com on 2 Feb 2013 at 1:31

[GoogleCodeExporter]

Done with deviantart, must work just like in old version, excluding in 
categories list you need to put URLs (not short paths).

Original comment by catgirlfighter on 7 Feb 2013 at 2:27

[GoogleCodeExporter]

First test: login.

17:00:26 deviantart.com error: Could not load SSL library.
17:00:26 login is failed

Original comment by JarieSui...@gmail.com on 8 Feb 2013 at 12:01

[GoogleCodeExporter]

It's normal. You need to install the OpenSSL library to access.
In Win7 it's preinstalled, but as I understand you're using another OS?
You can download the library from
https://code.google.com/p/nekopaw/downloads/detail?name=OpenSSL_Lib.zip

Original comment by catgirlfighter on 8 Feb 2013 at 8:32

[GoogleCodeExporter]

Actually, I am using Win7... (I would prefer my XP computer, but I've been 
using this one lately).

Hm, I'll try getting that and see what happens.

Original comment by JarieSui...@gmail.com on 8 Feb 2013 at 4:00

[GoogleCodeExporter]

Ok, logged in ok. ^_^;

So, I can see the user's gallery there, but the only info it seems to get is 
the Author. No Title. Also, I think DeviantArt has a kind of 18+ rating system, 
will the Rating tag work with that? (I haven't tested, just wonder if that 
should work).

Original comment by JarieSui...@gmail.com on 8 Feb 2013 at 5:12

[GoogleCodeExporter]

You're writing in "User name" the author and a title in "Tags". It will work.
If you're not logged in, then R18+ will not work (DArt not allowed it), but if 
you're authorised, and in profile settings R18+ is enabled, it will work 
(settings->browsing->mature content).

Also it can download some "private" pictures (with lock on preview) because it 
visible in search results and not protected from a direct access.

Original comment by catgirlfighter on 8 Feb 2013 at 6:30

[GoogleCodeExporter]

Also, Grabber at first trying to download resource using "download" feature 
("download" button on resource page), because of it you can download FULL SIZE 
pictures, FLASH, MUSIC and TEXTs, if it available to download from resource 
page. If it not available, Grabber can download only BIG and SAMPLE sized 
pictures, all another will be skipped with "can't get url" error.

Original comment by catgirlfighter on 8 Feb 2013 at 6:36

[GoogleCodeExporter]

Um, so, my ideal download would be:
ROOT\'%author%'\<%rating%\>'%author%' - %title%
Would that work? R-18 should get sub-folder? But... it doesn't load the Title.

ROOT\'%author%'\<%rating%\>$fname$
If I do this, then I get the filename, which works, but there's a number of 
pictures that don't have filenames that include their title (not sure why that 
happens).

So, you're saying that it first tries to access the image via the Download 
button, then moves to secondary locations (such as the big image you see on the 
image's page)?

Original comment by JarieSui...@gmail.com on 8 Feb 2013 at 7:24

[GoogleCodeExporter]

Oh, well, cought 1 bug (if you're not set categories, even simple 
"http://browse.deviantart.com/" it will get nothing in "by keywords" mode).

Actually it's realy don't get title, missed it somehow.

It can't get a filename if it's unknown (usually for flash/text and other non 
image stuff, it's havent direct urls to the source). Grabber will get it after 
check the "download" url (before download picture).

Anyway, please give me some sample sources of a bugs you're got (tags what 
you're searching for).

I'll fix it tomorrow.

>So, you're saying that it first tries to access the image via the Download 
button, then moves to secondary locations (such as the big image you see on the 
image's page)?
Yap. All pictures have another URLs to it on result page except preview, it's a 
BIG and SAMPLE URLs. If preview havent it then Grabber can't get a filename in 
a searching mode, and grabber will try to get it in download mode when getting 
URL from the "download" button.

Original comment by catgirlfighter on 8 Feb 2013 at 10:34

[GoogleCodeExporter]

>ROOT\'%author%'\<%rating%\>'%author%' - %title%

Care about duplicates with that fileformat (title may be repeated).
$fn$ key must solve this problem, but right about now it always hidden by <>, 
and work only without isolation.
In fact, if you're isolate the <$fn$> key, it must be visible only when a name 
is duplicated. I'll fix it with other stuff tomorrow :/

$fn$ works like $fnn$, but grows only when the filename is duplicated.

Original comment by catgirlfighter on 8 Feb 2013 at 10:43

[GoogleCodeExporter]

Fixed a %title% for the deviant.
Partialy fixed a $fn$ key. There is more troubles with "dinamic" filenames in 
$fname$, like %filename%.%ext%. Need to fix alot in a threads job.

Original comment by catgirlfighter on 9 Feb 2013 at 6:55

[GoogleCodeExporter]

I don't think there would be many (any?) name issues, since I'd be having them 
save to the artist's name folder, and I doubt they would name multiple works 
the same way.
But, doesn't the Rules option in Settings take care of that? I haven't tried 
testing it, but could it append a [1] to duplicate file names? Optionally, I 
guess?

Actually, since it takes the filename for the title, duplicates should never 
happen anyway. ^_^

Oh, I just noticed: Graber identifies the Category that the image is listed in, 
but doesn't seem able to include that in the download.
I used:
ROOT\'%author%'\<%rating%\>%category%\%title%
But it just put's "%category%" as part of the directory (a foldername, in this 
case).
It's not of much importance to me, but could be useful? Not sure how that would 
come out, since they can have the Category have subcategories as well.

Original comment by JarieSui...@gmail.com on 9 Feb 2013 at 9:02

[GoogleCodeExporter]

>ROOT\'%author%'\<%rating%\>%category%\%title%
Actually it's just because a %category% in the DArt called %categorypath%. 
Changed it to a %category% to avoid confusin.

Original comment by catgirlfighter on 9 Feb 2013 at 9:44

[GoogleCodeExporter]

Cool. Hm... when testing it, I noticed that I can (sensibly) either use 
ROOT\highcategory\lowcategory\ or ROOT\category... but neither make quite right.

(Sorry that I keep throwing more things at you...)

example:
username: Lunoz (currently has only 2 pictures, so easy to test with)

The Category for one is: manga\digital\3d
But, if you download with ROOT\category it comes out like this:
\manga_digital_3d\

If you use the high and low, it becomes:
\manga\3d\

So either you have a complete layout, but as one folder for each Category 
combination or else only the High and Low categories. I don't see a way to get:
\manga\digital\3d

Might that be possible and I'm just missing it?

Original comment by JarieSui...@gmail.com on 9 Feb 2013 at 11:28

[GoogleCodeExporter]

As default Grabber change all special symbols in a name to a safe "_" symbol to 
avoid any troubles with saving files.
I'll exclude "\".

Original comment by catgirlfighter on 10 Feb 2013 at 2:29

[GoogleCodeExporter]

So awesome! It works!

I think... that I seriously can't think of any other additions for those sites. 
You are amazing!

If I do think of anything, or find a bug or whatever, I'll let you know.

Thank you again for all your hard work! If there is anything I can do to help, 
just let me know!

Original comment by JarieSui...@gmail.com on 13 Feb 2013 at 10:12

[GoogleCodeExporter]

You welcome. Just released past-supported resources :)

Original comment by catgirlfighter on 13 Feb 2013 at 10:16

• Changed state: Done

[GoogleCodeExporter]

Oh... just thought of something. -_-;

Can I have it set to download more than one artist's gallery? Or even save them 
so I could have it just download the newest pictures for multiple artists 
(assuming I haven't deleted any, then they'd just be re-downloaded)?

Original comment by JarieSui...@gmail.com on 15 Feb 2013 at 5:21

[GoogleCodeExporter]

Right about now you can't, but it's in "to do list"... last half year. Idea of 
it it's multiple adding of the same resource to the list.

Original comment by catgirlfighter on 18 Feb 2013 at 8:32

[GoogleCodeExporter]

>Or even save them so I could have it just download the newest pictures for 
multiple artists.

It's in "to do list" too, must work like in old version, but before need to 
release a list saving feature.

Original comment by catgirlfighter on 18 Feb 2013 at 8:35