mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.96k stars 975 forks source link

Correct procedure to save Deviantart favorites json information? #3687

Open jherazob opened 1 year ago

jherazob commented 1 year ago

Hi! Quick query: From what i've been able to see, downloading from Deviantart even with metadata:true will not save the actual metadata anywhere, just the image. Since Deviantart "wisely" decided to remove favorites search, which is crippling for those of us that have been on the site for years and have thousands of them, decided to make a local database of this i can use to search. Gallery-dl is the tool of choice for it, using the json output which has all the info i'd need.

So, from what i've been able to gather, if i ask it to download my favorites with the json parameter, it should just throw the output of several thousand json files (after a long while of course). That seems impractical.

Is there a way to coax it to print individual json files for everything in my favorites? Like, do a first round gathering only the URL of each entry which would be saved somewhere, then use that list to get the json info for each one?

Is it even possible to retrieve ONLY the url?

It would be even better if i could tell it "Ok, just give me url, author username, date, description and title", but since there isn't i'll have to settle for processing the individual json files.

jherazob commented 1 year ago

Partially answering myself, --write-metadata --write-info-json will write json files with all the metadata for each image. Still haven't found a way to only write the metadata and not the images themselves, although worst case scenario i could just let it do it's thing and then get the info from the json files. Then periodically run it again to get only the newest favs (Ctrl-C once it gets to the old stuff to avoid wasting time and queries). Hopefully it'll work to save the whole thing, seen mentions of people with far more than me but hope 6K favs won't be too much.

ClosedPort22 commented 1 year ago

This saves metadata to individual JSON files:

gallery-dl --no-download --write-metadata -o original=false -o metadata=true https://www.deviantart.com/USER/favourites

Explanation: -o original=false: don't ask DA for download URL. Download the preview file instead --no-download: don't download any image files -o metadata=true: provide description field

It would be even better if i could tell it "Ok, just give me url, author username, date, description and title", but since there isn't i'll have to settle for processing the individual json files.

Check out the documentation: https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#metadatacontent-format

You can use "open": "a" to save metadata in a single file.