search

Xealeph / Yiff.party-Image-Scraper

A simple Image Scraper for yiff.party profiles.
MIT License
10 stars 4 forks source link

Download post as text #15

Open HydroHydro opened 4 years ago

HydroHydro commented 4 years ago

Many Patreons have links in their posts leading to 3rd party downloading sites. For example, an artist might link to a Mega.nz file, Vimeo file, or even a gfycat.com file, etc etc etc.

I assume adding a downloader for all cases would be too much. So perhaps a link detector instead? If a link is detected in a post, it could be put into a text file in the same Patreon download folder. That way a user could download the links themselves without having to scroll and check every single post.

One issue though: Many links (like Vimeo videos) are password protected. Either a "password detector" would be needed or maybe we would need the option to download all the posts as text files?

Xealeph commented 4 years ago

Good suggestion! Since I'm a bit busy at the moment, I will implement the Link-Detector tomorrow or the day after. The password issue on the other hand might take a bit longer, since I need to figure out a way to make it work. It would be a great help if you could provide an example of a yiff.party profile, where you encountered this issue. Otherwise thank you for helping improving this project. :)

HydroHydro commented 4 years ago

This one has a lot of links to 3rd party download sites: https://yiff.party/patreon/8049777

This one has password protected Vimeo links, but this creator will be deleted tomorrow: https://yiff.party/patreon/13186581 I've noticed that most Patreons either use "Pass:" or "Password:" in their posts. Not sure how helpful that is.

Another issue I found: https://yiff.party/patreon/3175689 If you find the posts titled "New Intro" or "Asset Swap Mix and Match", they have embedded Vimeo videos. This means that they are not password protected, BUT they will only play when embedded. They will NOT play if you try to go to their Vimeo URL. Not sure how or if you can solve that issue.

Xealeph commented 4 years ago

Thanks a lot! I'll look into it.

Xealeph commented 4 years ago

I added the feature to save 3rd party links into a text file in the folder of their respective author. I also tried to tackle the issue with the embedded videos (the Vimeo ones), but hit a wall regarding how to implement this feature. To work around this, I added the embedded links to the 3rd party links text file. This said, it'll probably take a bit longer to make it work, so you might have to wait a few additional days depending on how much time I can invest each day.

HydroHydro commented 4 years ago

Thanks! I'm just happy to have a good working Yiff downloader! Take your time! :)

Xealeph commented 4 years ago

I now added support for embedded Vimeo links. One issue with this is, that some Vimeo links don't get recognised properly and thus won't get downloaded. I'm not sure why. To work around this, the non-working links get appended to the 3rd party links file like before. I might look further into it, as to why this is happening, but I wanted to at least provide some kind of embedded video support.