search

bakape / hydron

media tagger and organizer
GNU Lesser General Public License v3.0
95 stars 9 forks source link

Requesting communication with HydrusDev #49

Open DonaldTsang opened 5 years ago

DonaldTsang commented 5 years ago

Since there is no complete feature list or UML-esque document of Hydrus online, it would be helpful if you can contact the dev privately or go to the Hydrus Discord server to co-ordinate.

bakape commented 5 years ago

Why would we need that? I don't follow.

DonaldTsang commented 5 years ago

As Hydron is trying to be a faster alternative to Hydrus it would be great if Hydron can match most of the features from Hydrus in order to find ways of optimization.

bakape commented 5 years ago

Optimization and dropping features that hinder performance come hand in hand. I don't see much point in this. If there are specific features you want, feel free to create issues and I will evaluate, if they are viable and how best to implement them.

On 19 September 2018 at 22:04, Donald Tsang notifications@github.com wrote:

As Hydron is trying to be a faster alternative to Hydrus it would be great if Hydron can match most of the features from Hydrus in order to find ways of optimization.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bakape/hydron/issues/49#issuecomment-422921884, or mute the thread https://github.com/notifications/unsubscribe-auth/AHfPsIzCkQAo-vO5If-rJxGvJircOT09ks5ucpU7gaJpZM4Wwtj6 .

DonaldTsang commented 5 years ago

Some of the big feature would be:

  1. Downloader script support Hydrus has parser scripts for 1000+ websites
  2. IPFS and WebUI support (with or without Siderus/Orion)
  3. Tag parents and tag siblings, and downloader support for tags
  4. Thread watcher support for *chan, Reddit/Voat, Twitter etc.
  5. Image deduplication support (both global image hash and SIFT/SURF/pippy360/etc.)
  6. ComicBook support (cbz, cbr, cb7, cbt, cba, downloading from ComicRack etc.)
  7. Video and Audio support (download from *tube, soundcloud, bandcamp, Bitchute)
  8. Social media downloader for Twitter, Tumblr, Blogger, Medium etc.
  9. Ebook (LibGen) and fanfiction support (AO3, FF.net, Wattpad, galleries etc.)
  10. CAD, MS Office and OpenDoc formats, SWF, APNG, WebP, FLIF etc.
  11. Automatic image de-duplication and selection by size/quality/age
  12. Tag/Artist clustering for genre and related artists discovery
  13. Waifu2x training built-in and model import/export
  14. Tag-based automatic file rejection
  15. Searching and sub-searching with AND, OR and NOT conditions
  16. Context-based image sorting and viewing for certain workloads
DonaldTsang commented 5 years ago

For reference https://8ch.net/hydrus/res/417.html

bakape commented 5 years ago

Downloader script support Hydrus has parser scripts for 1000+ websites

Planned support multiple boorus and iqdb. Currently only support gelbooru.

IPFS

What do you have in mind?

WebUI

Already there but not as complete as the CLI yet.

Tag parents and tag siblings, and downloader support for tags

What?

Image deduplication support (both global image hash and SIFT/SURF/pippy360/etc.)

Planned.

ComicBook support (cbz, cbr, cb7, cbt, cba, downloading from ComicRack etc.)

I feel that is out of project scope. Hydron mostly targets file formats that are supported by boorus.

Video and Audio support (download from *tube, soundcloud, bandcamp, Bitchute)

Ditto.

Social media downloader for Twitter, Tumblr, Blogger, Medium etc.

Planned.

Ebook (LibGen) and fanfiction support (AO3, FF.net, Wattpad, galleries etc.)

Out of scope.

CAD, MS Office and OpenDoc formats, SWF, APNG, WebP, FLIF etc.

Ditto.

DonaldTsang commented 5 years ago

@bakape update your list again, also when discussing file formats I would exclusively state a file reader for ebooks, audio, video, text (fanfic), WebP/FLIF, APNG, SWF. CAD and Office/Document files should have a trigger on the desktop to use native software to open

bakape commented 5 years ago

For reference https://8ch.net/hydrus/res/417.html

Link dead.

when discussing file formats I would exclusively state a file reader for ebooks, audio, video, text (fanfic), WebP/FLIF, APNG, SWF. CAD and Office/Document files should have a trigger on the desktop to use native software to open

If it is only opened by external applications, then that would not be any overhead-inducing complexity. I'll see to it at a later time. Working on other projects in the last few weeks.

DonaldTsang commented 5 years ago

@bakape https://8ch.net/hydrus/res/471.html (sorry)

If it is only opened by external applications, then that would not be any overhead-inducing complexity.

Hydrus is made to be a semi-universal file tagger, meant to replace traditional folder systems.

Chiiruno commented 5 years ago

APNG, WebP, FLIF

These would be good to implement IMO. Even if some browsers don't support them, ultimately it's up to the user, and since this is a personal thing, it should be okay to implement. APNG should be obvious, and FLIF is really cool.

bakape commented 5 years ago

Need thumbnailer support first.

On Wed, 26 Sep 2018, 09:05 チルノ, notifications@github.com wrote:

APNG, WebP, FLIF

These would be good to implement IMO. Even if some browsers don't support them, ultimately it's up to the user, and since this is a personal thing, it should be okay to implement. APNG should be obvious, and FLIF is really cool.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bakape/hydron/issues/49#issuecomment-424595644, or mute the thread https://github.com/notifications/unsubscribe-auth/AHfPsKdqtgXbsrl_7793efnnGY2_Ulkiks5uexlAgaJpZM4Wwtj6 .

Chiiruno commented 5 years ago

I'll make an issue for it, I should probably see what I can do about it as far as the Rust version goes, and if it's worth the effort. (e.g.: lack of possible libraries)

bakape commented 5 years ago

Concerning the Rust version, it would probably be less work for you to reuse the existing C code and call it from Rust.

On Wed, 26 Sep 2018, 09:09 チルノ, notifications@github.com wrote:

I'll make an issue for it, I should probably see what I can do about it as far as the Rust version goes, and if it's worth the effort. (e.g.: lack of possible libraries)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bakape/hydron/issues/49#issuecomment-424596331, or mute the thread https://github.com/notifications/unsubscribe-auth/AHfPsMyadDJ7GXFgNhqiOgsqou0vLp2zks5uexohgaJpZM4Wwtj6 .

Chiiruno commented 5 years ago

Depending on how things go, we might as well just use full C, but I'll leave that for the other thumbnailer issue if that does end up being the case.

bakape commented 5 years ago

If you ask me, full C is more practical. You can then use this C thumbnailer from Rust.

On Wed, 26 Sep 2018, 12:10 チルノ, notifications@github.com wrote:

Depending on how things go, we might as well just use full C, but I'll leave that for the other thumbnailer issue if that does end up being the case.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bakape/hydron/issues/49#issuecomment-424643313, or mute the thread https://github.com/notifications/unsubscribe-auth/AHfPsNaGKMmJFPR8Zv7bry36m4EmqOoJks5ue0SAgaJpZM4Wwtj6 .

Chiiruno commented 5 years ago

Well, the whole point of me even doing this in Rust is to git gud at it as well as fix the memory scaling issues. I guess the second point is kind of null since you're doing it in C, but fuck it.

DonaldTsang commented 5 years ago

@Chiiruno I would say that swf, webm and mp4 support is important (see Danbooru and Derpibooru)

Chiiruno commented 5 years ago

Webm is already supported, not sure if MP4 is, but it should be. SWF is a dying format and irrelevant IMO.

DonaldTsang commented 5 years ago

@Chiiruno wouldn't say that as there are too many good stuff in swf

Chiiruno commented 5 years ago

Without a doubt, but there's no point in implementing it in hydron. Can you even extract an image from an SWF with graphicsmagick/ffmpeg?

Chiiruno commented 5 years ago

Seems like it does for some SWF formats. https://stackoverflow.com/questions/9641550/render-swf-to-png-or-other-image-format

DonaldTsang commented 5 years ago

@bakape If I am going to be concrete, include FLIF, HEIF, BPG and SVG as images, and MP4, MKV, WEBM and OGV as video, with MP3, AAC, M4A, OGG, OPUS, WAV and FLAC as music.

DonaldTsang commented 5 years ago

"IPFS" what do you have in mind

To share images and tags with friends or people who are a part of a "private file-sharing club" (/ptg/-style)

Tag parents and tag siblings, and downloader support for tags "What?"

Tag synonyms and tag hierarchies, and also *booru downloaders that download tags along side images.