[Feature Request] Hydrus Network Support

[seniorm0ment]

Requesting support to connect to a Hydrus Network client. https://hydrusnetwork.github.io/hydrus

There is already a client api https://hydrusnetwork.github.io/hydrus/help/client_api.html

[NO-ob]

Will work on it got a bunch of uni work until january tho so might be a while

[seniorm0ment]

Alright, good luck!

[NO-ob]

Just from having a look at the api docs im not sure if hydrus actually hosts images at a url the app can access. I'll have to set it up on my machine and see at some point

[seniorm0ment]

Not sure if I entirely understand your question, but I believe it does. Just wanted to note, on the local network it would simply connect via 192.168.1.xx:hydrusport, however as stated for remote connections the user needs to setup ssl.

I don't see why Lolisnatcher would have issues reading from the api. You can utilize any of the frontends they list and connect via the same way. Looks like Animeboxes added support recently too according to the hydrus docs. So I can't imagine issues.

https://hydrusnetwork.github.io/hydrus/help/client_api.html

[NO-ob]

Done for images cant get video to work atm will try another time

[seniorm0ment]

Thank you so much

[seniorm0ment]

I can't seem to get this to connect, Hydrus doesn't detect the API request. However I tried doing it manually, and copying the access key from hydrus and inputting it, and it seems to still not want to connect.

Edit: Restarted the client api on hydrus. It's grabbing the api key now, however still having issues with the below saying no data returned. Hm.

I think it might be related to the address, not the api key. It's only on my lan (which I'm on on both devices). 192.168.1.xx:45869, also tried adding http://. If I don't add http:// it automatically tries to add https://, and I see no error when clicking test, however it doesn't seem to save when backing out to main menu so I don't think it worked? It shouldn't be using https though. if I use http://, it says no data returned.

If I try on the web browser on my phone, I can reach the Hydrus api page with the ascii art no issues.

[NO-ob]

Are you on the newest version of hydrus? i'm not sure why it wouldn't be working as it works fine on my system. I have these setting in the client api page

[seniorm0ment]

Not at my computer at this specific moment, but I am pretty sure mine looks like that.

I also again, can reach the client api via my browser on phone. Could you happen to provide a screenshot of loli snatcher settings that work for you just to ensure.

I'll double check that there's no new updates, I did just upsate Hydus 2 days ago,

Edit: yeah that's what my manage services looks like, and I'm on Hydrus 424 which is the latest. Looks like I wasn't the only one having this issue though.

[NO-ob]

Heres the screenshot, could you try doing this also when you have time

Install this build it logs a bunch of stuff: https://z.zz.ht/I55sO.apk

You will need to enable android debugging on your phone also

Open the app and go to the page to add the booru, fill in the information for hydrus and get the api key

before hitting test open the terminal on your computer and run adb logcat | grep flutter > log.txt

then hit test and wait for it to say no data is found then ctrl+c on the terminal and put the log here

[seniorm0ment]

https://bin.snopyta.org/?ff068f3e2b973887#FxPNxFfnmsonNthmbcjZByFmhoDEFfmxMAFkVjbBLKhc pass: lolisnatcher

Flutter output after hitting test and getting that error. I didn't input or grab the api while in the app though, just hit test after connecting to adb.

I was grabbing the logcat from adb, lemme read through the reply Edit: that logcat wasn't from the test build, hope that's okay. Also that screenshot is accurate to mine.

It looks like it might not be recognizing Hydrus? I see nothing referencing Hydrus in the logs, except the ip dump at the bottom. But I see nothing regarding hydrus.

[NO-ob]

https://bin.snopyta.org/?ff068f3e2b973887#FxPNxFfnmsonNthmbcjZByFmhoDEFfmxMAFkVjbBLKh pass: lolisnatcher

Flutter output after hitting test and getting that error. I didn't input or grab the api while in the app though, just hit test after connecting to adb.

I was grabbing the logcat from adb, lemme read through the reply Edit: that logcat wasn't from the test build, hope that's okay. Also that screenshot is accurate to mine.

It looks like it might not be recognizing Hydrus? I see nothing referencing Hydrus in the logs, except the ip dump at the bottom. But I see nothing regarding hydrus.

I'm getting cannot decrypt data on that page, also it needs to be the apk i posted because the release one doesn't have any of the logging stuff

should output something like this

01-10 13:34:50.028 17271 27496 I flutter : http://192.168.1.10:45869/get_files/search_files?tags=[]
01-10 13:34:50.066 17271 27496 I flutter : ------------------Hydrus Search---------------------
01-10 13:34:50.066 17271 27496 I flutter : Base url: http://192.168.1.10:45869
01-10 13:34:50.066 17271 27496 I flutter : Search url: http://192.168.1.10:45869/get_files/search_files?tags=[]
01-10 13:34:50.066 17271 27496 I flutter : API Key: 8206e61e40ab1d0441938933233fc74951f938d40871fc9c7568851752eed330
01-10 13:34:50.066 17271 27496 I flutter : Fetching file ids - Status: 200
01-10 13:34:50.066 17271 27496 I flutter : {"file_ids": [5, 6, 2, 3, 4, 1]}
01-10 13:34:50.083 17271 27496 I flutter : ------------------Hydrus Search---------------------
01-10 13:34:50.083 17271 27496 I flutter : Fetching file metadata - Status: 200

[seniorm0ment]

https://bin.snopyta.org/?ff068f3e2b973887#FxPNxFfnmsonNthmbcjZByFmhoDEFfmxMAFkVjbBLKhc Should work, just tested it. Ensure pass is all lowercase.

Gimme a few min on the apk Do you have another link to the apk, can't seem to access that page.

[NO-ob]

app-arm64-v8a-release.zip

[seniorm0ment]

https://bin.snopyta.org/?1ad3dd68612a2df8#DnNeZm4FtxG7pDZ8uc3bDvaeHXLx9mzxoNvmHMK5MSnK Here's the debug apk log. Same pass

[NO-ob]

https://bin.snopyta.org/?1ad3dd68612a2df8#DnNeZm4FtxG7pDZ8uc3bDvaeHXLx9mzxoNvmHMK5MSnK Here's the debug apk log. Same pass

weird i think its finding your files okay, also have you cut the log I need to see the bit after

------------------Hydrus Search---------------------
01-10 13:34:50.083 17271 27496 I flutter : Fetching file metadata - Status: 200

[seniorm0ment]

https://bin.snopyta.org/?c42ccc0e6c69bb16#HGPrJLKu2Wt8j2bg4zLkbu1DZJkq2DdxTtsG4Ji4gHno Same pass

I didn't cut anything out (intentionally) in the original, but I just redid the logcat. I grepped Flutter lines. I'm not seeing anything below that Status 200 line, that's missing in my logs without grepping.

Also note when reading, there's multiple search lines, that may say the api key wasn't found. I hadn't clicked apply on hydrus, the later hydrus search blocks are the working ones.

I'm not seeing anything noticible either though. Could it be an issue with Hydrus simply saving the input? Instead of saving it's just throwing the error maybe even though the input is correct?

[NO-ob]

oh okay i assumed you was trimming the log, its getting file ids but not the metadata so i'm not sure what wrong ill make an apk with more logs

[NO-ob]

app-arm64-v8a-release.zip I've added more prints can you post the log for this apk plox

[seniorm0ment]

https://bin.snopyta.org/?ecd455519ed23269#45ExvVUFvJ6wLatgri4R3Vow2mBQ2koA8dQTrRXayWW4 Same pass

Not sure why there's 2 different access keys appearing, it's a fresh log and only requested the api key once. The one that matches with hydrus starts with d7100. Maybe that's the issue?

[NO-ob]

It should be fixed now, i'm not sure about the 2 api keys lel. Also the issue is that hydrus returns thousands of file ids in one go and you have to put a list of them in the url to get the metadata for those files, I assume that the url was too long and it jsut wouldn't work. https://github.com/NO-ob/LoliSnatcher_Droid/releases/tag/1.7.0

[seniorm0ment]

Ah yeah that makes sense, I'll try it in a bit thank you

[seniorm0ment]

I grabbed the hotfix, and it is still throwing the error at me.

I assume you set a limit on the file ids in the fix? Was curious if it works foe you because I have like 60k files in Hydrus, but that may not be the issue if you limited it.

https://bin.snopyta.org/?88dc51ed89866285#6P36Y5d6f5LgP5nzma6u549jb7o2HWxku53stExQKXza Logcat from the hotfix apk, same pass

[NO-ob]

I added like 100 files to my hydrus and changed the hydrushandler to scroll through the array in batches of your limit, I have made it so its limit is changed to 20 if you have higher though as i'm not sure what the max length will be so think 20 is probably a safe number, The changes worked on my machine but it didn't work for one anon so I had to change the identifier for tags which worked on mine was "my tags" but that didn't work for them so it's been changed to "all_known_tags". I think it might be messing up on grabbing tags, heres an apk with some more more prints app-arm64-v8a-release.zip

[seniorm0ment]

That release is working for me, do you still want the logs from it? That's just the debug apk correct, or is it an actual new hotfix?

However it doesn't seem to want to grab more than 20 or so images without running a search. It says it is loading new pages but nothing new loads.

Also it seems as if it is only grabbing searches and files from my inbox, not also the archived ones. Or maybe not idk because I have far more than 20 in my inbox. The results are just extremely incomplete. Even when searching.

Edit: here are logs https://bin.snopyta.org/?b7c52fe94e1b2dc2#swzNBXGevm9hXukZA4qdWqJzXZNvLJP4KjCDmfCfzy7

[NO-ob]

If you got the hotfix apk when I originally posted yesterday that one was also broken i posted another a bit later,

Okay one problem i'm seeing is that on your output all known tags is showing as "all known tags: {1:" but on my searches ive been getting "all known tags: {0:" so i'm checking for ["all known tags"]["0"]. I will probably have to make it check for 1 aswell if 0 is null.

For the archived files I can see there is flags system_inbox : true or false (optional, defaulting to false) system_archive : true or false (optional, defaulting to false)

Will enabling these result in a bunch of duplicates? On my setup I was getting all results with these at the defualt value of false but I dont use hydrus so the files were only added before testing. I'm not really sure about how it groups files or whatever

[NO-ob]

app-arm64-v8a-release.zip Heres another apk

[seniorm0ment]

System inbox is where stuff first goes, it's meant as a place to sort and tag and do whatever, then you archive the files (or delete). It should not pull duplicates unless I misunderstood your question.

It could make sense if you were getting complete results, it might be only grabbing the inbox by default, I don't know where else it would grab from besides the inbox or archive.

Lemme check out the apk

[seniorm0ment]

Ok so, that apk seems to work! So it looks good besides one thing I tried, so there are system: tags that Hydrus has that don't seem to grab. So system:inbox should present my inbox files, system:everything would present all archived files, system:archive would present archice files, system:modified time, etc. There are a bunch more.

These do not seem to fetch anything for me. I figured because I had so many files in archive it was having trouble fetching thousands, but my inbox is only 150 files currently and it doesn't fetch with system:inbox as expected. So it seems to be the system: tags.

Besides that it seems good.

I know you suggested video support in the future, besides that I would suggest a few other things relating to hydrus. The ability to add/manage tags when browsing hydrus (maybe also hold to select multiple files then apply a tag to all), and also the ability to download to hydrus (so if you click download on a booru, you can choose if you also would like to send it to hydrus, if hydrus is only on lan and you are remote, it would cache then send them over when it achieves a connection to hydrus).

Just some suggestions. Appreciate it though, glad I can browse my library now. I'd suggest you play around with hydrus if you have lots saved locally. It's really nice.

[NO-ob]

I've just tested in browser by searching for system:inbox and system:archive and nothing returned so i doubt the other variations of that will work, I could add a check so it changes the url if the tag "system:inbox" or "system:archive" is searched by settings the booleans in the url but I dont think the api supports searching those. You will probably have issues if tags have spaces in them too i will probably have to move to use commas or something as a seperator in a future update

I will have a think about managing tags as I think its outside of the scope of what I want the app to do but downloading to hydrus might be a feature I will add

Also i have thought about setting it up but ive got a whole bunch of files id have to tag so if i was gonna do it id delete all my files and start from scratch lel

[NO-ob]

Ive put a build on the releases page, you might wanna grab that as it has none of the hydrus logging

[seniorm0ment]

Weird, maybe those tags will be supported in the future hopefully.

Also yeah I did notice the space issue I believe, hence why some tags weren't providing results despite having that tag. That explains that.

I completely understand tag and or file management being out of the scope of the project.

Also files do automatically fetch tags, so there is a massive public repository that does hash matching and if it's detected then it will grab tags automatically. So not only from boorus, but also say a chan thread that another Hydrus user tagged an original photo and commited those tags to the Hydrus repository. It is very neat.

Also thanks, Will grab it.

[NO-ob]

Oh I didn't know that it auto tagged files think ill check it out then as that will make things much easier

[NO-ob]

I fixed the videos not working check releases

[seniorm0ment]

Awesome! Looks to work, thanks!

[seniorm0ment]

I had tested it and it was working, providing results no issues. I was using it on 1.7.0 for a bit no issues, then when you released 7.1 I update and saw videos and was able to search.

Just went back and openee the app and now the results seem to show and not show. When it does show, it's in no way my entire library at all.

If I just click search without searching, it shows nothing, travel through the menus and back or reopen the app and hit search again, they show.

If I run a search, very little to no results.

If I go to Hydrus in settings, and click test it is throwing me an error again.

Lmk if you want logs, I can grab them in a bit or tomorrow. thanks

Edit: 1.7.0 still works. Just rolled back to it. I then tried upgrading to 1.7.1 and it caused issues. So it is a 1.7.1 issue.

Also did you revert back to the loading icon of Kana from the percent circle? Because if not, maybe there's an error. Didn't see you note that in your release notes so maybe something happened.

[Zweibach]

I've just tested in browser by searching for system:inbox and system:archive and nothing returned so i doubt the other variations of that will work, I could add a check so it changes the url if the tag "system:inbox" or "system:archive" is searched by settings the booleans in the url but I dont think the api supports searching those.

Yeah, this is unfortunately the case. Hydrus API don't support searching for system predicates as of this time.