lovasoa / dezoomify-rs

Zoomable image downloader for Google Arts & Culture, Zoomify, IIIF, and others
https://dezoomify-rs.ophir.dev
GNU General Public License v3.0
679 stars 62 forks source link

dezoomify-rs adding ending/trailing end to url #174

Open ymerouani opened 2 years ago

ymerouani commented 2 years ago

Hello! So I am working on trying to make dezoomify-rs work with ancestry.com images. I have thus far figured out that they use DZI for their zoomifyer, and of course also cookies is a must since its behind a login. And just figured out the header option for dezoomify-rs, which is perfect. But annoyingly, you need to add a unique security key to the ending of each image URL to pass security check of ancestry.com. URL example:

URL: https://mediasvc.ancestry.com/v2/tile/namespaces/62010/media/62010_b952499-00324.dzi
Should be URL: https://mediasvc.ancestry.com/v2/tile/namespaces/62010/media/62010_b952499-00324.dzi?securitytoken=xxxxx

Tile example

So for this tile: https://mediasvc.ancestry.com/v2/tile/namespaces/62010/media/62010_b952499-00324_files/12/3_0.jpg
It should be: https://mediasvc.ancestry.com/v2/tile/namespaces/62010/media/62010_b952499-00324_files/12/3_0.jpg?securitytoken=xxxxxx

So in short, I'm wondering how to go about editing the program to account for the security key in the url. Im guessing that Custom yaml may be the way to go? But any concrete help would be appreciated!

lovasoa commented 2 years ago

You could try to integrate something like rlua in the custom yaml dezoomer to let the user compute their own URL, in which they could compute security tokens

ymerouani commented 2 years ago

You could try to integrate something like rlua in the custom yaml dezoomer to let the user compute their own URL, in which they could compute security tokens

Thank you so much for your quick reply lovasoa, and of course thank you even more for your amazing tool.

I think the integration of rlua is beyond my skill level. I dont even begin to know whow to go about it. I was trying to figure out a way to maybe edit the way dezoomer-rs works with DZI by editing the "mod.rs" file. Maybe adding the security token part of the url as a string to this part, before "$" : let tile_re = Regex::new("_files/\\d+/\\d+_\\d+\\.(jpe?g|png)$").unwrap(); or maybe this part "{base}/{level}/{x}_{y}.{format}", . As you can see thats the extent of my knowledge, editing small things.

Is it doable that way? And if it is, any links that woudl show me hwo to recompile it?

ymerouani commented 2 years ago

Update, so I found a way by adding the string after "{format}" {base}/{level}/{x}_{y}.{format} inside the mod.rs file in dzi map, and then compiling. And it worked. Problem is I just figured that the securitytoken is dynamic, and each picture has its own security key. So this would warrant either scraping all the security tokens, for all pictures and then recompiling the file everytime for a new picture. O finding a better way. If I find a solution I will come back

Cosmicore commented 8 months ago

It seems that the only way is to write a script that will get the desired token and pass it to dezoomify-rs. All this will work quite well in batch mode. The script receives the token, and dezoomify-rs downloads the tiles and generates the image. This is only if the token is the same for each tile, i.e. it differs only for whole images.

If the token is different for each tile, I see another way. Maybe it is even easier. You can independently download each tile of all images (use a downloader program after the script or implement both functions in one script, which is easy). Then make tiles.yaml and build images with dezoomify-rs.

The last method is also suitable when tiles are encrypted and decrypted only when displayed on the screen by a non-standard viewer.