search

lovasoa / dezoom.sh

Download and assemble tiled images. Dezoomify for bash. Depends on imagemagick
BSD 2-Clause "Simplified" License
9 stars 2 forks source link

How to customize this script to merge already downloaded tiles? #3

Closed atultiwari closed 7 years ago

atultiwari commented 7 years ago

Hi, Since last 2 days I tried your script 8-10 times, about 5 times it worked 100% fine, but in some cases it again showed some blank areas (which might be because of some failed connection due to my bad network). In 2 of the images downloaded, the pathologically most important part of the slide didn't download. So, it forced me to think alternative. One alternative I could think of - I created localhost using wamp64 and created following test php file -

<?php
ini_set('max_execution_time', 0);
ini_set("allow_url_fopen", 1);

for ($y=1; $y<=98; $y++) { 
    for ($x=1; $x<=163; $x++) {
        echo "Value of x = " . $x . " & y = " . $y . "<br>";
        copy("http://122.160.116.48/DSStore/code/tileHandler.ashx?Id=16786&imageId=-1&tierIndex=0&tileLevel=0&tilePositionX={$x}&tilePositionY={$y}&contentType=image/jpeg&isCropped=1", "16786/16786_{$x}_{$y}.jpg");
    }
}
?>

Basically doing same thing (downloading all tiles), only difference, I am trying in PHP (as I understand it a bit, as compared to bash). So, now my image tiles will be downloaded in following format - ImageName_x_y.jpg

Is there any way I can use your bash script on this local folder to merge the images (both on X axis & Y axis). I also read about imagemagick, that it can do similar things, but I am yet to figure out how can I make a script, which can merge in 2-d (both horizontal and vertical).

I know my approach is some childish version of your script, but with it I will be able to figure out if all tiles are there or not, and even after (offline) merging, some important tile goes missing, I can manually download that particular tile only and use merge script again and ultimately saving my bandwidth

Thanks.

lovasoa commented 7 years ago

If you have a local file server, then you can point dezoom.sh to it. For instance : 

dezoom.sh 45 52 "http://localhost/16786/16786_%X_%Y.jpg"

Of course the path depends on where in your local server you put your files.

lovasoa commented 7 years ago

I think I should simply not remove the cache folder after the script terminates. So that when you launch the script again, it just downloads the tiles it didn't download the last time.

atultiwari commented 7 years ago

@lovasoa , thank you for your reply, although, I had already used the similar approach, just wanted to test before writing here.

I had modified my above mentioned code to the following -


<?php
ini_set('max_execution_time', 0);
ini_set("allow_url_fopen", 1);

$z=16787;
for ($y=0; $y<=123; $y++) { 
    for ($x=0; $x<=110; $x++) {
        $file_name = "{$z}/{$z}_{$x}_{$y}.jpg";
        if (file_exists($file_name)) {
            //echo "file + " . $file_name . "<br>";
        } else {
            copy("http://122.160.116.48/DSStore/code/tileHandler.ashx?Id={$z}&imageId=-1&tierIndex=0&tileLevel=0&tilePositionX={$x}&tilePositionY={$y}&contentType=image/jpeg&isCropped=1", "{$z}/{$z}_{$x}_{$y}.jpg");
            echo "x = " . $x . " & y = " . $y . "<br>";
        }
    }
}

?>

Now, I am double-checking by using file_exists, since even downloading to my localhost failed for some files and I to manually search for it. Now, I just reopen my php page to check if there is any file/tile missing. I don't why, but it didn't hit my mind that I can directly use the images from localhost. So, what I did was something stupid, and made following show_image.php file -

<?php
ini_set('max_execution_time', 0);
ini_set("allow_url_fopen", 1);
$image_id = $_GET['id'];
$x = $_GET['x'];
$y = $_GET['y'];
$remoteImage = "{$image_id}/{$image_id}_{$x}_{$y}.jpg";
$imginfo = getimagesize($remoteImage);
header("Content-type: {$imginfo['mime']}");
readfile($remoteImage);
?>

and then I used your script, on my localhost's php file, providing X and Y values. Now, after reading your reply, this php feels useless. I will directly use image path instead of displaying theme through PHP.

And, yes I about to suggest you the same-

I think I should simply not remove the cache folder after the script terminates. So that when you launch the script again, it just downloads the tiles it didn't download the last time.

Since I don't understand shell/bash that much, I was not able to make out where your temp files are storing and how? Also, now I am using wamp server along with bash on ubuntu on windows. Although your script while assembling images when it's is between 99-100%, it uses whole of my laptop's RAM. To counter this I tried to understand your code. I though if I could just use the relevant part through command prompt, it might save free RAM. Therefore, after reading your script some more times, I understood that, you are using Imagemagick, with following commands - montage - -geometry +0+0 -tile "$width_in_tiles"x"$height_in_tiles" result.jpg

I thought of giving a try this through my command prompt. So, I installed imagemagick in my windows. But so far, I could only learn some commands of Imagemagick. I am yet to figure out, if there is a way to make a .bat script that I can use to loop and put all the file names in imagemagick's command montage, geometry & tile. Any ideas?

Anyways thanks for your continuous help and support. I finally am able to get the images, I actually wanted and that too without any missing tile.

Regards

lovasoa commented 7 years ago

Yes, it eats a lot of memory when assembling tiles, but using imagemagick on windows won't avoid that. It just have to load all the pixels of your image in memory.

You can use my script directly on windows with cygwin.

atultiwari commented 7 years ago

Ok, Thank you for your prompt suggestion. I will do the same. I learnt a lot in past week about this and it was worth it. A quick suggestion though - what I feel is, it would be better if you could modify your script somehow, like my php code, by adding one extra step "to check for missing tiles before assembling" and keeping the cache tiles also and let user decide wether he wants to keep them or discard them, it would be better.

E.g. after providing X, Y and Url -> Step 1 - Download all tiles -> Step 2. Verify if all tiles within this X & Y range are present -> if not try to re-download them. -> Step 3. Even after re-downloading some tiles failed to download, warn user about their names, if he can manually put these files in temp folder with other files, otherwise start assembling -> Step 4. Ask user whether he wants to keep the temp folder or delete it.

Just got a new idea - instead of always naming it as "result.jpg", let user decide the final file name via some optional parameter provided with "./dezoom.sh"

I think, now I am sounding crazy, so I should stop here, but these are the ideas, which hit my mind, after going through your script so many times.

lovasoa commented 7 years ago

You can open new issues for your different suggestions and we'll discuss them.

lovasoa commented 7 years ago

I implemented a lot of what you proposed, and some new exciting features:

atultiwari commented 7 years ago

Thank you so much for adding these features and updating me about the same. Auto-detection of X & Y, reuse of already downloaded tiles in case of earlier failure will certainly save lot of human hours. I will give it a try tonight and will post my result here :)

lovasoa commented 7 years ago

I also updated the documentation on the main page. Don't hesitate to read it again in order to use the new features.