search

gee-community / geetools

A collection of tools to work with Google Earth Engine Python API
https://geetools.readthedocs.io/en/stable/
MIT License
508 stars 121 forks source link

Exporting a Collection into Local Computer Disk #27

Open KonstantinosF opened 5 years ago

KonstantinosF commented 5 years ago

HI!,

I would like to download an Image collection (more than 80 GB) into my personal computer, because google drive doesn't offer a lot of storage.

Is that possible? and if yes, How?

Thank you

fitoprincipe commented 5 years ago

Hi @KonstantinosF!

There is a way to use geetools for downloading a single ee.Image to a local folder, you can adapt it to download an entire image collection. I drop you an example that is working for me.

from geetools import batch
import ee
geom = ee.Geometry.Point([-72,-42]).buffer(1000)
col = ee.ImageCollection.fromImages([ee.Image.random(1), ee.Image.random(2), ee.Image.random(3)])
size = col.size().getInfo()
clist = col.toList(size)
for i in range(size):
    image = ee.Image(clist.get(i))
    iid = image.id().getInfo()
    name = 'test_'+iid
    print('downloading '+name)
    batch.Image.toLocal(image, name, scale=100, region=geom)
fitoprincipe commented 5 years ago

Hi @KonstantinosF , how did it go?

fitoprincipe commented 5 years ago

Hi @Adriano, I couldn't find this message in GitHub.. I haven't made good documentation yet, but I know I need to. Meanwhile you can do help(batch.Image.toFile)

El jue., 23 de may. de 2019 a la(s) 19:24, Adriano Matos ( notifications@github.com) escribió:

@fitoprincipe https://github.com/fitoprincipe I want to do the same. Is there documentation on all the functions in geetools? I can't find it the folder I downloaded, but I could easily be looking in the wrong spot. Also, can you specify the directory to save the images to when exporting to a local computer? Thanks in advance!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gee-community/gee_tools/issues/27?email_source=notifications&email_token=ACSA4EIEWOFXL727EMZCX2DPW4KSTA5CNFSM4G6ZZBWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWDU4XY#issuecomment-495406687, or mute the thread https://github.com/notifications/unsubscribe-auth/ACSA4EJTFYYSBKG37UNFFFDPW4KSTANCNFSM4G6ZZBWA .

-- Rodrigo E. Principe Forest Engineer linkedin.com/in/rprincipe82 http://www.linkedin.com/in/rprincipe82 github.com/fitoprincipe

adrianom-gh commented 5 years ago

Hi @fitoprincipe ! I am trying to also download imagery to my local computer, but I keep getting this error: "zipFile.BadZipFile: File is not a zip file." I think it has something to do with the image collection I am trying to export (particularly Landsat 8 Surface Reflectance). Are you familiar with this error? I provided a link to my script in case you wanted to replicate the error. It should work until you get that error I mentioned. It takes like 5 minutes until the error message appears.

[example.zip] (https://github.com/gee-community/gee_tools/files/3214444/example.zip)

Thanks so much in advance!

UPDATE: The script works when I select only one band in the image collection ("l8sr"), but it does not like it when I select 7 bands from the image collection to download.

adrianom-gh commented 5 years ago

Hi @adriano, I couldn't find this message in GitHub.. I haven't made good documentation yet, but I know I need to. Meanwhile you can do help(batch.Image.toFile) El jue., 23 de may. de 2019 a la(s) 19:24, Adriano Matos ( notifications@github.com) escribió: … @fitoprincipe https://github.com/fitoprincipe I want to do the same. Is there documentation on all the functions in geetools? I can't find it the folder I downloaded, but I could easily be looking in the wrong spot. Also, can you specify the directory to save the images to when exporting to a local computer? Thanks in advance! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27?email_source=notifications&email_token=ACSA4EIEWOFXL727EMZCX2DPW4KSTA5CNFSM4G6ZZBWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWDU4XY#issuecomment-495406687>, or mute the thread https://github.com/notifications/unsubscribe-auth/ACSA4EJTFYYSBKG37UNFFFDPW4KSTANCNFSM4G6ZZBWA . -- Rodrigo E. Principe Forest Engineer linkedin.com/in/rprincipe82 http://www.linkedin.com/in/rprincipe82 github.com/fitoprincipe

@fitoprincipe Oh sorry! I deleted the reply because I figured out the "toLocal" function myself. I did message you with an error message I am getting when using "toLocal." I will continue to look into this myself, unless you know of this error.

cratcliff commented 3 years ago

@fitoprincipe I'm trying to do the same but match the outputs to what I do in google earth engine.

I'm after a multi-band tif with named bands the same as the following, but saved to the local disk rather than google drive

var batch = require('users/fitoprincipe/geetools:batch')
batch.Download.ImageCollection.toDrive(S2_AOI, 'EP', 
                                      {name:ee.String(Name).cat("_{GRANULE_ID}"),region:inputShape,scale:10});

I have seen your suggestions, but toLocal() saves bands to individual tifs and there is no toLocal() for image collections.

Any tips would be appreciated.

giswqs commented 3 years ago

You can try geemap. see https://geemap.org/notebooks/11_export_image

fitoprincipe commented 3 years ago

I didn't see the filePerBand parameter before.. how long has been there? 😆 🤦‍♂️ thanks @giswqs! =)

cratcliff commented 3 years ago

Thanks for the suggestion - that's part way there.

I would also like to add the named bands as part of the export so the user can tell which band is which, to make like-for-like band selection easier later on down the track.

batch.imagecollection.toDrive() already does this and in QGIS it looks like this image

Compared to geemap.ee_export_image() image