jgomezdans / get_modis

Downloading MODIS data from the USGS repository
GNU General Public License v3.0
62 stars 41 forks source link

Question: Can I download Kd (diffuse attenuation coefficient) data from MODIS-Aqua? #9

Closed cc92 closed 7 years ago

cc92 commented 7 years ago

Hi Jose,

Thanks for this great repo - just wondering if I can use this script for downloading data from: https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/Mapped/Monthly/4km/Kd/ ?

I've had a look manually through https://e4ftl01.cr.usgs.gov/MOLA/ (which is linked in the code and is where I think the script downloads from) and I can't identify which product/parameters to pass to your script.

Alternatively, do you know if any repo/API exists for the data I'm after?

Many thanks for any help, Chris

jgomezdans commented 7 years ago

Hi Chris,

Sorry, but my area is land (and bits of the atmosphere), so have no experience in ocean products, and haven't considered them for this script. However, if all your data are in that folder, can't you just use wget? It is also quite possible to download all this with Python. A very simple script would be

import os
import requests
from bs4 import BeautifulSoup

parent_url = 'https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/Mapped/Monthly/4km/Kd/'
r = requests.get(parent_url) # Get the listing page
soup = BeautifulSoup(r.text) # Parse using BS4

#Now find the table, and then all its rows
rows = soup.find("table").find_all("tr")
# Iterate row by row, store the url for later downloading it
all_urls = []
for row in rows:
    for datum in row.find_all("td"):
        all_urls.append( "%s%s" % (parent_url, datum.get_text()))
# Now download and save each granule
for url in all_urls:
    output_fname = url.split("/")[-1]
    if not os.path.exists(output_fname):
        r = requests.get(url, stream=True)
        # Save with temporary filename...
        with open(output_fname+".partial", 'wb') as fp:
            for block in r.iter_content(65536):
                fp.write(block)
        # Rename to definitive filename
        os.rename(output_fname+".partial", output_fname)
    else:
        # File was already present
        print "file %s is already here" % output_fname

But mostly, this is untested, and I don't know whether this is what you want ;)

jgomezdans commented 7 years ago

I'll close the issue if you don't mind.

asmith26 commented 6 years ago

Thanks soooooo much for providing the script above, helped me out too!