podaac / data-subscriber

Subscribe and bulk download collections of data at PO.DAAC
Apache License 2.0
83 stars 29 forks source link

bounding box specification issue #134

Closed btupper closed 1 year ago

btupper commented 1 year ago

Hi,

I don't seem to have control over the the bounding box for a subscription. Below I show two commands for subscription (a) no -b specification and (b) specifying a small bounding box. The downloaded files are the same size (and each has the world-sized number of pixels.)

I can subset the local files, but it would consume fewer resources if we could transfer just the smaller subset.

Thanks! Ben

(a) the world (bounding box unspecified)

podaac-data-subscriber -c MUR-JPL-L4-GLOB-v4.1 -d world -sd 2021-01-21T00:00:00Z -ed 2021-01-23T00:00:00Z --verbose >> subscriber.log
# btupper@c4-6 tmp $ ll world
#   total 2231200
#   -rw-rw-r--. 1 btupper btupper 759327856 May 12 13:35 20210121090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc
#   -rw-rw-r--. 1 btupper btupper 757346424 May 12 13:35 20210122090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc
#   -rw-rw-r--. 1 btupper btupper 759097943 May 12 13:34 20210123090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc
#   -rw-rw-r--. 1 btupper btupper       197 May 12 13:35 MUR-JPL-L4-GLOB-v4.1.citation.txt

(b) a smaller subset specified by the -b argument.

podaac-data-subscriber -c MUR-JPL-L4-GLOB-v4.1 -d nwa2 -sd 2021-01-21T00:00:00Z -ed 2021-01-23T00:00:00Z -b="-77,36.5,-42.5,56.7" --verbose >> subscriber.log
# btupper@c4-6 tmp $ ll nwa2
#   total 2231200
#   -rw-rw-r--. 1 btupper btupper 759327856 May 12 13:43 20210121090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc
#   -rw-rw-r--. 1 btupper btupper 757346424 May 12 13:43 20210122090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc
#   -rw-rw-r--. 1 btupper btupper 759097943 May 12 13:43 20210123090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc
#   -rw-rw-r--. 1 btupper btupper       197 May 12 13:44 MUR-JPL-L4-GLOB-v4.1.citation.txt
#### 
#   version info
####
btupper@c4-6 tmp $ podaac-data-subscriber --version
#   PO.DAAC data subscriber 1.13.1
btupper@c4-6 tmp $ cat /etc/os-release
#   NAME="Rocky Linux"
#   VERSION="8.7 (Green Obsidian)"
#   ID="rocky"
#   ID_LIKE="rhel centos fedora"
#   VERSION_ID="8.7"
#   PLATFORM_ID="platform:el8"
#   PRETTY_NAME="Rocky Linux 8.7 (Green Obsidian)"
#   ANSI_COLOR="0;32"
#   LOGO="fedora-logo-icon"
#   CPE_NAME="cpe:/o:rocky:rocky:8:GA"
#   HOME_URL="https://rockylinux.org/"
#   BUG_REPORT_URL="https://bugs.rockylinux.org/"
#   ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
#   ROCKY_SUPPORT_PRODUCT_VERSION="8.7"
#   REDHAT_SUPPORT_PRODUCT="Rocky Linux"
#   REDHAT_SUPPORT_PRODUCT_VERSION="8.7"
mike-gangl commented 1 year ago

Hi Brian,

This is actually working as intended right now. The 'bbox' parameter only filters search results that do not pass through your region of interest. Because MUR data are essentially global data products, any bbox will match, and it downloads the entire file.

We have some work in progress on this exact functionality: https://github.com/podaac/data-subscriber/issues/36

Its still in development, but the good news is your data set (MUR) is high on the list of test cases for it. when this is delivered, an option -subset parameter will be provided to actually subset based on your space (bbox)and time bounds (start/stop dates).

We definitely appreciate the comment and detailed report, and are working hard to get the subset work pushed out to the users.

btupper commented 1 year ago

Oh, I see. Thank you! Ben