Unable to get data from NOAA

[ChrisHarrod]

Hi, Trying to download bathymetry data from NOAA gives the following error

getNOAA.bathy(lon1=-20,lon2=-90,lat1=50,lat2=20, resolution=10)

Querying NOAA database ... This may take seconds to minutes, depending on grid size Error in if (ncol(x) == 3 & !exists("bathy", inherits = FALSE)) { : argument is of length zero

Have tried with both the current CRAN and Github versions of the package.

Am running R 4.0.0 (Win 10) and as far as I can see, all necessary packages are updated

[abfleishman]

Does it matter that your lat2 is smaller than lat1? If you switch those numbers does it work?

---

Abram B. Fleishman | Research Scientist Conservation Metrics, Inc. http://www.conservationmetrics.com UCSC Coastal Science Campus 145 McAllister Way, Santa Cruz, CA 95060 (541)-207-2335 | abram@conservationmetrics.com

On Wed, May 20, 2020 at 3:06 PM ChrisHarrod notifications@github.com wrote:

Hi, Trying to download bathymetry data from NOAA gives the following error

getNOAA.bathy(lon1=-20,lon2=-90,lat1=50,lat2=20, resolution=10)

Querying NOAA database ... This may take seconds to minutes, depending on grid size

Error in if (ncol(x) == 3 & !exists("bathy", inherits = FALSE)) { : argument is of length zero

Have tried with both the current CRAN and Github versions of the package.

Am running R 4.0.0 (Win 10) and as far as I can see, all necessary packages are updated

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACGK6DDRP3SYX2RL3HAZF63RSRH6HANCNFSM4NGLDYPQ .

[ChrisHarrod]

Hi Abram

Many thanks for responding.

No, that doesn’t make any difference.

This example from the ‘Making and using bathymetric maps in R with Marmap’ vignette isn’t working either

papoue <- getNOAA.bathy(lon1 = 140, lon2 = 155, lat1 = -13, lat2 = 0, resolution = 2)

Cheers Chris

From: Abram Fleishman notifications@github.com Sent: Wednesday, May 20, 2020 7:09 PM To: ericpante/marmap marmap@noreply.github.com Cc: Chris Harrod Chris@harrodlab.net; Author author@noreply.github.com Subject: Re: [ericpante/marmap] Unable to get data from NOAA (#10)

Does it matter that your lat2 is smaller than lat1? If you switch those numbers does it work?

---

Abram B. Fleishman | Research Scientist Conservation Metrics, Inc. http://www.conservationmetrics.com UCSC Coastal Science Campus 145 McAllister Way, Santa Cruz, CA 95060 (541)-207-2335 | abram@conservationmetrics.commailto:abram@conservationmetrics.com

On Wed, May 20, 2020 at 3:06 PM ChrisHarrod notifications@github.com<mailto:notifications@github.com> wrote:

Hi, Trying to download bathymetry data from NOAA gives the following error

getNOAA.bathy(lon1=-20,lon2=-90,lat1=50,lat2=20, resolution=10)

Querying NOAA database ... This may take seconds to minutes, depending on grid size

Error in if (ncol(x) == 3 & !exists("bathy", inherits = FALSE)) { : argument is of length zero

Have tried with both the current CRAN and Github versions of the package.

Am running R 4.0.0 (Win 10) and as far as I can see, all necessary packages are updated

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACGK6DDRP3SYX2RL3HAZF63RSRH6HANCNFSM4NGLDYPQ .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ericpante/marmap/issues/10#issuecomment-631784391, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACNZSNBSQN22BZXPV3UZVITRSRPITANCNFSM4NGLDYPQ.

[besibo]

The way to get ETOPO1 gridded data from NOAA servers has recently changed. So we had to update getNOAA.bathy() to take this change into account. If you installed the lastest marmap version (1.0.4), you should also update both rgdal et raster packages. This should fix the issue.

[ChrisHarrod]

Many thanks!

From: Benoit Simon-Bouhet notifications@github.com Sent: Thursday, May 21, 2020 3:21 AM To: ericpante/marmap marmap@noreply.github.com Cc: Chris Harrod Chris@harrodlab.net; Author author@noreply.github.com Subject: Re: [ericpante/marmap] Unable to get data from NOAA (#10)

The way to get ETOPO1 gridded data from NOAA servers has recently changed. So we had to update getNOAA.bathy() to take this change into account. If you installed the lastest marmap version (1.0.4), you should also update both rgdal et raster packages. This should fix the issue.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ericpante/marmap/issues/10#issuecomment-631928645, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACNZSNHAHWEACAAG7RDDJPDRSTI4VANCNFSM4NGLDYPQ.

[jesseflamb]

Hello, I've done all of the following: updated marmap (1.04), raster (1.4.1, updated 6/27), & rgdal (1.5-12, updated 6/26) and am getting the same error with getNOAA.bathy(): "Error in if (ncol(x) == 3 & !exists("bathy", inherits = FALSE)) { : argument is of length zero"

Perhaps it's because of the most recent updates to both raster and rgdal?

[besibo]

Hi @jesseflamb,

I've tried the following:

library(marmap)
a <- getNOAA.bathy(-10,10,-20,0)

Everything is working fine on my side with marmap v1.0.4, rgdal v1.5-12 (updated just now) and raster v3.1-5 (updated just now as well). Something wrong with your code maybe?

[jesseflamb]

Hello Benoit,

Figured it out: Your code worked, yet mine still did not. It was not until I decreased the amount of decimal places in my coordinates from 4 to 2 that it worked. I looked through the documentation for a mention of this and didn't find anything.

Thanks for your time and the code.

On Wed, Jul 1, 2020 at 10:44 PM Benoit Simon-Bouhet < notifications@github.com> wrote:

Hi @jesseflamb https://github.com/jesseflamb,

I've tried the following:

library(marmap) a <- getNOAA.bathy(-10,10,-20,0)

Everything is working fine on my side with marmap v1.0.4, rgdal v1.5-12 (updated just now) and raster v3.1-5 (updated just now as well. Smoething wrong with your code maybe?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10#issuecomment-652796482, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJTISIHJ5NWQP77KDRWH6BLRZQNE3ANCNFSM4NGLDYPQ .

--

Jesse Lamb

Research Fisheries Biologist/NMFSC/AFSC- Seattle

NOAA Fisheries | U.S. Department of Commerce

Office: (206) 526-4125

www.fisheries.noaa.gov

[besibo]

Thanks for the update. Are you running R on Windows? Coud it be related to this issue?

[jesseflamb]

Benoit,

Maybe. I've been trying to fix the code so I don't actually have to input the physical numbers into getNOAA.bathy, for example:

b <- getNOAA.bathy(min(Dmns$LON3),max(Dmns$LON3), min(Dmns$LAT3),max(Dmns$LAT3), resolution = 1, keep = TRUE)

Running the above, gives me the same error. All values are numeric. However, when I input the actual number values into the code (2 decimals), it works fine. Before all of these updates, the above code worked.

On Thu, Jul 2, 2020 at 6:43 AM Benoit Simon-Bouhet notifications@github.com wrote:

Thanks for the update. Are you running R on Windows? Coud it be related to this issue https://github.com/ericpante/marmap/issues/15?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10#issuecomment-653013344, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJTISICZSSQJTNL3GTEYBZDRZSFGDANCNFSM4NGLDYPQ .

--

Jesse Lamb

Research Fisheries Biologist/NMFSC/AFSC- Seattle

NOAA Fisheries | U.S. Department of Commerce

Office: (206) 526-4125

www.fisheries.noaa.gov

[abfleishman]

Try rounding the coordinates like this:

b <- getNOAA.bathy(floor(min(Dmns$LON3) ,digits=2
),ceiling(max(Dmns$LON3),digits=2),
floor(min(Dmns$LAT3) ,digits=2 ),ceiling(max(Dmns$LAT3) ,digits=2)  ,
resolution = 1, keep = TRUE)

---

Abram B. Fleishman | Research Scientist Conservation Metrics, Inc. http://www.conservationmetrics.com UCSC Coastal Science Campus 145 McAllister Way, Santa Cruz, CA 95060 (541)-207-2335 | abram@conservationmetrics.com

On Thu, Jul 2, 2020 at 7:06 AM jesseflamb notifications@github.com wrote:

Benoit,

Maybe. I've been trying to fix the code so I don't actually have to input the physical numbers into getNOAA.bathy, for example:

b <- getNOAA.bathy(min(Dmns$LON3),max(Dmns$LON3), min(Dmns$LAT3),max(Dmns$LAT3), resolution = 1, keep = TRUE)

Running the above, gives me the same error. All values are numeric. However, when I input the actual number values into the code (2 decimals), it works fine. Before all of these updates, the above code worked.

On Thu, Jul 2, 2020 at 6:43 AM Benoit Simon-Bouhet < notifications@github.com> wrote:

Thanks for the update. Are you running R on Windows? Coud it be related to this issue https://github.com/ericpante/marmap/issues/15?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10#issuecomment-653013344, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AJTISICZSSQJTNL3GTEYBZDRZSFGDANCNFSM4NGLDYPQ

.

--

Jesse Lamb

Research Fisheries Biologist/NMFSC/AFSC- Seattle

NOAA Fisheries | U.S. Department of Commerce

Office: (206) 526-4125

www.fisheries.noaa.gov

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10#issuecomment-653026708, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACGK6DD5OZDSUCK6VTFRGXLRZSH7TANCNFSM4NGLDYPQ .

[jesseflamb]

I had to get rid of the 'digits' code (gave me an error "2 arguments passed to 'floor' which requires 1"), but worked with just 'floor' and 'ceiling' code. Thanks!

On Thu, Jul 2, 2020 at 2:05 PM Abram Fleishman notifications@github.com wrote:

Try rounding the coordinates like this:

b <- getNOAA.bathy(floor(min(Dmns$LON3) ,digits=2
),ceiling(max(Dmns$LON3),digits=2),
floor(min(Dmns$LAT3) ,digits=2 ),ceiling(max(Dmns$LAT3) ,digits=2) ,
resolution = 1, keep = TRUE)

---

Abram B. Fleishman | Research Scientist Conservation Metrics, Inc. http://www.conservationmetrics.com UCSC Coastal Science Campus 145 McAllister Way, Santa Cruz, CA 95060 (541)-207-2335 | abram@conservationmetrics.com

On Thu, Jul 2, 2020 at 7:06 AM jesseflamb notifications@github.com wrote:

Benoit,

Maybe. I've been trying to fix the code so I don't actually have to input the physical numbers into getNOAA.bathy, for example:

b <- getNOAA.bathy(min(Dmns$LON3),max(Dmns$LON3), min(Dmns$LAT3),max(Dmns$LAT3), resolution = 1, keep = TRUE)

Running the above, gives me the same error. All values are numeric. However, when I input the actual number values into the code (2 decimals), it works fine. Before all of these updates, the above code worked.

On Thu, Jul 2, 2020 at 6:43 AM Benoit Simon-Bouhet < notifications@github.com> wrote:

Thanks for the update. Are you running R on Windows? Coud it be related to this issue https://github.com/ericpante/marmap/issues/15?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/ericpante/marmap/issues/10#issuecomment-653013344 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AJTISICZSSQJTNL3GTEYBZDRZSFGDANCNFSM4NGLDYPQ

.

--

Jesse Lamb

Research Fisheries Biologist/NMFSC/AFSC- Seattle

NOAA Fisheries | U.S. Department of Commerce

Office: (206) 526-4125

www.fisheries.noaa.gov

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10#issuecomment-653026708, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACGK6DD5OZDSUCK6VTFRGXLRZSH7TANCNFSM4NGLDYPQ

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10#issuecomment-653220061, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJTISIBHO6E37C62DOJ2S53RZTZCJANCNFSM4NGLDYPQ .

--

Jesse Lamb

Research Fisheries Biologist/NMFSC/AFSC- Seattle

NOAA Fisheries | U.S. Department of Commerce

Office: (206) 526-4125

www.fisheries.noaa.gov

[besibo]

I've updated getNOAA.bathy() to shorten the url of web requests made on NOAA servers. Basically, it should now get rid of unnecessary decimals. @jesseflamb could you please install the latest development version, run the command that generated the error message, and report the results here?

[jesseflamb]

Ok, I copied the function from Github and it worked when I ran my original code. Thanks!

On Thu, Jul 9, 2020 at 3:03 AM Benoit Simon-Bouhet notifications@github.com wrote:

I've updated getNOAA.bathy() to shorten the url of web requests made on NOAA servers. Basically, it should now get rid of unnecessary decimals. @jesseflamb https://github.com/jesseflamb could you please install the latest development version, run the command that generated the error message, and report the results here?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericpante/marmap/issues/10#issuecomment-656035441, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJTISIHZ7HG6UWFC2WVF3WLR2WIXFANCNFSM4NGLDYPQ .

--

Jesse Lamb

Research Fisheries Biologist/NMFSC/AFSC- Seattle

NOAA Fisheries | U.S. Department of Commerce

Office: (206) 526-4125

www.fisheries.noaa.gov

[besibo]

Great! I'll push the update to CRAN ASAP. Thanks for your help and feedback.

[AngeliaMiller]

Hello,

I have updated versions of marmap (1.0.8), rgdal (1.6-2), and raster (3.6-11) loaded, and am still getting this error.

Error in if (ncol(x) == 3 & !exists("bathy", inherits = FALSE)) { : argument is of length zero

My code is as follows:

mab <- getNOAA.bathy(lon1 = -76, lon2 = -69, lat1 = 34, lat2 = 81, resolution = 10)

Any help would be greatly appreciated!

[besibo]

Have you tried this? https://github.com/ericpante/marmap/issues/25#issuecomment-982955852

[AngeliaMiller]

Yes, and was unsuccessful.

I did try #16 (comment) which was helpful, but the actual area I'm interested in, the continental shelf of the Mid-Atlantic Bight, was not included in the tif file I downloaded from grid extractor, or at least it seems according to the plot I generated from the data? So perhaps I am not using the right layer from grid extractor, which was ETOPO_2022 Hillshade (Bedrock; 60arcsecond)?

[besibo]

I meant " have you tried going back to an older version of the raster package ?". It should solve your problem with the getNOAA.bathy() error.

As for the Mid-Atlantic Bight, you have the correct dataset (Hillshade Bedrock), but your resolution is way too low for it to appear at that scale. On your plot, the deep blue color corresponds to depth of more than -3500m, while the New York Bight is only a few tens of meters deeper than the surrounding areas.

Here is an example with the maximum resolution of ETOPO 2022 (15 arcseconds) on a zoomed-in view of the New-York Bight:

mab <- getNOAA.bathy(lon1 = -75, lon2 = -72.5, lat1 = 39.5, lat2 = 41, resolution = 0.25)

plot(mab, image=TRUE, deep =-200, shallow = 0, step = 2, lwd = 0.2)
plot(mab, deep = -200, shallow = 0, step = 20, lwd = 0.4, add = TRUE)
plot(mab, deep = 0, shallow = 0, step = 0, lwd = 1, add = TRUE)

[AngeliaMiller]

Yes, I did try and go back to an older version of raster, specifically the one mentioned in the comment, but it did not help with the error. That is when I tried the manual download.

Thank you for the feedback on the resolution. That is helpful!

[besibo]

If somehow you manage to solve your issue with getNOAA.bathy(), I'd love to hear from you! This issue has been bothering me for too long and I can't reproduce the error on my machine... Thanks in advance!

[hongchicken]

Hi,

I have the same issue with getNOAA.bathy(). I tried the code bath<- getNOAA.bathy(min(longlatcoor$coords.x1), max(longlatcoor$coords.x1), min(longlatcoor$coords.x2), max(longlatcoor$coords.x2), resolution=2)

and got "Error in if (ncol(x) == 3 & !exists("bathy", inherits = FALSE)) { : argument is of length zero"

I've updated raster, rgdal, marmap, and also tried the old version of raster but didn't go through anything.

FYI, attached my session info below.

sessionInfo() R version 4.2.2 (2022-10-31 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 22621)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.utf8 [2] LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 [4] LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8

attached base packages: [1] stats graphics grDevices utils datasets [6] methods base

other attached packages: [1] sdmTMB_0.1.0 forcats_0.5.2 stringr_1.4.1
[4] dplyr_1.0.10 purrr_0.3.5 readr_2.1.3
[7] tidyr_1.2.1 tibble_3.1.8 ggplot2_3.4.0
[10] tidyverse_1.3.2 raster_3.4-13 rgdal_1.6-2
[13] sp_1.5-1 marmap_1.0.8 devtools_2.4.5 [16] usethis_2.1.6

[besibo]

I'll try and make a few tests on a Windows machine tomorrow. I suspect a limit on the size of urls that the Windows version of the raster package can handle. Indeed, the web query submitted by getNOAA.bathy() to NOAA's servers can be pretty long (more than 330 characters). If that's the case, a fix could be to shorten the query using the urlshorteneR package.

[besibo]

I have been able to confirm that the issue comes from a length limit of the urls the raster package can handle on Windows. I am currently working on a fix that should solve the problem (and give more info to users while downloading data from NOAA's servers). The fix should be available on Github by the end of the WE. Thank you all for your patience!

[besibo]

marmap v1.0.9 is now available on GitHub:

remotes::install_github("ericpante/marmap")

This version should solve the infamous Error message encountered by some (most ?) Windows users:

Error in if (ncol(x) == 3 & !exists("bathy", inherits = FALSE)) { :
argument is of length zero

I have been able to confirm that the error was due to a limitation in the length of urls that the raster package can handle on Windows. The geotif files from NOAA's servers are now downloaded with utils::download.file() in a temporary file on the user's disk and then imported in R using raster::raster().

I'd be glad if some of you could confirm that this update solves the issue before submitting it to CRAN.

[asimonis]

The update resolves the error for me. Thanks @besibo!

[kstierhoff]

This resolved my error too (Windows user)! Thanks for tackling this bug.

[hongchicken]

Resolved my issue too. Thank you so much!

Hi,

I have the same issue with getNOAA.bathy(). I tried the code bath<- getNOAA.bathy(min(longlatcoor$coords.x1), max(longlatcoor$coords.x1), min(longlatcoor$coords.x2), max(longlatcoor$coords.x2), resolution=2)

and got "Error in if (ncol(x) == 3 & !exists("bathy", inherits = FALSE)) { : argument is of length zero"

I've updated raster, rgdal, marmap, and also tried the old version of raster but didn't go through anything.

FYI, attached my session info below.

sessionInfo() R version 4.2.2 (2022-10-31 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 22621)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.utf8 [2] LC_CTYPE=English_United States.utf8 [3] LC_MONETARY=English_United States.utf8 [4] LC_NUMERIC=C [5] LC_TIME=English_United States.utf8

attached base packages: [1] stats graphics grDevices utils datasets [6] methods base

other attached packages: [1] sdmTMB_0.1.0 forcats_0.5.2 stringr_1.4.1 [4] dplyr_1.0.10 purrr_0.3.5 readr_2.1.3 [7] tidyr_1.2.1 tibble_3.1.8 ggplot2_3.4.0 [10] tidyverse_1.3.2 raster_3.4-13 rgdal_1.6-2 [13] sp_1.5-1 marmap_1.0.8 devtools_2.4.5 [16] usethis_2.1.6