search

ropensci-archive / wishlist

:no_entry: ARCHIVED :no_entry:
https://discuss.ropensci.org/c/wishlist/6
50 stars 4 forks source link

Access the PTAGIS database of tagged fish population dynamics #13

Closed cboettig closed 2 years ago

cboettig commented 9 years ago

What will the package do?

Access the PTAGIS database of tagged fish population dynamics

Relevant links (For e.g. API docs, related software)

Got the idea from a recent comment on this dynamic ecology post. The blog accurately points out the lack of detailed, long term & spatially explicit individual level data of births and deaths. This repository provides uniquely extensive data on this, but the high-level summary data isn't well-enough resolved to be useful. All raw data is available on the FTP site, but is somewhat impenetrable. A package could assist in both accessing the data as it is updated, and then synthesizing the data into more meaningful data frames for analysis.

sckott commented 9 years ago

Continuing discussion from the email thread, paging @rmendels

Yeah, the https://github.com/ropensci/rerddap package is a general purpose client for ERDDAP servers, so if you get this dataset into an ERDDAP server, you could import rerddap, then move on from there in the package processing data, etc. It supports netcdf usage via either ncdf or ncdf4 packages, depending on what users have available on their machine (https://github.com/ropensci/rerddap/blob/master/DESCRIPTION)

rmendels commented 9 years ago

Hi All:

Scott, I don’t remember, are you still using .csv files for tabledap output? These type of data would most likely be in tabledap.

The advantage of using ERDDAP is reusable code. A very simple interface provides access to a wide variety of data (literally petabytes), and specific R code does not have to be written for each new database. I should add that ATN is using ERDDAP for their tagged data, and therefore can easily access both the tag data as well as relevant environmental data. For example, the ATN web page http://oceanview.pfeg.noaa.gov/ATN/ uses ERDDAP as a backend.

My only question is rights to the PTAGIS data. I am a strong believer that all of these types of data should be open, but we only serve data with the agreement of the “owners”, though we often find the open data policy of the funders of the collecting of the data aren’t the same as the rights claimed by the “owners” (usually the funding agency requires much more open access). BTW - I am not saying this is the case with PTAGIS, I have no idea. Just a general observation.

-Roy

On Jun 18, 2015, at 6:59 AM, Scott Chamberlain notifications@github.com wrote:

Continuing discussion from the email thread, paging @rmendels

Yeah, the https://github.com/ropensci/rerddap package is a general purpose client for ERDDAP servers, so if you get this dataset into an ERDDAP server, you could import rerddap, then move on from there in the package processing data, etc. It supports netcdf usage via either ncdf or ncdf4 packages, depending on what users have available on their machine (https://github.com/ropensci/rerddap/blob/master/DESCRIPTION)

— Reply to this email directly or view it on GitHub.

"The contents of this message do not reflect any position of the U.S. Government or NOAA."

Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center _Note new address and phone_ 110 Shaffer Road Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: Roy.Mendelssohn@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

sckott commented 9 years ago

yes, using csv for tabledap, but nc for griddap

sckott commented 8 years ago

@cboettig i can help if you still want it to happen

cboettig commented 8 years ago

cool. not a priority at the moment but hope to revisit eventually. Meanwhile been meaning to ping you over on EML, have gotten back to rewriting that (though still sad I don't have an xml2 that can write XML) and wanted to get your input, particularly on some UI design questions.

sckott commented 8 years ago

not a priority at the moment but hope to revisit eventually

okay, sounds good

been meaning to ping you over on EML

im all ears

john10e commented 7 years ago

Hello - I stumbled onto this today. I'm reaching out to let you know there are much better ways of accessing PTAGIS data than raw data files to fulfill your request. If there is still a need for this please contact us through our website and we can discuss further.

John Tenney PTAGIS Program Manager

cboettig commented 7 years ago

@john10e Thanks for reaching out! Just to be clear, we are looking for a mechanism that would supports automated access, e.g. that user 1 can write an R script using the package which would access the data and perform some analysis, and user 2 can run the same script and get the same results, ideally without having to separately register for or request access to the particular dataset. Is that feasible?

rmendels commented 7 years ago

We were hoping put the the data on an ERDDAP server. Now that there is rerddap, that would allow access by scripts. ERDDAP servers around the world provide access to literally petabytes of data, including the ATN for example.

-Roy

On May 4, 2017, at 8:14 PM, Carl Boettiger notifications@github.com wrote:

@john10e Thanks for reaching out! Just to be clear, we are looking for a mechanism that would supports automated access, e.g. that user 1 can write an R script using the package which would access the data and perform some analysis, and user 2 can run the same script and get the same results, ideally without having to separately register for or request access to the particular dataset. Is that feasible?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

"The contents of this message do not reflect any position of the U.S. Government or NOAA."

Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center Note new street address 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: Roy.Mendelssohn@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

maelle commented 2 years ago

Thank you!

Note that future ideas should go to our wishlist forum category.