ioos / ioosngdac

IOOS National Glider Data Assembly Center (V2)
https://ioos.github.io/ioosngdac/
8 stars 18 forks source link

Deploy ERDDAP at glider DAC with all 2.0 files #17

Closed dpsnowden closed 9 years ago

dpsnowden commented 9 years ago

@kerfoot and @lukecampbell, I'm starting this issue to track the progress and outstanding actions in deploying ERDDAP to the glider DAC servers. We need a date by which this will happen soon.

@BobSimons, FYI.
@kknee I assigned this to you for your awareness, reassign to whoever makes sense.

We have currently...

  1. The development ERDDAP at rutgers
  2. The production THREDDS which, I believe, contains only 1.0 files

The objective, according to the architecture diagram, is to have a public ERDDAP and public THREDDS serving 2.0 files. Another mandatory objective is to maintain the current 1.0 THREDDS implementation at least until we have a clear transition for NDBC that they approve, can implement, and have implemented.

lukecampbell commented 9 years ago

Just an update on the state of affairs.

Glider DAC 2 is up and running and has been for over a month with production-ready ERDDAP instances. We've been in a steady state of testing to make sure that it's stable and ready for the production environment.

ERDDAP 1.5 is running as both the public and private instances.

The missing piece here is the data, I think.

BeckyBaltes commented 9 years ago

@lukecampbell: Can you share a link for ERDDAP 1.5? Are the Scripps files going there? Also is this some interval between DAC 1.0 and 2.0 or is it the 2.0 version?

I'm working on the data providers. USF's Chad Lembke and Mike (not sure last name) are working on 2.0 files from their current deployment. Even if they don't get it done before they recover the glider, I would like to push to get their files into the DAC. I'm not sure if they will come through TDS or ERDDAP.

Thanks, Becky

lukecampbell commented 9 years ago

I'm in the process of updating the code on production to include the email support right now. I'd take it as a kind-ness if you allow me to post the link tomorrow when the server is up and running on a stable platform.

BobSimons commented 9 years ago

ERDDAP 1.54 is available for download now. It contains changes related to the ISO 19115 documents that ERDDAP produces. There are no datasets.xml or setup.xml changes required for upgrading from ERDDAP 1.50 to 1.54. All you need to do is plop the new erddap.war file into tomcat's /webapps and restart Tomcat. If you have any questions, please let me know.

On 2014-11-03 10:51 AM, Luke Campbell wrote:

Just an FYI on the state of affairs.

Glider DAC 2 is up and running and has been for over a month with production-ready ERDDAP instances. We've been in a steady state of testing to make sure that it's stable and ready for the production environment.

ERDDAP 1.5 is running as both the public and private instances.

The missing piece here is the data, I think.

— Reply to this email directly or view it on GitHub https://github.com/ioos/ioosngdac/issues/17#issuecomment-61527503.

Sincerely,

Bob Simons IT Specialist Environmental Research Division NOAA Southwest Fisheries Science Center 99 Pacific St, Suite 255A (New!) Monterey, CA 93940 (New!) Phone: (831)333-9878 (New!) Fax: (831)648-8440 Email: bob.simons@noaa.gov

The contents of this message are mine personally and do not necessarily reflect any position of the Government or the National Oceanic and Atmospheric Administration. <>< <>< <>< <>< <>< <>< <>< <>< <>< <><

BeckyBaltes commented 9 years ago

@lukecampbell: That's fine.

dpsnowden commented 9 years ago

What will the URL be when it is up? Do you need to register another URL with our godaddy acct? The current TDS is http://tds.gliders.ioos.us/thredds/catalog.html which is both long and redundant but fine. Do you anticipate erddap.gliders.ioos.us/erddap?

kknee commented 9 years ago

The URL is an outstanding question - we talked last week about switching the existing domain from 1.0 to 2.0 at some point.

Do we want anew URL instead? What about a URL for TDS?

dpsnowden commented 9 years ago

Hhmm

I generally don't like the idea of switching things that are deployed currently. But, in reality we might not have any customers outside of NDBC so it may still be possible.

Does THREDDS have the ability to do redirects like a normal web server?

A couple options...staying with the current model.

tds.gliders.ioos.us/thredds (Exists, and we could just create a 1.0 catalog that we deprecate and a 2.0 folder going forward.) erddap.gliders.ioos.us/erddap

Maybe simpler and more extensible going forward (Kelly, Im thinking forward to the EDS project)

data.ioos.us/thredds/gliders data.ioos.us/erddap/gliders

If we're able to redirect or some other way maintain the original TDS URL for some period of time it would make me more comfortable changing to a new URL scheme.

On Mon, Nov 3, 2014 at 6:16 PM, Kelly Knee notifications@github.com wrote:

The URL is an outstanding question - we talked last week about switching the existing domain from 1.0 to 2.0 at some point.

Do we want anew URL instead? What about a URL for TDS?

— Reply to this email directly or view it on GitHub https://github.com/ioos/ioosngdac/issues/17#issuecomment-61567410.

Derrick Snowden System Architect US IOOS http://www.ioos.noaa.gov 1100 Wayne Ave, Suite 1225 Silver Spring, MD 20912 +1 301 427 2464 (o), +1 301 427 2073 (f)

Find us on Facebook http://www.facebook.com/usioosgov

kknee commented 9 years ago

I like the idea of using data.ioos.ushttp://data.ioos.us as the base URL.

@lukecampbell - insight on tds redirect?

Sent from my iPhone

On Nov 4, 2014, at 11:57 AM, "Derrick Snowden" notifications@github.com<mailto:notifications@github.com> wrote:

Hhmm

I generally don't like the idea of switching things that are deployed currently. But, in reality we might not have any customers outside of NDBC so it may still be possible.

Does THREDDS have the ability to do redirects like a normal web server?

A couple options...staying with the current model.

tds.gliders.ioos.us/threddshttp://tds.gliders.ioos.us/thredds (Exists, and we could just create a 1.0 catalog that we deprecate and a 2.0 folder going forward.) erddap.gliders.ioos.us/erddaphttp://erddap.gliders.ioos.us/erddap

Maybe simpler and more extensible going forward (Kelly, Im thinking forward to the EDS project)

data.ioos.us/thredds/glidershttp://data.ioos.us/thredds/gliders data.ioos.us/erddap/glidershttp://data.ioos.us/erddap/gliders

If we're able to redirect or some other way maintain the original TDS URL for some period of time it would make me more comfortable changing to a new URL scheme.

On Mon, Nov 3, 2014 at 6:16 PM, Kelly Knee notifications@github.com<mailto:notifications@github.com> wrote:

The URL is an outstanding question - we talked last week about switching the existing domain from 1.0 to 2.0 at some point.

Do we want anew URL instead? What about a URL for TDS?

— Reply to this email directly or view it on GitHub https://github.com/ioos/ioosngdac/issues/17#issuecomment-61567410.

Derrick Snowden System Architect US IOOS http://www.ioos.noaa.gov 1100 Wayne Ave, Suite 1225 Silver Spring, MD 20912 +1 301 427 2464 (o), +1 301 427 2073 (f)

Find us on Facebook http://www.facebook.com/usioosgov

— Reply to this email directly or view it on GitHubhttps://github.com/ioos/ioosngdac/issues/17#issuecomment-61673777.

lukecampbell commented 9 years ago

The short answer is I don't know, but I can look into it.

BobSimons commented 9 years ago

For ERDDAP, it can be data.ioos.us/erddap/ but all subsequent file and directory names are controlled by ERDDAP (although they may include datasetIDs).

On 2014-11-04 8:57 AM, Derrick Snowden wrote:

Hhmm

I generally don't like the idea of switching things that are deployed currently. But, in reality we might not have any customers outside of NDBC so it may still be possible.

Does THREDDS have the ability to do redirects like a normal web server?

A couple options...staying with the current model.

tds.gliders.ioos.us/thredds (Exists, and we could just create a 1.0 catalog that we deprecate and a 2.0 folder going forward.) erddap.gliders.ioos.us/erddap

Maybe simpler and more extensible going forward (Kelly, Im thinking forward to the EDS project)

data.ioos.us/thredds/gliders data.ioos.us/erddap/gliders

If we're able to redirect or some other way maintain the original TDS URL for some period of time it would make me more comfortable changing to a new URL scheme.

On Mon, Nov 3, 2014 at 6:16 PM, Kelly Knee notifications@github.com wrote:

The URL is an outstanding question - we talked last week about switching the existing domain from 1.0 to 2.0 at some point.

Do we want anew URL instead? What about a URL for TDS?

— Reply to this email directly or view it on GitHub https://github.com/ioos/ioosngdac/issues/17#issuecomment-61567410.

Derrick Snowden System Architect US IOOS http://www.ioos.noaa.gov 1100 Wayne Ave, Suite 1225 Silver Spring, MD 20912 +1 301 427 2464 (o), +1 301 427 2073 (f)

Find us on Facebook http://www.facebook.com/usioosgov

— Reply to this email directly or view it on GitHub https://github.com/ioos/ioosngdac/issues/17#issuecomment-61673777.

Sincerely,

Bob Simons IT Specialist Environmental Research Division NOAA Southwest Fisheries Science Center 99 Pacific St, Suite 255A (New!) Monterey, CA 93940 (New!) Phone: (831)333-9878 (New!) Fax: (831)648-8440 Email: bob.simons@noaa.gov

The contents of this message are mine personally and do not necessarily reflect any position of the Government or the National Oceanic and Atmospheric Administration. <>< <>< <>< <>< <>< <>< <>< <>< <>< <><

lukecampbell commented 9 years ago

We have two hosts currently, one that runs the FTP server that receives data from providers and has a simple web interface to manage those uploads. The other hosts runs many things:

We could leave the FTP upload site without a domain name and tell the providers the IP address, which adds a small layer of safety and security (but not much). And we could register the domain data.ioos.us and point the record to the DAP host which is running all of those services.

Right now the reverse-proxy server, nginx, is routing traffic to "/erddap" to the public erddap instance and "/thredds/" to the thredds instance and denying all other requests. Configuring it to forward requests to another domain may be doable but I've never configured it or attempted it.

dpsnowden commented 9 years ago

That sounds good to me. I think it's fine to leave the current gliders.ioos.us ftp/upload site as it is currently.

On Tue, Nov 4, 2014 at 1:54 PM, Luke Campbell notifications@github.com wrote:

We have two hosts currently, one that runs the FTP server that receives data from providers and has a simple web interface to manage those uploads. The other hosts runs many things:

  • Private ERDDAP instance that shouldn't be visible to outsiders
  • Public ERDDAP instance that is available
  • Public THREDDS instance that is available.

We could leave the FTP upload site without a domain name and tell the providers the IP address, which adds a small layer of safety and security (but not much). And we could register the domain data.ioos.us and point the record to the DAP host which is running all of those services.

Right now the reverse-proxy server, nginx, is routing traffic to "/erddap" to the public erddap instance and "/thredds/" to the thredds instance and denying all other requests. Configuring it to forward requests to another domain may be doable but I've never configured it or attempted it.

— Reply to this email directly or view it on GitHub https://github.com/ioos/ioosngdac/issues/17#issuecomment-61693454.

Derrick Snowden System Architect US IOOS http://www.ioos.noaa.gov 1100 Wayne Ave, Suite 1225 Silver Spring, MD 20912 +1 301 427 2464 (o), +1 301 427 2073 (f)

Find us on Facebook http://www.facebook.com/usioosgov

kerfoot commented 9 years ago

closed