search

USGS-CMG / usgs-cmg-portal

do_convert.sh
6 stars 13 forks source link

where does the Mooring ID displayed in portal come from, and can it be a string? #288

Open emontgomery-usgs opened 6 years ago

emontgomery-usgs commented 6 years ago

@kwilcox, @rsignell-usgs

The mooring ID used in Woods Hole data is shown in the red boxes below. The other centers use different platform identification schemes. How should I put something like "MCR13S1T" in "mooring_id (currently cast to int at https://github.com/USGS-CMG/usgs-cmg-portal/blob/master/woods_hole_obs_data/collect.py#L545-L556) and not make things break downstream? image

Thanks!

kwilcox commented 6 years ago

If you were willing to update all of the existing files with a MOORING global attribute we could make it a generic string field to accommodate new centers.

emontgomery-usgs commented 6 years ago

The existing files from WHSC all already have a global attribute "MOORING". Not sure about the ones from the other centers, but adding that as part of the conversion process would certainly be possible.

Sounds like this is a good way to go instead of parsing the file name.

On Fri, Jul 6, 2018 at 11:21 AM, Kyle Wilcox notifications@github.com wrote:

If you were willing to update all of the existing files with a MOORING global attribute we could make it a generic string field to accommodate new centers.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/USGS-CMG/usgs-cmg-portal/issues/288#issuecomment-403065472, or mute the thread https://github.com/notifications/unsubscribe-auth/ABsHnggB9Zz0w-B4RXQa5x698txrJ-jyks5uD4B9gaJpZM4U_c6E .

-- Ellyn Montgomery, Oceanographer and Data Manager U.S. Geological Survey Woods Hole Coastal and Marine Science Center 384 Woods Hole Rd., Woods Hole, MA, 02543-1598 (508) 548-8700 x2356 https://stellwagen.er.usgs.gov; https://woodshole.er.usgs.gov/operations/stg/

kwilcox commented 6 years ago

Using a reliable global attribute would indeed be much better. As I recall we could not rely on the attribute being present or having correct information so we had to generate it through the filename.

emontgomery-usgs commented 6 years ago

should be easy enough to check and fix if it's not present. Will let you know when I've checked. Thanks!

On Fri, Jul 6, 2018 at 12:04 PM, Kyle Wilcox notifications@github.com wrote:

Using a reliable global attribute would indeed be much better. As I recall we could not rely on the attribute being present or having correct information so we have to generate it through the filename.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/USGS-CMG/usgs-cmg-portal/issues/288#issuecomment-403077026, or mute the thread https://github.com/notifications/unsubscribe-auth/ABsHnghrPojJyVViw2fvMnehIp9DVIgoks5uD4qrgaJpZM4U_c6E .

-- Ellyn Montgomery, Oceanographer and Data Manager U.S. Geological Survey Woods Hole Coastal and Marine Science Center 384 Woods Hole Rd., Woods Hole, MA, 02543-1598 (508) 548-8700 x2356 https://stellwagen.er.usgs.gov; https://woodshole.er.usgs.gov/operations/stg/

emontgomery-usgs commented 6 years ago

@kwilcox , @rsignell-usgs

As a test, I have assembled a few directories with different things in their "MOORING" attributes as a test of using the attribute instead of the filename to determine the Mooring ID. All are now of string type; ours contain numeric identifiers, others use ASCII characters.

Could you provide a version of collect.py that will work with this new scheme? could the outputs be ingested into sandbox portal where the data could be checked without exposing potentially funky data on the real portal? ticket #271 suggest there is one, but it may only work with real-time data?

Thanks!