schism-dev / pyschism

Python interface for handling the SCHISM model.
https://schism-dev.github.io/schism/master/getting-started/pre-processing-with-pyschism/overview.html
Apache License 2.0
24 stars 20 forks source link

Generating GeoJSON to support NowCoast team #44

Open saeed-moghimi-noaa opened 2 years ago

saeed-moghimi-noaa commented 2 years ago

@Erin-nagel @agibbons-NOAA @josephzhang8 @cuill @feiye-vims @moghimis @gseroka @yangz888

Jul 19, 2022 | STOFS 3D GeoJSON Attendees: Adam Gibbons - NOAA Affiliate Erin Nagel - NOAA Affiliate Fei Ye John Kelley - NOAA Federal Linlin Cui Saeed Moghimi - NOAA Federal Y. Joseph Zhang Zizang Yang - NOAA Federal

Action items

Generating GeoJSON for STOFS3D

Metadata to GeoJSON


1st priority: Add above mention information to current max elev GeoJSON file

2nd: Then work on generating hourly GeoJSON files

Hi Saeed,

Thank you for reviewing the max water level STOFS3D web map service. Adam is currently working on the map interaction feature to query the service, however the GeoJSON only contains RGBA values rather than water level values and we would like to request some modifications to the current GeoJSON to include some additional attributes. Besides adding water level values is there any additional information your team would like to include?

Moving forward we would like to add an hourly STOFS3D service and I have included some general requirements and additional metadata for the GeoJSON files your team produces, please pass the requirements along to your team and let me know if you have any questions.

GeoJSON STOFS3D Product Requirements for nowCOAST:

Format: The filename must include the valid cycle time of the forecast for the first timestamp and the valid hourly forecast time or forecast time of the maximum water level disturbance for the second timestamp depending on the product Each feature-level object must include the following property names: "id","maxWaterLevel", "minWaterLevel", "verticalDatum", "units", "rgba" Example:

 properties": {
  "id": 1,
  "maxVal": 0.1,
  "minVal": 0.4,
  "verticalDatum": "NAVD88",
  "unit": "meters",
  "rgba": "#000080"
 }

Note: If there is only one water level elevation value per polygon, then include the property name "waterLevel" instead of "minWaterLevel" and "maxWaterLevel", but this attribute should be consistent for each product All property names must use the "camelCase" naming convention "id","maxWaterLevel", "minWaterLevel" properties must be encoded as true numeric values (i.e. not enclosed with quotes) and each feature-level object must have a unique id The top-level object should contain a coordinate reference system property Example:

"crs": { "type": "name", "properties": { "name": "EPSG:4326" } Polygons and MultiPolygons should follow the right-hand rule, as follows: The exterior ring should be counterclockwise. Interior rings should be clockwise. Example: The Shapely User Manual — Shapely 1.8.2 documentation All Polygons and MultiPolygons must be checked for validity and may not possess any overlapping exterior or interior rings, self intersections, duplicate nodes, sliver polygons, or gaps between polygons 1) Max Water Level Forecast Product

Suggested Product Path and Filename Format: s3://noaa-nos-stofs3d-pds/yyyymmdd/system_region_product_YYYYMMDDtHHz_YYYYMMDDtHHz.json

Example: s3://noaa-nos-stofs3d-pds/20220430/stofs3d_atlantic_maxwldisturbance 20220430t12z_20220430t23z .json

Note: Filename must include the valid cycle time of the forecast for the first timestamp (e.g 20220430T12Z) and the valid forecast time of the maximum water level disturbance for the second timestamp (e.g 20220430t23z)

2) Hourly Water Level Forecast Product

Suggested Product Path and Filename Format: s3://noaa-nos-stofs3d-pds/yyyymmdd/forecast_cycle/region/system_region_fcst_YYYYMMDDtHHz_YYYYMMDDtHHz.json :

Examples:

  1. s3://noaa-nos-stofs3d-pds/20220430/12/atlantic/stofs3d_atlantic_f001_20220430t12z_20220430t13z.json
  2. s3://noaa-nos-stofs3d-pds/20220430/12/atlantic/stofs3d_atlantic_f002_20220430t12z_20220430t14z.json
  3. s3://noaa-nos-stofs3d-pds/20220430/12/atlantic/stofs3d_atlantic_f003_20220430t12z_20220430t15z.json
  4. s3://noaa-nos-stofs3d-pds/20220430/12/atlantic/stofs3d_atlantic_f004_20220430t12z_20220430t16z.json
  5. s3://noaa-nos-stofs3d-pds/20220430/12/atlantic/stofs3d_atlantic_f005_20220430t12z_20220430t17z.json
  6. s3://noaa-nos-stofs3d-pds/20220430/12/atlantic/stofs3d_atlantic_f006_20220430t12z_20220430t18z.json

Note: Filename must include the cycle time of the forecast for the first timestamp (e.g. 20220430t12z) and the valid hourly forecast time for the second timestamp (e.g 20220430t18z)

Thank you,

nowCOAST Team

Erin Nagel - NOAA Affiliate

Fri, Jul 8, 10:24 AM (11 days ago)

to me, John, Adam

josephzhang8 commented 1 year ago

Note that we explained the same thing to CERA on their display also.

-Joseph

Y. Joseph Zhang Web: schism.wiki Office: 804 684 7466

From: John G.W. Kelley @.> Sent: Friday, October 14, 2022 11:49 AM To: schism-dev/pyschism @.> Cc: Y. Joseph Zhang @.>; Mention @.> Subject: Re: [schism-dev/pyschism] Generating GeoJSON to support NowCoast team (Issue #44)

[EXTERNAL to VIMS received message]

Joseph,

Have such occurrences of high water disturbances in isolated areas due rainfall been verified in previous hindcasts or forecasts?

I am concerned how users of STOFS3D forecast guidance maps will interpret these isolated areas of high water disturbances/inundation.

John

John G.W. Kelley Coastal Marine Modeling Branch Coast Survey Development Laboratory NOAA/National Ocean Service NOAA-UNH Joint Hydrographic Center Durham, New Hampshire 03824 603-862-1628

On Fri, Oct 14, 2022 at 11:38 AM Joseph Zhang @.<mailto:@.>> wrote:

Hi John:

I think some of those areas are due to rainfall on higher ground.

- Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1279166052, or unsubscribe https://github.com/notifications/unsubscribe-auth/APYSEZL5P3XYF26JP3OYNPLWDF46PANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.<mailto:@.>>

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fpyschism%2Fissues%2F44%23issuecomment-1279177283&data=05%7C01%7Cyjzhang%40vims.edu%7Ccbd765c3ddd04df19f0508daadfb9ae8%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638013593407374777%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sUONT7z3EwLpttBozBdVDS6kQnQ1A8dLYiwasRr4Xqo%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ3Y2CEUF7URM2SFZZLWDF6GTANCNFSM54BFEXFA&data=05%7C01%7Cyjzhang%40vims.edu%7Ccbd765c3ddd04df19f0508daadfb9ae8%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638013593407374777%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WLzjVY8kD2tziBCdc3MEsF5%2Bcbg0l78CW2gPjamtrPs%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.**@.>>

JohnGWKelley commented 1 year ago

I am concerned on how user will interpret for example this area in NJ (along Hackensack River/ near Meadowlands Sports Complex) which has values up above 1.5

I suspect that a user will think these are being forecast to be inundated by STOFS3D.

[image: image.png]

On Fri, Oct 14, 2022 at 11:51 AM Fei Ye @.***> wrote:

Linlin mask out inundations > 2 m in the previous plots. We can try setting the threshold at 1 m if necessary.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1279180025, or unsubscribe https://github.com/notifications/unsubscribe-auth/APYSEZNTI7XECDEGKJILS4TWDF6QXANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

josephzhang8 commented 1 year ago

Most people have experienced nuisance flooding (defined as inundation depth of 10cm) on high ground. I agree that large surface elev values tend to scare off some, but that's what USGS HWMs indicate also. Maybe we can add some explanations on the web site to try to educate the users on this (as CERA did)? What do you think? Thx.

-Joseph

Y. Joseph Zhang Web: schism.wiki Office: 804 684 7466

From: John G.W. Kelley @.> Sent: Friday, October 14, 2022 12:00 PM To: schism-dev/pyschism @.> Cc: Y. Joseph Zhang @.>; Mention @.> Subject: Re: [schism-dev/pyschism] Generating GeoJSON to support NowCoast team (Issue #44)

[EXTERNAL to VIMS received message]

I am concerned on how user will interpret for example this area in NJ (along Hackensack River/ near Meadowlands Sports Complex) which has values up above 1.5

I suspect that a user will think these are being forecast to be inundated by STOFS3D.

[image: image.png]

On Fri, Oct 14, 2022 at 11:51 AM Fei Ye @.<mailto:@.>> wrote:

Linlin mask out inundations > 2 m in the previous plots. We can try setting the threshold at 1 m if necessary.

- Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1279180025, or unsubscribe https://github.com/notifications/unsubscribe-auth/APYSEZNTI7XECDEGKJILS4TWDF6QXANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.<mailto:@.>>

- Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fschism-dev%2Fpyschism%2Fissues%2F44%23issuecomment-1279188416&data=05%7C01%7Cyjzhang%40vims.edu%7C424c186ea46e4c03ed5a08daadfd33b0%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638013600262329306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=cCIpZ9zDTqK6IKT%2BTz1s8waBg9Mhy3fE9wEUhzVs%2BOc%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFBKNZ4YVWIPWO4TVW5OR4DWDF7RNANCNFSM54BFEXFA&data=05%7C01%7Cyjzhang%40vims.edu%7C424c186ea46e4c03ed5a08daadfd33b0%7C8cbcddd9588d4e3b9c1e2367dbdf1740%7C0%7C0%7C638013600262329306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=msgHGqerD13VTwjwbRKW1wCbGlNpMrV9ZxC2KktCG88%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.**@.>>

yangz888 commented 1 year ago

Thanks to everyone for the super helpful discussions! While we continue the discussion via emails, let's meet at tomorrow's meeting at 4:00 pm, Tuesday, Oct. 17. (I'll re-initiate the meeting invitation after posting this message.)

saeed-moghimi-noaa commented 1 year ago

@erin-nagel @agibbons-NOAA @josephzhang8 @cuill @gseroka @yangz888

All,

Following on today meeting:

I hope this works for everybody,

Thanks all, -Saeed

cuill commented 1 year ago

@yangz888

Hi Zizang,

The revised script is ready for your test: /work2/07933/cuivims/ICOGS3D/Forecast/PostP/For_zizang/gen_geojson.py

Linlin

yangz888 commented 1 year ago

Got it. Thanks, Linlin! - Zizang

On Thu, Oct 27, 2022 at 10:04 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888

Hi Zizang,

The revised script is ready for your test: /work2/07933/cuivims/ICOGS3D/Forecast/PostP/For_zizang/gen_geojson.py

Linlin

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293575269, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSDFQU7BTNLIVLB4JK3WFKDUBANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Hi Linlin,

Just double check with you: I see the following "20221027". Does it matter?

diff gen_geojson.py gen_geojson.py_v1 # gen_geojson.py is the new version; the _v1 is the old version.* 173,174c173,174 < #filenames = [f'./stofs3d_atlanticdisturbance{dates[0].strftime("%Y%m%d")}' \ < # + f't00z_{dates[i].strftime("%Y%m%d")}t{dates[i].strftime("%H")}z.gpkg' for i in range(len(times))]

filenames =

[f'./stofs3d_atlanticdisturbance{dates[0].strftime("%Y%m%d")}' \ + f't12z_{dates[i].strftime("%Y%m%d")}t{dates[i].strftime("%H")}z.gpkg' for i in range(len(times))] 176,178d175 < filenames = [f'stofs3d_atlantic_disturbance_n{i-1:03d}.20221027.t12z.gpkg' for i in range(24, 0,-1)] < for i in range(48): < filenames.append(f'stofs3d_atlantic_disturbance_f{i+1:03d}.20221027.t12z.gpkg')

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:04 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888

Hi Zizang,

The revised script is ready for your test: /work2/07933/cuivims/ICOGS3D/Forecast/PostP/For_zizang/gen_geojson.py

Linlin

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293575269, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSDFQU7BTNLIVLB4JK3WFKDUBANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

cuill commented 1 year ago

@yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

yangz888 commented 1 year ago

Copied. Thanks, Linlin! - Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Hi Erin and John,

Please see the newly created gpkg files with the new naming convention at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/GPKG_newNaming_2022_1010/. If the file data contents and names sound ok, I'll then submit the scripts to NCO for para testing.

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Zizang et al.,

I will get back to you before the COB today...

John

John G.W. Kelley Coastal Marine Modeling Branch Coast Survey Development Laboratory NOAA/National Ocean Service NOAA-UNH Joint Hydrographic Center Durham, New Hampshire 03824 603-862-1628

On Thu, Oct 27, 2022 at 12:06 PM Zizang Yang - NOAA Federal < @.***> wrote:

Hi Erin and John,

Please see the newly created gpkg files with the new naming convention at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/GPKG_newNaming_2022_1010/. If the file data contents and names sound ok, I'll then submit the scripts to NCO for para testing.

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Sounds good. Thanks, John! - Zizang

On Thu, Oct 27, 2022 at 1:56 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I will get back to you before the COB today...

John

John G.W. Kelley Coastal Marine Modeling Branch Coast Survey Development Laboratory NOAA/National Ocean Service NOAA-UNH Joint Hydrographic Center Durham, New Hampshire 03824 603-862-1628

On Thu, Oct 27, 2022 at 12:06 PM Zizang Yang - NOAA Federal < @.***> wrote:

Hi Erin and John,

Please see the newly created gpkg files with the new naming convention at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/GPKG_newNaming_2022_1010/. If the file data contents and names sound ok, I'll then submit the scripts to NCO for para testing.

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Zizang et al.,

I reviewed and realized that I should be consistent in notation. My apologies.

Would it be possible to use just . (dot) and remove the _.? This would make the naming convention similar to that used for NOS OFSs and Global ESTOFS (stofs2d) output files.

stofs3d.atlantic.disturbance.nHHH.YYYYMMDD.tCCz.gpkg

stofs3d.atlantic.disturbance.fHHH.YYYYMMDD.tCCz.gpkg

John

On Thu, Oct 27, 2022 at 3:00 PM Zizang Yang - NOAA Federal < @.***> wrote:

Sounds good. Thanks, John! - Zizang

On Thu, Oct 27, 2022 at 1:56 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I will get back to you before the COB today...

John

John G.W. Kelley Coastal Marine Modeling Branch Coast Survey Development Laboratory NOAA/National Ocean Service NOAA-UNH Joint Hydrographic Center Durham, New Hampshire 03824 603-862-1628

On Thu, Oct 27, 2022 at 12:06 PM Zizang Yang - NOAA Federal < @.***> wrote:

Hi Erin and John,

Please see the newly created gpkg files with the new naming convention at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/GPKG_newNaming_2022_1010/. If the file data contents and names sound ok, I'll then submit the scripts to NCO for para testing.

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Got it. Thanks, John! - Zizang

On Thu, Oct 27, 2022 at 4:04 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I reviewed and realized that I should be consistent in notation. My apologies.

Would it be possible to use just . (dot) and remove the _.? This would make the naming convention similar to that used for NOS OFSs and Global ESTOFS (stofs2d) output files.

stofs3d.atlantic.disturbance.nHHH.YYYYMMDD.tCCz.gpkg

stofs3d.atlantic.disturbance.fHHH.YYYYMMDD.tCCz.gpkg

John

On Thu, Oct 27, 2022 at 3:00 PM Zizang Yang - NOAA Federal < @.***> wrote:

Sounds good. Thanks, John! - Zizang

On Thu, Oct 27, 2022 at 1:56 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I will get back to you before the COB today...

John

John G.W. Kelley Coastal Marine Modeling Branch Coast Survey Development Laboratory NOAA/National Ocean Service NOAA-UNH Joint Hydrographic Center Durham, New Hampshire 03824 603-862-1628

On Thu, Oct 27, 2022 at 12:06 PM Zizang Yang - NOAA Federal < @.***> wrote:

Hi Erin and John,

Please see the newly created gpkg files with the new naming convention at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/GPKG_newNaming_2022_1010/. If the file data contents and names sound ok, I'll then submit the scripts to NCO for para testing.

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

cuill commented 1 year ago

@yangz888

The script on Frontera has been updated based on John's suggestion.

Linlin

yangz888 commented 1 year ago

Will test it. Thanks, Linlin! - Zizang

On Thu, Oct 27, 2022 at 4:37 PM cuill @.***> wrote:

@yangz888 https://github.com/yangz888

The script on Frontera has been updated based on John's suggestion.

Linlin

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1294036339, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSBNC5PTYJWEWUAG6QLWFLRYTANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Hi John and Lin,

Please see the outputs from the updated gen_geojson.py at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/gpkb_20221010_test2A/ .

Thank you, Zizang

On Thu, Oct 27, 2022 at 4:04 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I reviewed and realized that I should be consistent in notation. My apologies.

Would it be possible to use just . (dot) and remove the _.? This would make the naming convention similar to that used for NOS OFSs and Global ESTOFS (stofs2d) output files.

stofs3d.atlantic.disturbance.nHHH.YYYYMMDD.tCCz.gpkg

stofs3d.atlantic.disturbance.fHHH.YYYYMMDD.tCCz.gpkg

John

On Thu, Oct 27, 2022 at 3:00 PM Zizang Yang - NOAA Federal < @.***> wrote:

Sounds good. Thanks, John! - Zizang

On Thu, Oct 27, 2022 at 1:56 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I will get back to you before the COB today...

John

John G.W. Kelley Coastal Marine Modeling Branch Coast Survey Development Laboratory NOAA/National Ocean Service NOAA-UNH Joint Hydrographic Center Durham, New Hampshire 03824 603-862-1628

On Thu, Oct 27, 2022 at 12:06 PM Zizang Yang - NOAA Federal < @.***> wrote:

Hi Erin and John,

Please see the newly created gpkg files with the new naming convention at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/GPKG_newNaming_2022_1010/. If the file data contents and names sound ok, I'll then submit the scripts to NCO for para testing.

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Hi Linlin,

One question, I notice the nowcast files in https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/gpkb_20221010_test2A/, begins at n000 and ends at n023. Should it be begins at n001 and ends at n024? Note that the forecast files begins at f001 and ends at f048.

Thanks, Zizang

On Fri, Oct 28, 2022 at 9:11 AM Zizang Yang - NOAA Federal < @.***> wrote:

Hi John and Lin,

Please see the outputs from the updated gen_geojson.py at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/gpkb_20221010_test2A/ .

Thank you, Zizang

On Thu, Oct 27, 2022 at 4:04 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I reviewed and realized that I should be consistent in notation. My apologies.

Would it be possible to use just . (dot) and remove the _.? This would make the naming convention similar to that used for NOS OFSs and Global ESTOFS (stofs2d) output files.

stofs3d.atlantic.disturbance.nHHH.YYYYMMDD.tCCz.gpkg

stofs3d.atlantic.disturbance.fHHH.YYYYMMDD.tCCz.gpkg

John

On Thu, Oct 27, 2022 at 3:00 PM Zizang Yang - NOAA Federal < @.***> wrote:

Sounds good. Thanks, John! - Zizang

On Thu, Oct 27, 2022 at 1:56 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I will get back to you before the COB today...

John

John G.W. Kelley Coastal Marine Modeling Branch Coast Survey Development Laboratory NOAA/National Ocean Service NOAA-UNH Joint Hydrographic Center Durham, New Hampshire 03824 603-862-1628

On Thu, Oct 27, 2022 at 12:06 PM Zizang Yang - NOAA Federal < @.***> wrote:

Hi Erin and John,

Please see the newly created gpkg files with the new naming convention at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/GPKG_newNaming_2022_1010/. If the file data contents and names sound ok, I'll then submit the scripts to NCO for para testing.

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

cuill commented 1 year ago

@yangz888

This email communication can answer your question. I attached correct picture this time.

Hi Linlin,

I wanted to make sure if I understood correctly. Per the picture below, the model start time is 2022102512, and the forecast start time is 2022102612. Based on my understanding, the filenames will start with n024.20221026.t12z.gpkg and end with n000.20221026.t12z.gpkg for nowcast, and start with f000.20221026.t12z.gpkg and end with f048.20221026.t12z.gpkg for the forecast. Is this correct?

FYI, SCHISM won’t output t00, which means no n024.20221026.t12z.gpkg. Also, n000.20221026.t12z.gpkg and f000.20221026.t12z.gpkg are the same, which one would you prefer to keep, or both?

John

=================================================================================================================================================================

On Wed, Oct 26, 2022 at 5:41 PM Linlin Cui [lcui@vims.edu](mailto:lcui@vims.edu) wrote: Hi John,

Thanks for the information.

I wanted to make sure if I understood correctly. Per the picture below, the model start time is 2022102512, and the forecast start time is 2022102612. Based on my understanding, the filenames will start with n024.20221026.t12z.gpkg and end with n000.20221026.t12z.gpkg for nowcast, and start with f000.20221026.t12z.gpkg and end with f048.20221026.t12z.gpkg for the forecast. Is this correct?

FYI, SCHISM won’t output t00, which means no n024.20221026.t12z.gpkg. Also, n000.20221026.t12z.gpkg and f000.20221026.t12z.gpkg are the same, which one would you prefer to keep, or both?

Thanks,

Linlin

Screen Shot 2022-10-28 at 9 56 29 AM

yangz888 commented 1 year ago

Thanks, Linlin! - Zizang

On Fri, Oct 28, 2022 at 10:08 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888

This email communication can answer your question. I attached correct picture this time.

Hi Linlin,

I wanted to make sure if I understood correctly. Per the picture below, the model start time is 2022102512, and the forecast start time is

  1. Based on my understanding, the filenames will start with n024.20221026.t12z.gpkg and end with n000.20221026.t12z.gpkg for nowcast, and start with f000.20221026.t12z.gpkg and end with f048.20221026.t12z.gpkg for the forecast. Is this correct?

    • YES (I assume you have typos on the right side of the picture...you wanted all 'f' instead of n and N from f000 to f048)

FYI, SCHISM won’t output t00, which means no n024.20221026.t12z.gpkg. Also, n000.20221026.t12z.gpkg and f000.20221026.t12z.gpkg are the same, which one would you prefer to keep, or both?

  • I would prefer to keep n000.20221026.t12z.gpkg and start forecast files from f001.

John

=================================================================================================================================================================

On Wed, Oct 26, 2022 at 5:41 PM Linlin Cui @.***> wrote: Hi John,

Thanks for the information.

I wanted to make sure if I understood correctly. Per the picture below, the model start time is 2022102512, and the forecast start time is

  1. Based on my understanding, the filenames will start with n024.20221026.t12z.gpkg and end with n000.20221026.t12z.gpkg for nowcast, and start with f000.20221026.t12z.gpkg and end with f048.20221026.t12z.gpkg for the forecast. Is this correct?

FYI, SCHISM won’t output t00, which means no n024.20221026.t12z.gpkg. Also, n000.20221026.t12z.gpkg and f000.20221026.t12z.gpkg are the same, which one would you prefer to keep, or both?

Thanks,

Linlin

[image: Screen Shot 2022-10-28 at 9 56 29 AM] https://user-images.githubusercontent.com/6502663/198630194-06c2d023-7b32-4cb1-b247-942278f2023a.png

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1295046314, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSBNCBXVQDTBKY6MLATWFPM5RANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

yangz888 commented 1 year ago

Hi Linlin,

Could you help revise the gen_geojson.py to let the output file names be as follows:

stofs_3d_atl.t${cyc}z.disturbance.n/f{hr}.gpkg

Thanks, Zizang

FYI, John agrees to use the above naming convention. The NCO has the following request:

By the way, I just noticed gpkg file name is stofs3d.atlantic.disturbance.n{hr}.yyyymmdd.{cycle}.gpkg, which is not following other file naming convention. Is the name requested by downstream user? Can we change to stofs_3d_atl.t${cyc}z.disturbance.n/f{hr}.gpkg? similar to other file names? I don't want another yyyymmdd in the filename when we have directory name with stofs_3d_atl.yyyymmdd already unless the filename's yyyymmdd is different from directory's one.

On Thu, Oct 27, 2022 at 4:07 PM Zizang Yang - NOAA Federal < @.***> wrote:

Got it. Thanks, John! - Zizang

On Thu, Oct 27, 2022 at 4:04 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I reviewed and realized that I should be consistent in notation. My apologies.

Would it be possible to use just . (dot) and remove the _.? This would make the naming convention similar to that used for NOS OFSs and Global ESTOFS (stofs2d) output files.

stofs3d.atlantic.disturbance.nHHH.YYYYMMDD.tCCz.gpkg

stofs3d.atlantic.disturbance.fHHH.YYYYMMDD.tCCz.gpkg

John

On Thu, Oct 27, 2022 at 3:00 PM Zizang Yang - NOAA Federal < @.***> wrote:

Sounds good. Thanks, John! - Zizang

On Thu, Oct 27, 2022 at 1:56 PM John Kelley - NOAA Federal < @.***> wrote:

Zizang et al.,

I will get back to you before the COB today...

John

John G.W. Kelley Coastal Marine Modeling Branch Coast Survey Development Laboratory NOAA/National Ocean Service NOAA-UNH Joint Hydrographic Center Durham, New Hampshire 03824 603-862-1628

On Thu, Oct 27, 2022 at 12:06 PM Zizang Yang - NOAA Federal < @.***> wrote:

Hi Erin and John,

Please see the newly created gpkg files with the new naming convention at https://noaa-gestofs-pds.s3.amazonaws.com/index.html#_stofs-3D-Atlantic/GPKG_newNaming_2022_1010/. If the file data contents and names sound ok, I'll then submit the scripts to NCO for para testing.

Thanks, Zizang

On Thu, Oct 27, 2022 at 10:15 AM cuill @.***> wrote:

@yangz888 https://github.com/yangz888 I just changed hard-coded 20221027 to dates[30]. Please copy the latest one.

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1293591633, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSFSTPMYRHGC64TEYTTWFKFALANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>

cuill commented 1 year ago

@yangz888

The script was revised to change the file names.

/home1/07933/cuivims/work2/ICOGS3D/Forecast/PostP/For_zizang/gen_geojson.py

yangz888 commented 1 year ago

Thanks, Linlin! - Zizang

On Mon, Oct 31, 2022 at 12:05 PM cuill @.***> wrote:

@yangz888 https://github.com/yangz888

The script was revised to change the file names.

/home1/07933/cuivims/work2/ICOGS3D/Forecast/PostP/For_zizang/gen_geojson.py

— Reply to this email directly, view it on GitHub https://github.com/schism-dev/pyschism/issues/44#issuecomment-1297318986, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7TWSAEXOCKDIX2555V26LWF7U4TANCNFSM54BFEXFA . You are receiving this because you were mentioned.Message ID: @.***>