Collecting WAF scripts

[robragsdale]

@rsignell-usgs could you offer up the scripts that you have created/use to create WAFs? We would like to post these and others that we get, starting with the Wiki page https://github.com/ioos/registry/wiki/Python-Scripts-for-creating-WAFs.

Thanks.

[robragsdale]

@kwilcox we are gathering scripts that have been written to help in create a WAF. Is there anything that has been written for AOOS and CeNCOOS that can be shared here? I am putting everything that we get on this Wiki page https://github.com/ioos/registry/wiki/Python-Scripts-for-creating-WAFs. Thanks.

[robragsdale]

@kknee @lukecampbell have any scripts been written for GLOS that we can include here https://github.com/ioos/registry/wiki/Python-Scripts-for-creating-WAFs? Thanks.

[lukecampbell]

I don't believe we have any scripts that generate WAFs.

[dpsnowden]

@lukecampbell @robragsdale This is a good start for GLOS. https://github.com/glos/ you may also need to ask Tad S. GLOS may not maintain a WAF yet but they do have a pretty sophisticated ISO management system involving github. I think they were the original use case for metadown and thredds_crawler.

[kknee]

Right, GLOS does not maintain a WAF (that I'm aware of). Metadown scrapes ISO from THREDDS and GeoNetwork and SWH from 52N, ISOs are then stored in a BaseX DB to support the GLOS search functionality.

[robragsdale]

The list of RA WAFs on this Wiki page https://github.com/ioos/registry/wiki/List-of-RAs-that-maintain-WAFs have been registered with the Registry.

[kwilcox]

@robragsdale @lukecampbell Take a look here: https://github.com/kwilcox/thredds_crawler#use-case. This is a script that can be used to harvest ISO files from THREDDS catalogs. Both CeNCOOS and AOOS use something very similar to harvest ISO files into WAFs. There is also a higher level package that uses thredds_crawler internally to harvest THREDDS catalogs: https://github.com/asascience-open/metadown. metadown also supports harvesting from GeoNetwork.

[rsignell-usgs]

Aren't these the GLOS ISOs? https://github.com/glos/glos_catalog/tree/master/ISOs

[rsignell-usgs]

I just run ncISO in a bash script, edited manually to specify catalogs, like this:

#!/bin/bash
for full_cat in http://tds.marine.rutgers.edu/thredds/catalog/roms/espresso/2013_da/avg_Best
/catalog.xml \
http://tds.marine.rutgers.edu/thredds/catalog/roms/espresso/2013_da/his_Best/catalog.xml \
http://tds.marine.rutgers.edu/thredds/roms/espresso/2009_da/catalog.xml
do
  echo $full_cat
  java -Xms1024m -Xmx1024m -jar /usgs/data2/rsignell/waf/ncISO-2.3.jar \
    -ts ${full_cat} -num 100000 -depth 20 -iso true \
    -waf /usgs/data2/rsignell/waf/data/rutgers
done

Eric Bridger's approach of calling ncISO from python (above) is more powerful.