oxinabox / DataDepsGenerators.jl

Utility for developers to help define DataDeps registration blocks, for reusing existing Data with DataDeps.jl
Other
18 stars 6 forks source link

generate does not accept a datadep_name #54

Closed hearnsj closed 6 years ago

hearnsj commented 6 years ago

Julia 0/6/4 Windows 10 Testing with the Google dataset search https://toolbox.google.com/datasetsearch/search?query=NOAA%20Optimum%20Interpolation%201%2F4%20Degree%20Daily%20Sea%20Surface%20Temperature%20(OISST)%20Analysis%2C%20Version%202&docid=p2%2FAYFQuCU3hv%2FksAAAAAA%3D%3D

I am sure I am calling the generate function wrong … please enlighten me

julia> generate("https://data.nodc.noaa.gov/cgi-bin/iso?id=gov.noaa.ncdc:C00844","oisst") ERROR: MethodError: no method matching generate(::String, ::String) Closest candidates are: generate(::DataDepsGenerators.DataRepo, ::Any; kwargs...) at C:\Users\hearn.julia\v0.6\DataDepsGenerators\src\DataDepsGenerators.jl:173 generate(::Array{T,1} where T, ::Any; shortname, show_failures) at C:\Users\hearn.julia\v0.6\DataDepsGenerators\src\DataDepsGenerators.jl:183 generate(::Any; kwargs...) at C:\Users\hearn.julia\v0.6\DataDepsGenerators\src\DataDepsGenerators.jl:177

oxinabox commented 6 years ago

No, you are doing it right. I think we have just missed the method

oxinabox commented 6 years ago

I haven't yet fixed the bug with the dataname not being accepted, (just waiting til #55 is all done) but it is pleasing to see the thing itself works (so long as you don't pass in the shortname)


julia> generate("https://data.nodc.noaa.gov/cgi-bin/iso?id=gov.noaa.ncdc:C00844"; show_failures=true) |> println

register(DataDep( 
"NOAA Optimum Interpolation 14 Degree Daily Sea Surface Temperature (OISST) Analysis, Version 2",                                  """
        Dataset: NOAA Optimum Interpolation 1/4 Degree Daily Sea Surface Temperature (OISST) Analysis, Version 2
        Website: https://data.nodc.noaa.gov/cgi-bin/iso?id=gov.noaa.ncdc:C00844
        Author: Cite dataset when used as a source: Richard W. Reynolds, Viva F. Banzon, and NOAA CDR Program (2008):
        Date of Publication: November 12, 2008

        This high-resolution sea surface temperature (SST) analysis product was developed using an optimum interpolation (OI) techn
ique. The SST analysis has a spatial grid resolution of 0.25 (1/4) degree and temporal resolution of 1 day. The product uses Advanc
ed Very High Resolution Radiometer (AVHRR) satellite data from the Pathfinder AVHRR SST dataset when available for September 1981 t
hrough December 2005, and the operational Navy AVHRR Multi-Channel SST data for 2006 to the present day. Pathfinder AVHRR SST was c
hosen because of good agreement with the in situ observation data. The product also uses sea ice datasets, in situ data from shipsa
nd buoys, and includes a large-scale adjustment of satellite biases with respect to the in situ data. In areas where sea ice is pre
sent, SST is estimated from sea ice concentration datasets from NASA GSFC before 2005 and then from NOAA NCEP from 2005 onwards. Pr
eliminary products are produced daily in near real-time (1-day latency) and may be updated in the first few days if critical input
data become available after the initial production time. After two weeks, a complete or final product is generated with no addition
al changes expected. The preliminary products from near real-time data productions began in October 2008 for Version 2.0. The produ
ct is in netCDF-3 following the Climate and Forecast (CF) Conventions, and is accompanied by algorithm documentation, data flow dia
gram and source code under the NOAA CDR Program.
        """,
        Any[],
))

Interesting, the metadata they provide has the spot for the data url. (That is the distribution field in JSON-LD) But they don't fill it.