search

USGCRP / gcis

Global Change Information System
https://data.globalchange.gov
Other
20 stars 16 forks source link

dcterms:subject (free-text) #175

Closed justgo129 closed 9 years ago

justgo129 commented 9 years ago

Inspired by the discussion of free-form text strings at: https://github.com/USGCRP/gcis/issues/150 hence I'm broaching it at this point.

Regarding the use of "attributes" for images, e.g.: http://data.globalchange.gov/image/ff6a7a8e-d886-4b30-acd7-a3538a787baf

Note that the line beginning with "dcterms:subject" also contains multiple objects.

I can think of the use case where someone wishes to query all images pertaining to "precipitation." As written now at: http://data.globalchange.gov/image/ff6a7a8e-d886-4b30-acd7-a3538a787baf (1) Will this image come up given the multiple objects? (2) If not, could we fix this?

Thanks.

bduggan commented 9 years ago

1) Yes, I believe t is possible to construct such a SPARQL query.

justgo129 commented 9 years ago

I'd like to reopen this but don't have the permissions. I'l provide the rationale upon the reopening (it would make it easier to follow on my part).

bduggan commented 9 years ago

What is the rationale?

justgo129 commented 9 years ago

@zednis is it a problem for dcterms:subject to contain multiple objects? Basically, it's an extension of the discussion in https://github.com/USGCRP/gcis/issues/150 where a similar discussion occurred vis a vis output file names (i.e. splitting them up in the turtle or not).

Once I gain some clarification on it, I am all right with closing.

zednis commented 9 years ago

Two things:

1) We should avoid collapsing multiple values into single literals. The property is not intended to reference an embedded collection but a single value. The correct way to do this in RDF is to make multiple RDF statements reusing the property with different values.

2) dcterms:subject is intended to be used as an object property and not with literal values.

from http://dublincore.org/documents/dcmi-terms/#terms-subject

This term is intended to be used with non-literal values as defined in the DCMI Abstract Model (http://dublincore.org/documents/abstract-model/). As of December 2007, the DCMI Usage Board is seeking a way to express this intention with a formal range declaration.

Most terms in dcterms are intended to be used with non-literal values. This is what separates dublin core terms properties from regular dublin core properties - which are intended to be used with literal values.

Guidance on usage at available at http://wiki.dublincore.org/index.php/User_Guide/Publishing_Metadata#dcterms:subject

The correct representation in RDF for dcterms:subject would be like this:

<http://data.globalchange.gov/image/ff6a7a8e-d886-4b30-acd7-a3538a787baf>
  dcterms:subject <http://data.globalchange.gov/term/Precipitation> ,
                            <http://data.globalchange.gov/term/observed>,
                            <http://data.globalchange.gov/term/global> .

This is why dcterms:subject is usually used to reference an instance of a skos:Concept.

The correct representation in RDF for dc:subject would be like this:

<http://data.globalchange.gov/image/ff6a7a8e-d886-4b30-acd7-a3538a787baf>
  dc:subject "Precipitation"^^xsd:string, 
                   "observed"^^xsd:string, 
                   "global"^^xsd:string .
zednis commented 9 years ago

@bduggan this issues should be reopened and we should modify your usage of dcterms:subject because it is incorrect.

bduggan commented 9 years ago

I would propose removing dcterms:subject from the rdf. This falls into the larger category of controlled vocabularies which is work in progress. Again, this data was provided as a free form text field, and we do not want to make any assertions about splitting the contents and making URIs for various phrases there.

zednis commented 9 years ago

I think this information is too important to just drop. What if we have a student work on a post-process to split free-text provided contents into separate text keywords. We can then use either dc:subject or dcterms:subject as mentioned above.

bduggan commented 9 years ago

The lists in the attributes are not constrained, and there are no definitions. An example: "Precipitation, projections, seasonal, CMIP5, RCP2.6" -- collections of keywords and a model and a scenario. I would argue that treating these as dcterms:subject's is not a good idea -- better would be for the image to be associated with /model/cmip5 and /scenario/rcp2.6. Once we start properly managing controlled vocabularies we will want definitions for each of the terms as well as how they relate to other controlled vocabularies.

I think these lists are important to keep in mind and should help inform our discussions about representing (and curating) controlled vocabularies.

zednis commented 9 years ago

I don't want to just throw this data out.

I think it makes sense to use dc:subject in the meantime before we have controlled vocabularies fully in use - and it will allow us to support keywords that don't map to a term in a controlled vocabulary.

We should do some data science to post-process the free-text data we have into delimited keywords, but thats why we have students ;-)

bduggan commented 9 years ago

I'm not so worried about the effort of splitting these up, I'm more worried about making a category mistake with this data. Besides models and scenarios, there are also regions and even platforms/instruments: things for which we already have URIs.

Anyway, eventually, yes, we do want URIs for term, perhaps under "/term" (in your example)., To support this, we need to decide what is returned from the /term endpoint. i.e. what are the attributes and relationships of a term? (not in the lexicon sense but in the controlled vocabulary sense.)

Also, I wasn't suggesting throwing it out, just leaving it out of the RDF for now; it'll still be in the database and we can add it once we iron out our representation of controlled vocabularies.

aulenbac commented 9 years ago

@bduggan is right. Think engineered. Think lean and automated. And think of an operational GCIS ontology that is automatically generated on a weekly schedule using that lean automated approach. Manual work is not an option; we do not have students for new work like this nor should we assume we shall in the future.

Actually, I am okay with throwing this "data" out. Based on two years of GCIS work, Attributes is a spotty, usually blank, catch-all that few bother to fill out. When they do, the values are amazingly varied and not terribly useful. Perhaps we should remove the Attributes field completely.

justgo129 commented 9 years ago

I'd be all right with removing the "attributes" field completely given the lack of criteria for the process of assigning them. Let me confirm first that this is so and will update.

zednis commented 9 years ago

ok, I have been persuaded to the leave this content out of the RDF for now. I support Brian's suggesting of revisiting this and eventually bringing it back with controlled vocabularies.

justgo129 commented 9 years ago

Works for me.

On Wed, Jun 24, 2015 at 2:26 PM, Stephan Zednik notifications@github.com wrote:

ok, I have been persuaded to the leave this content out of the RDF for now. I support Brian's suggesting of revisiting this and eventually bringing it back with controlled vocabularies.

— Reply to this email directly or view it on GitHub https://github.com/USGCRP/gcis/issues/175#issuecomment-114972411.

Justin Goldstein, Ph.D. Advance Science Climate Data and Observing Systems Coordinator US Global Change Research Program 1717 Pennsylvania Ave NW, Suite #250 Washington, DC 20006

O: (202) 419-3496 M: (202) 285-3005

e-mail: jgoldstein AT usgcrp Dot gov http://www.globalchange.gov