search

OpenTreeOfLife / opentree

Opentree browsing and curation web site. For overarching or cross-repo concerns, please see the 'germinator' repo.
http://tree.opentreeoflife.org/
BSD 2-Clause "Simplified" License
108 stars 26 forks source link

Collect a URL when user submits classification/phylogeny advice #744

Closed jar398 closed 8 years ago

jar398 commented 9 years ago

If the user selects 'Feedback on tree of life' then some effort should be made to collect from the user a URL of an authoritative source (preferably a DOI). This is a requirement we have on taxonomy changes, and the feedback given here may bear on the taxonomy, if it bears on phylogeny it's just as important.

Something like 'When proposing a correction or improvement to the tree, please provide the DOI or URL of a published source or database that documents the desired topology or nomenclature.'

jar398 commented 9 years ago

Here is an example of what I'm talking about: https://github.com/OpenTreeOfLife/feedback/issues/101 The submitter provided a free-text reference, but no URL. We really really want a URL, ideally a DOI, in these cases.

jar398 commented 9 years ago

of course they didn't use the feedback form for this submission since the feature is down now, but you get the idea.

jar398 commented 8 years ago

I wonder if we could increase the priority of this one. I've received probably 20 or 30 URL-less taxonomy corrections, and more are coming. Here's just one: https://github.com/OpenTreeOfLife/feedback/issues/153

kcranston commented 8 years ago

Good idea. I can probably tackle this tomorrow.

jimallman commented 8 years ago

@kcranston, are you working on this one? If not, I think I can knock it out pretty quickly.

jimallman commented 8 years ago

One more question: Should this URL be a required field?

jar398 commented 8 years ago

Good question. When the feedback is scientific advice, then I think yes, a URL should be required. But other kinds of feedback, such as "it would be nice if the labels were in Comic Sans", don't need a URL.

If someone really wants to dig in their heels and provide their advice without evidence, they can provide a nonsense URL like about:blank or http://google.com/.

jimallman commented 8 years ago

other kinds of feedback, such as "it would be nice if the labels were in Comic Sans", don't need a URL.

Certainly. I thought we'd hide or show this field based on the feedback type selected. (Don't we do some of this already?)

jar398 commented 8 years ago

Hmm. There are kinds of taxonomy advice that do not require URLs, e.g. duplicates. So now I'm not so sure about what I said about absolutely requiring a URL and nonsense URLs. E.g. https://github.com/OpenTreeOfLife/feedback/issues/189 (I wonder why so many feedback issues have no flags associated with them?...)

jimallman commented 8 years ago

I wonder why so many feedback issues have no flags associated with them?

I'm guessing some users are just going directly to the GitHub issue tracker, bypassing the webapp entirely.

jimallman commented 8 years ago

Hmm. There are kinds of taxonomy advice that do not require URLs, e.g. duplicates. So now I'm not so sure about what I said about absolutely requiring a URL...

The latest code shows (and requires) this field if the curator chooses "Reporting an error in phylogeny".

screen shot 2016-02-07 at 5 59 14 pm

The validation could also consider the sub-type of feedback, with only some requiring this URL:

screen shot 2016-02-07 at 5 55 20 pm
jar398 commented 8 years ago

The natural way to describe a duplicate - if you don't understand the difference between the taxonomy and the phylogeny, and most people won't - is as an error in phylogeny - which could plausibly be interpreted to mean error in the phylogenetic tree (although that interpretation is not correct).​ And if the nodes are close enough together, it's an error, and saying so requires no evidence.

I don't understand what "node placement" means... the tree is not about where nodes are placed, it's about relationships between taxa (or, equivalently, which groupings are or aren't monophyletic).

I think a better category name might be something like "about relationships shown in the synthetic tree". Anything you claim about relationships has to have evidence. Reporting a duplicate would then be forced into a different category, either taxon data or general.

But hmm. There are things you can say about relationships that don't require evidence, such as "I didn't know skunks were weasels!" - so the category of comments requiring URLs is actually corrections and enrichments to the tree, not just anything "about" the tree.

What is a scenario in which someone would want to comment "about node placement in the source tree"? The comment form isn't even available when you're looking at a source tree. Maybe instead ask about phylogenetic sources that have better or additional information? So the category could be something like "about additional phylogenetic sources" - which overlaps with the other one, but maybe that's ok. But maybe we should remove this category altogether.

Would be worth reviewing the feedback we've received so far to see if it falls into natural categories.

I think this deserves more discussion, and have marked it so.

jimallman commented 8 years ago

I think this deserves more discussion, and have marked it so.

Agreed. Please note that these options (for "intended scope" of synth-tree feedback) were based on my (very) incomplete understanding when the feedback tool was built. Suggestions for more sensible options are most welcome!

Actually, @kcranston and I were actually on the verge of removing this selection in the other pending feedback-refactoring branch. It will be interesting to see what choices people have made so far.

jar398 commented 8 years ago

Yeah, I figured, sorry to give you a hard time.

Maybe get rid of the 2nd pulldown and simply provide one:

What kind of feedback is this? General comment Report a correction to relationships in the synthetic tree Suggest additional phylogenetic information to incorporate Correction to names (taxonomy) Bug report (website behavior) New feature request

Thoughts, @kcranston?

The URL would be required for correct, additional, and taxonomy, but not for the others.

I just did a quick scan of the feedback issues. Often people assign the wrong category. I think the above will be a bit clearer, but not foolproof. The vast majority of issues have to do with names, not phylogeny. By describing categories as "correction" or "information" my hope is that people who only have questions or comments should, in principle, be steered to other categories and won't hit the URL-required-when-none-makes-sense trap.

But this takes us full circle on duplicates! Duplicates are reasonably described as name corrections, and we want URLs for name corrections, in general. Maybe there needs to be a checkbox for "it's goddamn obvious, you idiots" or "this is so obvious no evidence is needed" which, when checked, means the form won't insist on a URL.

(I think this is important because the feedback issues seem to be a successful crowdsourcing focus for us, and if we can just get people to provide URLs the issue corpus will be much more valuable.)

jimallman commented 8 years ago

The work so far is available for review on devtree.

jar398 commented 8 years ago

Not that we really need more, but just for the record, here is another case where this feature would have been nice https://github.com/OpenTreeOfLife/feedback/issues/221 I replied to the issue with a request, but got no response. Next step for me (as OTT manager) is to send personal email.

jimallman commented 8 years ago

Note: This conversation has moved to the comments for the related pull request.