Top results on study landing page, solicited from users

[jimmymathews]

The web application has, from the point of view of data structures, a relatively well-constrained system of available operations. The metrics that are calculated, the comparisons that are made, etc., in many cases lead to a simple result statement with p-value in the form of that verbalization.

Now that such findings are also reproducible with the URL, we are technically close to being able to solicit significant findings from the users at runtime, to eventually show up on the findings listing which is shown to users. The reproduction part of this is already solved.

Still, adding such a "leaderboard" would require working out a number of new things.

1. Attribution

I think user-contributed findings should come with attribution. Unless the user wishes to be anonymous for some reason. But how is user identity defined? Possible options:

• Use ORCID's verification API. I think I remember that this is pretty well documented.
• Something similar for GitHub accounts.
• Something similar for accounts with some other sites with public-facing profiles? LinkedIn?

2. Manual filtering

The contributed findings should end up in a queue that we manually review before anything gets promoted to the public listing. I am thinking about possible cases where a person complains that they are being impersonated, or wants a result which is attributed to them to be taken down, etc.

3. Data policy (our first POST request)

So far the API is technically read-only (although many requests trigger complex computations and caching on the backend). This has made security less of a concern and privacy not a concern at all. But to have users submit data directly, especially data with association to identity, we will need to make sure our policies are in order. Things like ownership, consent, data usage, privacy. We also need to make sure we are minimally compliant with any regulations or MSK policies, etc.

4. The listing widget itself

Currently the findings list is a sentence with a URL to reproduce. It shows up on the study summary page as a list of links. But for the proposed improvement, we would need something more like a table, with p-value if available and associated "quantity" (multiplier, statistic, whatever) available for sorting, to give the sense of "ranking" results. So the data model also needs to be updated to incorporate the contributed findings. Currently the data model is not really changed, we added the findings table as an afterthought.

[jimmymathews]

This ORCID documentation page explains how to use the "Implicit OAuth" workflow, and they recommend it in pretty much exactly our case: short-lived credentials, the user does very minimal authorization (we just need a name and ORCID ID), no backend server needed.

[jimmymathews]

However, since we still will be expecting to receive payloads from clients with these ORCID IDs, and we ourselves need to make sure they are verified (not just according to the browser), there is a possibility that we'll still need the normal server-handled OAuth. One way or another we need to verify that the identification succeeded according to ORCID, not just according to the client/browser.