openfoodfacts / folksonomy_api

A light REST API designed for Open Food Facts folksonomy engine
https://wiki.openfoodfacts.org/Folksonomy_Engine
GNU Affero General Public License v3.0
11 stars 7 forks source link

Conventions and UIs for grouped properties-values #151

Closed CharlesNepote closed 4 months ago

CharlesNepote commented 11 months ago

Use cases

Properties related to revisions

Sometimes we need to tag some versions of a product. For example, we would like to express that there is a data issue on the revision 310 of a product, and this issue is now fixed.

Reusing a group of property with different values

Sometimes we need to give context to some properties. For example we could want to monitor recalled products. But a product can be recalled more than one time. Having recalled_product=yes does not have much context: is it finished? when? For a given recall, we would like to know many more information related to this particular recall.

Suggestion 1

We can multiply the properties like this:

Suggestion 2

We can store the revision in the database. 3749372783457:310 => data_issue=Wrong serving size

But we need to modify both the database and the interfaces. And this solution does not solve the problem when properties are reused.

Suggestion 3

We can use the namespaces:

We can even use the namespace to tag more things on a specific revision. Eg.

With suggestion 3, we would only need to modify interfaces: we should be able to list all the data_issues, and not just the ones corresponding to data_issue:rev_310.

alexgarel commented 10 months ago

Regarding Suggestion 3

I don't thing this is interface only, because taking this approach, after a while the list of properties in the backend will send you back a list of millions of properties, as now the data is inside !

We could mitigate that by adopting a convention to mark pieces of the property that are "values" (and not static identifiers), like the revision number ?

In your example values are digits, but that might not always be the case.

I don't have a good idea in mind for that.

But really I don't think it solves our problem without changing the backend, because we won't be able to have the list of properties in a simple SQL request.

Suggestion 4

The problem here is that for each key in folksonomy engine we must have only one value (otherwise I would have propose to add variable part in the value).

So I would introduce a new concept of "variable part" in the key called args. So we would change the interface and the database model to accept key,args,values

This of course introduce new interfaces in folksonomy engine to get the list of args for a key, etc. It's not cheap… We could also in some way maintain the current interface through conventions and parsing, to put the args part in the key, but don't list them when we list keys.

You will have something like:

data_issue(rev_310,1)=Wrong serving size encoded as `k=data_issue, v='Wrong serving size', args=rev_310,1

alexgarel commented 7 months ago

re-reading this, I think suggestion 3 is the best, but with suggestion 4 for encoding.

That is: