A moderation tool for Open Food Facts

[raphael0202]

Context (extracted from NGI ethical report)

Open Food Facts has been started in 2012. The project mainly trusts users to enter good information, and it worked quite well until now.

Machine learning is used to find photos with inappropriate content, so far it only posts to a Slack channel, daily watched off by some volunteer moderators, which manually remove those images from the database. Users might signal us errors, but until now they have to reach us by email, Slack or other social media.

We know that current status is not ideal. For example we don’t have a real processing queue for inappropriate images or content, which does not make moderation robust. We would also need some scores on edits to signal eventual “dangerous” edits using heuristics. The system for a moderator to assert review on a product is also very basic, because it can’t be done, part by part, and we do not provide a differential since last verification (which could help speed them up). We would need a specific project to make this happen. We tried to bootstrap a project for Google Summer Of Code but we were not retained this year. The project description can be found here: https://wiki.openfoodfacts.org/GSOC/2023_ideas_list#Data_Quality_.26_Moderation_Hub

What we would like to have:

• enable users to flag products’ issues (image, fields) directly in the platform and the mobile application
• enable users to flag search results’ issues directly in the platform and the mobile application
• have a list of all moderation issues, fed by machine learning, heuristics or users, with a priority
• expose task lists to moderators for efficient resolution
• monitor the resolution of each issue by moderators

Possible moderation tasks:

1. remove image(s)
2. move image(s) to another product or project (OBF, OPFF, OPF,...)
3. Remove text fields containing personal information or spam
4. move full product (image + data) to another project. We need to support product merge in the long-run.
5. revert edit

Where should moderation live?

Data quality and moderations tasks are very similar: in both case we need to examine the product and take some action.

Many moderation tasks can be expressed under the insight paradigm of Robotoff.

Data quality tasks:

1. Fix nutrition/ingredient of the product (with possibly ML suggestions):

   1. fix errors (data and image don't match)
   2. detected outdated data

2. Unselect image

We need a queue of tasks, ranked by urgency (the scoring used can be discussed later). A user (or moderator for moderation tasks) handles a new task and mark it as completed. For simple tasks such as image deletion, the task can handled directly by the service: a question similar to the ones we have in Robotoff works well. For more complex task, the user can fix the issue on the service web interface.

At first, I considered coding this moderation system as a new service, but Robotoff could also be a good candidate:

The system offers many of the requirements of a task queue:

1. read/write integration with Product Opener
2. checks on data validity in importers: Does the image/product still exist?
3. we already have mechanisms to update/delete insights when changes are made on Product Opener

Proposal 1: use product_insight table

The product_insight table may be used for storing these kind of tasks.

Pros

• Most of the fields in the product_insight table would be reused as it (ex: id, barcode, type, timestamp, completed_at, username, annotation, n_votes, source_image).
• Very small code changes required, we only need to adapt the current insight table and add new importers/annotator.

Cons

• Some fields in product_insight don't make much sense (value, value_tag, predictor)
• we have several data sources (ML prediction, user report), and the product_insight table does reflect this: we would have to store the user report data inside product.data.
• it makes the Robotoff code harder to understand

Note: we would need to add some fields to product_insight table to deal with this (moderator_only, user_reports).

We also need to be able to limit some tasks to moderators.

Proposal 2: create a new task table

We can create a new task table. A task can be marked as completed, by someone.

These tasks would only contain non-binary tasks.

In any case, we need a user_report table: a user reported an issue with an image, a product or a product field. A moderation task is created from user reports.

Proposal 3: create a separate service

Pros

• easier to understand than other proposal
• separation of concerns

Cons

• Hunger games would have to fetch from Robotoff + this new service
• we cannot reuse insight logic implemented in Robotoff
• where should data quality tasks be handled? Robotoff or this new repository? The boundary is thin.

[alexgarel]

Just to make it clear, there are different sort of problems: vandalism or offending content, fraud and errors. The first one is the most important target at the moment. Second one is also important (by fraud, I imagine a tentative to slander a specific product/brand or, on the contrary to display a better score).

You did not mention the way moderation tasks are created, and this is important:

1. user of the website / the mobile app signal a problem on a. an image b. part of the data (eg offending product name) c. part of the website (attack mis-using a url to prompt text, or offending translation, or other attack on the website)
2. an algorithm detect a problem (mainly robotoff), here again on image or data.

We must have an API to create moderation issues, but the source must be clearly identified (eg. username).

On the contrary to robotoff we should be able to have discussions on issues, this might not be always evident which action to take and one might want to ask others, and it's better to have a track of exchanges.

I'm not in favor of augmenting robotoff for:

• Insight table is already big and adding extra stuff there will run the risk of having a bloated table (moreover moderation tasks would not be generated from predictions).
• It's already hard for a newcomer to hack in robotoff

I'm not sure that hungergame should be the tool to use for moderation, there again I would prefer to keep distinct tools. (it's not a strong opinion)

Of course having as much of open food facts API logic in the SDK would help.

It's true that the "moderation task update" on product change problem is quite of a twin to "insight update", but for me it's robotoff which creates moderation task based on an insight, and if an insight disappear, close the moderation task. So we should have a field to link moderation task to the insight that created it (or the other way around). I would keep the detection logic in robotoff. The moderation tool, would only be dealing with tasks, discussions and actions.

[Valimp]

I write this spec, if you see any modification, don't hesitate :

Must have

Allow users to flag issues related to products, such as problematic images or incorrect fields, directly within the platform and the mobile application. This functionality will create tickets in the moderation interface, and if a ticket for the same issue already exists, it will increase the ticket priority. Notifications will be sent to relevant parties for immediate attention.

• enable users to flag search results’ issues directly in the platform and the mobile application

• have a list of all moderation issues, fed by machine learning, heuristics or users, with a priority

• expose task lists to moderators for efficient resolution

• monitor the resolution of each issue by moderators

Requirements:

User Flagging:

• Users can flag issues related to products by clicking on a designated flag icon or button in both the web platform and mobile application.

Issue Categories:

• Define specific issue categories (e.g., inappropriate content, incorrect product details, offensive images) that users can select when flagging issues.

Issue Details:

• Allow users to provide additional details or comments when flagging an issue to help moderators understand the problem.

Ticket Creation:

• When a user flags an issue, a new ticket is created in the moderation interface. Include the following information in the ticket:
  • User information (username or ID).
  • Date and time of the report.
  • Issue category.
  • Description of the issue.
  • Product information (e.g., product name, ID, URL).
  • product link
• Close ticket button
• Delete image button

Duplicate Ticket Handling:

• Before creating a new ticket, check if a ticket for the same issue (e.g., the same product) already exists in the system. If a ticket exists, increase its priority instead of creating a new one.

Ticket Priority:

• Implement a priority system for tickets. Assign higher priority to recurring or severe issues.

Notification System:

• Notify designated moderators or administrators when a new issue is flagged. Notifications should include:
  • Issue category.
  • Product information.
  • User details.
  • A link to access the ticket in the moderation interface.

Moderation Interface Integration:

• Ensure that the moderation interface can receive, store, and display user-generated tickets with relevant details.

[raphael0202]

Here is my feedback on the spec you wrote @Valimp:

• we should store all reports (user + robotoff) in a table distinct from the ticket table.
• to identify the product on Open Food Facts, we only need the barcode (EAN). URLs are not needed, we can reconstruct it later. Adding the flavor (open food facts, open beauty facts,...) would be great to be future-proof (as we also want to support other projects in the future).
• we also need some status in the ticketing system (at least open/close).

Notify designated moderators or administrators when a new issue is flagged We have to be careful not to implement the current notification system for moderation in Slack again, I would only send a single message a day in the moderator channel if there are some important tickets (ex: images) waiting to be handled

Can you write a more detailed spec about image moderation? For inappropriate images, I think we want to display the full product information to check if there are no other fields that were modified or added by the user.

[alexgarel]

@Valimp my feedback:

• delete image button, depends on the type of issue, it's more than depending on the type of issues, we have a specific interface based on issue type, helping with quick resolutions:
  • for image --> remove image
  • for specific fields --> remove field value (or, if possible, revert to previous version (displaying it))
  • for products flagged --> remove product
• When we close an issue, it's important to have the reason, why, still being able to quickly do it. So I would have a [(close as) false positive] button for robotoff generated tickets. Also I would imagine a [requalify as quality issue] to quickly eliminate a ticket that can be time consuming but is not really a moderation issue
• it's important to have a "list view" of tickets where they can be batched process
  • and add filtering / sorting on it
• @raphael0202 said that for products we don't need a url, but for searches we need one :-)

[alexfauquette]

Just to make it clear, there are different sort of problems: vandalism or offending content, fraud and errors.

I would add a third one that we could have in mind which is coordination. If a community is interested in a specific category of products and needs to clean them, coordination is currently hard. Let's say you want to add the category muesli with nuts. For now, you can loop into the category muesli. But nothing allows you to know which one has not already been checked by someone else. The idea would be to also use this database as a "to-do list"

Ticket Priority:

Implement a priority system for tickets. Assign higher priority to recurring or severe issues.

Sound good, but I would not put a lot of effort into this strategy. If tickets are handled in a week, you might never get duplicates. I would recommend not trying to solve problem before they exist.

Allow users to provide additional details or comments when flagging an issue to help moderators understand the problem.

A feature for latter (but good to have it in mind) is to be able to respond to users. Such that they are not limited to reporting vandalism, but they can also ask questions when they don't know what to do, and get an answer. This could lead to more educated contributions, like in open street map when you're ensure you can add a note asking if something is considered as a bike lane, bike, road, or other infrastructure.

[teolemon]

• Can we transfer this to the nutri-patrol repository ?
• I haven't had time to catch up yet on the exchanges
• We forgot something: Flagged users. Case in point: a couple of users vandalizing this morning. We should be able to flag them, and to send them a "We saw you" warning email (and possibly more)

[alexgarel]

@alexfauquette: Although it is a very important topic, I think the quality coordination tool needs to be though on its own, and might be a different tool.

[alexfauquette]

Although it is a very important topic, I think the quality coordination tool needs to be though on its own, and might be a different tool.

Not sure about which point you're talking about; I assume the idea of coordination in general.

From what I understood The scheme would be the next one

Some systems (the website, the app, hunger-games, robotoff, other apps, ...) send a note to the API saying:

• Who they are
• Which product is concerned, which field/image
• What kind of problem it is (inappropriate content, fake information, ...)

A front collects that feedback according to some filtering logic:

• a page to delete/move images
• a page to clear text fields

When you say coordination needs its own tool do you mean for the back, the front, or both? For me, it would make sense to have the same backend and different fronts.

[alexgarel]

@alexfauquette oh yes, sorry, coordination is one of the main goal of the tool, indeed.

I was talking about "quality coordination".

It's a bit difference in the sense that for moderation it's unlikely you want to focus on a type of product, or brand, or etc… It's more about the importance of the problem. It's the point of priority stated above. It's not the same to have a user complaining about sugar on a product, than having a part of the body exposed on a most scanned product…

[alexfauquette]

It's the point of priority stated above

The priority as proposed is to see how many people report a product. That's an approach, but another one could be to consider the type of issue:

Inappropriate image > non food image > vandalized data > user question > quality coordination > products with old photos we should refresh

My point is that managing a list of products on a server is something that has been missing since a long time. If it could be more general than just moderation, it could help in other use cases.

[raphael0202]

Closing this issue, as we now have Nutripatrol that takes care of moderation.