Product addition (tracker)

[teolemon]

Who for

• Contributors

  What

#### Product addition
- [ ] https://github.com/openfoodfacts/smooth-app/issues/4155
- [ ] #1399
- [ ] (Product Addition) We should take a photo of the scanned barcode right on barcode detection, and integrate it as a first photo in product addition
- [ ] #518 
- [ ] #519 

#### More substantial addition
- [ ] #2910
- [ ] #515
- [ ] #3298
- [ ] #1701
- [ ] #3297
- [ ] #3295
- [ ] #3279
- [ ] https://github.com/openfoodfacts/smooth-app/issues/517
- [ ] #511
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3963
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3985
- [ ] https://github.com/openfoodfacts/smooth-app/issues/3986
- [ ] https://github.com/openfoodfacts/smooth-app/issues/4068

#### Done
- [x] #1398
- [x] #2978
- [x] #2975
- [x] #2974
- [x] #2977
- [x] #2944
- [x] #1397
- [x] #1395
- [x] #1194
- [x] #1400
- [x] #1393
- [x] #1396

Video

https://photos.app.goo.gl/V1cwkaVEM9ArwJBL7

Part of

• 503

[teolemon]

We still have an issue on the road to scores, and a wizard product addition is needed. Thus all the open bugs @monsieurtanuki and this proposal for a wizard revamp: https://github.com/openfoodfacts/smooth-app/discussions/3270

[monsieurtanuki]

@teolemon To me, the thing would be to openly hack the nutriscore/ecoscore and focus on it.

The current "new product page" is a boring list of upload image buttons that provide no instant feedback to the user.

Assuming that the nutriscore is about category, ingredients and nutritional facts only, I guess it would be more fun (🥳 ) to show something like "Let's compute the nutriscore in 1-2-3 steps!" with 3 buttons. Then, more advanced users may weight individually clean and dry parts of the packaging, but it would make sense to prevent less advanced users from giving 1 star review.

[teolemon]

[monsieurtanuki]

Was it a barcode search? A scan search with wrong barcode detection? A text search with the only 2 Romanian results for "snickers"?

[teolemon]

• It could either be a scan with a product with just "Snickers" or a product that does not exist at all.
• It could be a search that timed out due to the current infrastructure upgrade.

I'm very much aligned with the Nutri-Score and Eco-Score pipelines. We have those old issues

• 518

• 519

• that should be mixed with #3270

[teolemon]

@monsieurtanuki a quick and dirty fix would be one button per pipeline in the addition screen (plus two concatenations of editing sections)

[monsieurtanuki]

@teolemon The pipelines (actions behind "help!" buttons) do not exist yet, right?

I would even put the "help!" buttons on top, as it's probably what users want first. I mean, 1 star users may want an answer quickly, rather than contribute with pictures that they already see (no added value for them) (plus the gamification: as a user I see that my first inputs already have an impact).

[teolemon]

• They don't. I just had a look at the mockups in Figma, but they are too ambitious.
• Stacking the necessary existing editing sections on top of each other should be a good first.

[monsieurtanuki]

@teolemon Something like that?	top	bottom

[teolemon]

Looks reasonable. Perhaps being honest ? "Help compute the Nutri-Score in 2 minutes."

Bugs to open

• [ ] We should reword to "Ajouter la catégorie du produit"
• [x] #3951

[teolemon]

• Probably a beautification of the title (larger, using the same title as the Knowledge Panel titles Health and Environnment)
• using those (probably already cached assets): https://static.openfoodfacts.org/images/attributes/nutriscore-unknown.svg https://static.openfoodfacts.org/images/attributes/ecoscore-unknown.svg

[monsieurtanuki]

@teolemon Something like that?

Regarding a short "add THE product category", it means that we'll have to maintain 3 labels:

• add the product category (new product / empty category)
• Add missing product category (when the category is not good enough, cf. KP, cf. "skyr" that is not a precise enough category) (?)
• category entered! (or we just remove the button)

Same for nutrition facts and ingredients, right?

[teolemon]

yes, much better 👍

[teolemon]

and the grey nutri-score is replaced by the actual one, as soon as we have it

[raphael0202]

After discussing the issue with @teolemon, here is my opinion on implementing a new product addition pipeline. First, from an AI perspective, we need to ask the user to upload images first, as there image upload takes time (especially if the user network is slow) and as prediction extraction by Robotoff also takes time.

The idea is to assist the user with information predicted by Robotoff during the product addition process.

I would suggest to set up the following simple pipeline with the following screens:

1. asking for most important images: front, nutrition, ingredient (in that order) + crops. I would drop the "packaging" and "additional photos" at this stage.
2. asking for product name + weight. If we have product_weight insights from Robotoff at this stage, we pre-fill the field.
3. asking to fill the nutritional information. At this step all Robotoff processing should be finished. We can ask Robotoff with the /predict/nutrition endpoint nutrition information and pre-fill the nutrient fields.
4. asking user to extract ingredients. Currently, Robotoff cannot do it automatically, but there is an ongoing AI project to do this.
5. asking user to provide the category. We should have a pretty good category prediction at this point with all the previously provided data (product name, nutrients, ingredients, images). We should provide Robotoff category insight as a category suggestion.
6. (optional for a first pipeline version) Asking user to add labels, using Robotoff suggestions.
7. (optional for a first pipeline version) Asking user to take packaging + additional pictures.

We should send data points as soon as the user submit the data at each pipeline stage, to be sure we get partial data in case the user drops the product addition.

I don't think it's valuable to talk about ecoscore or nutriscore at the beginning of the pipeline, as the user doesn't necessarily know what these scores are about. I think the user expect to have to provide some data to get information about the product, he/she just needs to be guided.

[monsieurtanuki]

I don't think it's valuable to talk about ecoscore or nutriscore at the beginning of the pipeline, as the user doesn't necessarily know what these scores are about. I think the user expect to have to provide some data to get information about the product, he/she just needs to be guided.

@raphael0202 I have no idea what users expect, but it would be fair to assume that at some point nutriscore and ecoscore mean something for them in the app, as:

• they are displayed on top of each product card
• there are only 3 attributes deemed important enough to be part of the default settings and they're among them (nutriscore: very_important, ecoscore: important, nova: important)

We could dismiss the nutriscore / ecoscore input in the "add new product" page for users that say they don't care about those attributes.

That said, my point here was to code a quick fix (slightly better looking, more interactive) by adding a couple of buttons. Your suggestion is much more ambitious, and deserves a proper issue (if not several issues). And keep in mind that the images are uploaded in the background, which means some glitches in the pipeline.

[raphael0202]

@monsieurtanuki My point about the ecoscore/nutriscore wasn't very clear, sorry for that. I was taking the perspective of a user in a country with low product coverage (such as Romania, where we're gaining some traction lately), where the user has a high chance of scanning a product not referenced by Open Food Facts. As such, his/her first "interaction" with nutriscore/ecoscore would be in the product addition page, and I find this a bit confusing. I think it's preferable that users learn more about what is the ecoscore and the nutriscore on the product page instead.

I think it's a good time to implement a first version of the pipeline, as it would bring more value than quick fixes (and it's something that we have been postponing for a long time, we already talked about this for the old android app).

It's not an issue if images are not uploaded on time, as ML predictions only come as a bonus, it should be possible to complete the pipeline without it.

This can be implemented step by step, we don't need to implement the full pipeline now. Just adding step (2) (asking for product name + weight + Robotoff weight prediction) could be a good start.

I'm also curious about your feedback on this proposal @teolemon @g123k

[teolemon]

Composite pipelines

• For existing products, we will need those composite Nutri-Score and Eco-Score completion pipelines in the product page anyway (we currently have some mechanism in place thanks to the Knowledge Panel actions, but they are not linear, especially for Eco-Score)

  We need to wire existing editing components to Robotoff (including for existing products)

• We need to fix #3326 (something was merged but does not work) so that we can validate robotoff inline and proceed to the easy steps

  Easy

  • [ ] We can add Robotoff questions filtered on category in the category editing section
  • [ ] We can add Robotoff questions filtered on labels in the labels editing section
  • [ ] We can add Robotoff questions filtered on weight in the basic details section (#3279)

Harder

• [ ] We can wire the Robotoff nutrition facts prediction to the nutrition table editing section (remains to be seen how we integrate with the nutrition table itself, but we can do that in 2 steps possibly) (#3278)

Better Eco-Score and Nutri-Score onboarding

• We definitely need to add a good Eco-Score and Nutri-Score one-time explainer in app, beyond what we have for knowledge panels. It would require a reflexion: is a video good (like the official ones) ? Can we have a vertical video (none of them exist to my knowledge) to appear more native ? Should we have something animated by code instead (difficult) ? Is text and image enough, and will people actually engage with that little?

• 3982

• 3983

Be tougher on incomplete products / Towards personal pipelines

• Show a drive to complete data on the scan card when there's close to zero data on the product. Eg deny access to the product, unless there's significant data.
• https://github.com/openfoodfacts/smooth-app/issues/1080
• https://github.com/openfoodfacts/smooth-app/issues/1118