Add a button to take a picture of the recycling instructions

[teolemon]

Overview

Add a button on the product page to take a picture of the recycling instructions and upload it to the server.

• This can be achieved by cloning the "Take another picture" code. (Most simple implementation)
• This can be achieved by cloning the "Take ingredients picture" code
• This is very important for the Environnemental Scoring project.

Feature visible in the Web version

On Android

You can see a similar button on the ingredient tab here: https://github.com/openfoodfacts/openfoodfacts-androidapp/blob/c30f75f70fd20c84a1d3d3cf803e9b120b93bab6/app/src/main/res/layout/fragment_ingredients_product.xml#L49 We would insert it in the Environnement tab here: https://github.com/openfoodfacts/openfoodfacts-androidapp/blob/c30f75f70fd20c84a1d3d3cf803e9b120b93bab6/app/src/main/res/layout/fragment_environment_product.xml and as a result, make the Environnement tab always visible so that people can help with taking photos.

Flow

It works exactly like the ingredients field, tab and addition workflow. This means although I've redocumented everything, you can mostly clone the ingredients code, or refactor it to be able to handle the packaging case

1. Upload a packaging_image API doc to upload an image: https://documenter.getpostman.com/view/8470508/SVtN3Wzy?version=latest#6159dfb9-dd05-4ef8-8bf0-2c2ea7585760 Same process as ingredients but field is: packaging_fr , packaging_en …
2. Request OCR output Request: cgi/packaging.pl?code=3454681006511&id=packaging_fr&process_image=1&ocr_engine=google_cloud_vision Response: {"packaging_text_from_image":"tia ac ","packaging_text_from_image_orig":null,"status":0} API to perform OCR on it: https://documenter.getpostman.com/view/8470508/SVtN3Wzy?version=latest#c911e8c8-2b88-44cd-8512-52fa55131396
3. Get the user to validate it
4. Send the validated result to packaging_text Same process as ingredients but field is: packaging_text_fr , packaging_text_en …

Server-side issue: openfoodfacts/openfoodfacts-server#4286

[charchitmangal]

I would Like to Work on this Issue

[teolemon]

@charchithere I've summarized everything on top. Are you still interested by the issue ?

[mohsin]

Hi @teolemon, I would like to take this up. May I?

[teolemon]

Yes you may @SaifurRahmanMohsin Another elegant way to implement this would be to tackle: https://github.com/openfoodfacts/openfoodfacts-androidapp/issues/3543 It wouldn't bring the OCR part, but would enable getting the photos at large scale (and would be more generic)

[mohsin]

@teolemon Was it verified that the API returns the packaging URL when field ID packaging_en is sent? I even tried sending image_packaging_url_en but even that didn't return the packaging image URL. Here was the request:

https://ssl-api.openfoodfacts.org/api/v0/product/8901063017252.json?fields=origins,other_information,link,packaging,packaging_en,packaging_url,ingredients_that_may_be_from_palm_oil_tags,product_name_en,image_nutrition_url,image_packaging_url,generic_name_en,traces,vitamins_tags,states_tags,nutrition_data_per,ingredients_analysis_tags,purchase_places,ingredients,recycling_instructions_to_recycle,product_name,ingredients_text,warning_en,generic_name,customer_service,ingredients_from_palm_oil_tags,image_front_url_en,ingredients_from_or_that_may_be_from_palm_oil_n,amino_acids_tags,countries,conservation_conditions_en,lang,customer_service_en,recycling_instructions_to_discard_en,stores,allergens,languages_codes,nova_groups,brands,conservation_conditions,serving_size,traces_tags,recycling_instructions_to_discard,last_modified_by,code,quantity,no_nutrition_data,image_small_url,ingredients_text_en,creator,additives_tags,manufacturing_places,other,other_nutritional_substances_tags,image_nutrition_url_en,image_packaging_url_en,nutrition_grades_tags,recycling_instructions_to_recycle_en,cities_tags,nutriments,url,selected_images,editors_tags,allergens_hierarchy,minerals_tags,brands_tags,last_modified_t,image_front_url,labels_hierarchy,emb_codes_tags,warning,allergens_tags,image_ingredients_url_en,environment_impact_level_tags,nutrient_levels,image_url,created_t,environment_infocard,nutrition_grade_fr,ingredients_from_palm_oil_n,other_information_en,countries_tags,image_ingredients_url,categories_tags,labels_tags

And the response:

{"status_verbose":"product found","status":1,"product":{"other_nutritional_substances_tags":[],"countries":"India","serving_size":"10g","countries_tags":["en:india"],"ingredients":[],"generic_name_en":"","link":"","allergens_tags":[],"last_modified_by":"mohsin","no_nutrition_data":"","allergens":"","stores":"","product_name_en":"50-50","nutriments":{"energy_100g":5021,"sugars":10,"energy-kcal_value":1200,"energy_serving":502,"carbohydrates_unit":"g","sugars_unit":"g","proteins_unit":"g","proteins_serving":0.7,"saturated-fat_unit":"g","fat_serving":2.6,"saturated-fat_100g":13,"energy_value":1200,"fat_unit":"g","energy-kcal_100g":1200,"fat_100g":26,"sugars_serving":1,"proteins_100g":7,"saturated-fat":13,"sugars_value":10,"energy_unit":"kcal","energy-kcal_serving":120,"fat_value":26,"energy-kcal_unit":"kcal","carbohydrates_100g":60,"carbohydrates_serving":6,"carbohydrates_value":60,"sugars_100g":10,"saturated-fat_serving":1.3,"carbohydrates":60,"proteins_value":7,"energy":5021,"fat":26,"saturated-fat_value":13,"energy-kcal":1200,"proteins":7},"labels_tags":["en:green-dot"],"image_small_url":"https://static.openfoodfacts.org/images/products/890/106/301/7252/front_en.18.200.jpg","image_ingredients_url":"https://static.openfoodfacts.org/images/products/890/106/301/7252/ingredients_en.24.400.jpg","additives_tags":[],"ingredients_that_may_be_from_palm_oil_tags":[],"nutrition_data_per":"100g","traces_tags":[],"nutrition_grades_tags":["not-applicable"],"code":"8901063017252","traces":"","purchase_places":"","product_name":"50-50","vitamins_tags":[],"labels_hierarchy":["en:green-dot"],"lang":"en","image_front_url":"https://static.openfoodfacts.org/images/products/890/106/301/7252/front_en.18.400.jpg","states_tags":["en:to-be-completed","en:nutrition-facts-completed","en:ingredients-to-be-completed","en:expiration-date-to-be-completed","en:packaging-code-to-be-completed","en:characteristics-to-be-completed","en:categories-to-be-completed","en:brands-completed","en:packaging-completed","en:quantity-completed","en:product-name-completed","en:photos-validated","en:packaging-photo-selected","en:nutrition-photo-selected","en:ingredients-photo-selected","en:front-photo-selected","en:photos-uploaded"],"amino_acids_tags":[],"cities_tags":[],"allergens_hierarchy":[],"created_t":1593879923,"editors_tags":["mohsin","vinittiwari32-gmail-com","openfoodfacts-contributors","halal-app-chakib"],"image_nutrition_url":"https://static.openfoodfacts.org/images/products/890/106/301/7252/nutrition_en.21.400.jpg","emb_codes_tags":[],"ingredients_text_en":"","brands_tags":["britannia"],"image_url":"https://static.openfoodfacts.org/images/products/890/106/301/7252/front_en.18.400.jpg","manufacturing_places":"","brands":"Britannia","quantity":"50g","nutrient_levels":{},"last_modified_t":1603705899,"categories_tags":[],"languages_codes":{"en":4},"origins":"","minerals_tags":[],"selected_images":{"front":{"small":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/front_en.18.200.jpg"},"thumb":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/front_en.18.100.jpg"},"display":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/front_en.18.400.jpg"}},"nutrition":{"display":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/nutrition_en.21.400.jpg"},"small":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/nutrition_en.21.200.jpg"},"thumb":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/nutrition_en.21.100.jpg"}},"ingredients":{"display":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/ingredients_en.24.400.jpg"},"thumb":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/ingredients_en.24.100.jpg"},"small":{"en":"https://static.openfoodfacts.org/images/products/890/106/301/7252/ingredients_en.24.200.jpg"}}},"ingredients_from_palm_oil_tags":[],"creator":"halal-app-chakib"},"code":"8901063017252"}

I updated the app code to upload the packaging URL and that worked and the image I uploaded does appear on the website under the packaging section with this link.

[mohsin]

Hey @teolemon could you kindly verify and get back to me on this? I'm ready with some of the changes and am waiting only because of this blocker.

[teolemon]

good catch @stephanegigandet ? https://world.openfoodfacts.org/cgi/product_image_crop.pl?code=8901063017252&id=packaging_en&imgid=14&x1=0&y1=0&x2=0&y2=0&coordinates_image_size=full&angle=0&normalize=false&white_magic=false {"image":{"display_url":"packaging_en.41.400.jpg"},"status":"status ok","imagefield":"packaging_en"}

[mohsin]

@teolemon Ok I'm glad that the image is still accessible via image crop API, but should I use this API node instead for this? The product/<product-barcode-id> API is already called when the product item is opened and there is an image_packaging_url field in the Product class to which gets filled when this product load happens. I think it makes better sense to fix the product/<product-barcode-id> to include this field than making an additional API request in the Environment tab, right?

[teolemon]

Closing, since the initial implem is done