openfoodfacts / openfoodfacts-server

Open Food Facts database, API server and web interface - 🐪🦋 Perl, CSS and JS coders welcome 😊 For helping in Python, see Robotoff or taxonomy-editor
GNU Affero General Public License v3.0
629 stars 366 forks source link

Store EAN-8 products in /products/000/001/234/5678 #3818

Open stephanegigandet opened 4 years ago

stephanegigandet commented 4 years ago

We currently store EAN8 products in /products/12345678/ which creates tons of directories directly under /products/

It would be best to store them like the EAN13 codes.

In addition, we have been normalizing EAN8 prefixed by 0s to EAN8 numbers (removing the 0s). But we still have many products that were added with numbers like 0000007730009 and stored in 000/000/773/0009 (like we would like to do today).

So we need to move the products in the right place, and try to merge the old products currently in 000/00[0-9]/

stephanegigandet commented 4 years ago

That's one of the reasons why we had some broken products on the front page. e.g. https://fr.openfoodfacts.org/produit/26043050/petits-pois-et-carottes-a-l-etuvee-extra-fins-fleurs-des-champs

stephanegigandet commented 4 years ago

Some rough numbers of affected products:

invalid: 7176 --> those are the products with barcodes with less than 8 digits. We could decide to pad them with 0s, and possibly mark them as invalid in one of the facets, so that we can review them. (I'm not not sure if EAN8 can start with 0)

moved: 82323 -> EAN8 for which we would change the paths

not moved: 505 -> those are tricky, they are products that we have in duplicate, as they were added with both an EAN8 and the same EAN8 padded with 0s. I think it's probably best to keep the products added with the EAN8 digits. (unfortunately there's no easy way to merge products and it would be difficult to make one).

github-actions[bot] commented 6 months ago

This issue has been open 90 days with no activity. Can you give it a little love by linking it to a parent issue, adding relevant labels and projets, creating a mockup if applicable, adding code pointers from https://github.com/openfoodfacts/openfoodfacts-server/blob/main/.github/labeler.yml, giving it a priority, editing the original issue to have a more comprehensive description… Thank you very much for your contribution to 🍊 Open Food Facts