openfoodfacts / openfoodfacts-python

🐍 Python package for Open Food Facts
https://openfoodfacts.github.io/openfoodfacts-python/
Other
317 stars 66 forks source link

Implement FacetContainer Class #106

Open Ansh-Sarkar opened 2 years ago

Ansh-Sarkar commented 2 years ago

With reference to Issue #69 and the proposed changes to the method of retrieval of data from facets in https://github.com/openfoodfacts/openfoodfacts-python/issues/69#issuecomment-1135774535 a brief summary of the expected changes is as follows

Quoting Proposed Changes

Even though, we can surely increase the limit on the number of records returned, it will almost certainly lead to a decrease in performance and increased waiting times. A suggested way to solve this issue would be to create a set of new functions which handle pagination. We could have 2 different types of functions : get_all_<facet_name>() and get_page_<facet_name>() The get_all_<facet_name>() function would internally call the get_page_<facet_name>() function repeatedly until all the pages have been fetched one by one. Since this data can be large we can create a FacetContainer which shall store the entire fetched data while also providing easy and efficient access to functions which can be helpful in manipulating and moving the data around.

Combined, these 2 suggestions if implemented, should be able to solve the following issues

Originally Post by @Ansh-Sarkar in https://github.com/openfoodfacts/openfoodfacts-python/issues/69#issuecomment-1135774535

Ansh-Sarkar commented 2 years ago

Have started working on the implementation of the FacetContainer Class. Will open the first PR once its completed.

alexgarel commented 2 years ago

@Ansh-Sarkar 100% in favor of this.