raphael0202
commented
1 year ago Full Slack thread discussing this issue: https://openfoodfacts.slack.com/archives/C1FPYCWM7/p1683272552984549
Here is a summary of the discussion so far: We decided to purchase a new server that will be hosted by Free. This server will be powerful enough to host Robotoff and the new search project. We can add 1 or 2 GPUs to the server to speed up ML inference. With the search project grant, we have ~10k of budget for infrastructure, so we can use it to buy a high-end server (GPU will be bought with a different budget).
Requirements:
- a recent and powerful CPU
- enough RAM for Robotoff + elasticsearch
- NVMe disks, with a few TB of storage to be able to use this server as backup too
We don't necessarily need to use proxmod + QEMU, we can use baremetal here for additional performance gain. Besides, only PostgreSQL database is important for Robotoff, the rest of the data can be generated easily.
T4 GPU seems promising (better performance than P4), and with 1/4th the consumption of V100 GPUs (that are also more expensive). However, T4 GPUs only have 16 GB of RAM, I will conduct benchmarks to see if this is an issue (as we serve multiple ML models). Having 2 T4 GPUs instead of a single one is also an option.
alexgarel
We're currently lacking space storage on OVH2, due to Robotoff. We store in PostgreSQL all logo embeddings (7M currently, but many are missing) + all image embeddings (7M images, but only ~1M are in DB now). Elasticsearch indexes for approximate nearest neighbors (ANN) also take up a lot of storage disk.
We would also like to increase the number of workers on Robotoff to make processing faster (workers are responsible for performing actions after product updates or image upload), but we cannot as increasing the number of workers would mean having a high server load.
It is expected that storage, CPU and memory needs are going to increase for Robotoff, due to the introduction of new models and detections. However, we don't have much option on OVH2: we cannot increase the number of CPU cores (?). We also cannot increase storage, as we zfs-sync OVH2 to OVH3 and as OVH1 doesn't have a lot of remaining space.
I suggest we move Robotoff to a distinct server, either rented or hosted at OVH datacenter.