Sysadmin repository for the various parts of the Open Food Facts infrastructure.
We also have a specific repository regarding monitoring
As of 2023 our current priorities are:
We started logging incidents by server:
Link to Github Page
The infrastructure documentation is as follows:
Mail - servers mail setup
Free Datacenter - Data center with main production servers
Linux Server - servers general setup
Mail - servers mail setup
An introduction to ZFS - ZFS is much used in our infrastructure
Proxmox - about proxmox management
CICD - continuous integration and deployment
Observability - doc on monitoring / logs / etc.
The main services:
Some services:
Also look at all install and post-mortem reports in docs/reports
Title | State | OS | CPU # | RAM | SSD (Local) | HDD (Remote) | Services |
---|---|---|---|---|---|---|---|
Wordpress test CT [#200] | open | Debian last Stable. | 3 | [Explain if > 4 Gb.] | 6 GB. | 0 | Apache, PHP, Wordpress. |
Monitoring - VM (QEMU host for docker) [#159] | open | Debian | * 4 CPUs | * 12G for we have influxdb and elastic-search that needs memory | * 30 Go disk (it is currently around 14G, but this will grow because we want to harvest more logs and more metrics) | * 50Go for ES backups | Docker, docker-compose |
CT for new blog engine [#80] | open | Debian stable. | 3 CPU. | 2 GB. | 10 GB | -- | LAMP + wordpress. |
CT for Folksonomy Engine API dev [#76] | open | Default to Debian last Stable. | 2 | 1 GB | 12 GB. | - | PostgreSQL, Python3. |
Wild School Eco-Score project [#37] | open | Debian 10 | 4 | 16 Gb | 30 Gb | 0 | MongoDB |
slack-org [#36] | open | Debian 10 | 1 | 1 Gb | 10 Gb | None | Node.js |
adminer-org [#29] | open | Debian 10 | 2 | 512 Mb. | 4 Gb or even less. | 0 | Nginx, PHP, Adminer. |
Containers (x2) to build a replica set for OFF database [#28] | open | Debian 10 | 4 | 32 GB | 50 GB (DB = 20 GB). | 0 | Mongodb. |
feedme-org [#27] | open | Debian 10 | 3 | 3 Gb. | 15 Gb. | 0 | PostgreSQL, Node.js, Nginx. |
off-wiki-org [#21] | open | Debian 10 | 2 | 3 Gb | 14 Gb. | 14 Gb | Apache, PHP, MySQL, Mediawiki. |
VM for the Community Portal [#124] | closed | Debian last Stable. | [Explain if > 4.] | [Explain if > 4 Gb.] | [Explain if > 32 Gb.] | [Explain if > 1 Tb.] | Python/Django, probably PostgreSQL, probably Apache and all Dockerized |
VM for the Taxonomy Editor [#123] | closed | Debian last Stable. | [Explain if > 4.] | [Explain if > 4 Gb.] | [Explain if > 32 Gb.] | [Explain if > 1 Tb.] | Python, probably PostgreSQL, probably Apache for lightweight API serving from Docker |
New VM QEMU for prod docker containers [#71] | closed | Debian 11 (stable) | 8 | 24 GB | 256 GB. | - | Services deployed in production: |
monitoring [#59] | closed | Debian 11 | 4 | 32GB | 64GB | 500GB (ovh3 mount) | Docker: ElasticSearch (Kibana?, Logstash?), Grafana, InfluxDB, Prometheus, Alertmanager |
impactestimator-net [#55] | closed | Debian 11 | 1 | 1GB | 1Gb | 0 | https://github.com/openfoodfacts/impactestimator |
robotoff-ml [#53] | closed | Debian 11 | 8 | 96GB (Tensorflow, ANN) | 192GB [ML models] | 100GB | Tensorflow + ElasticSearch |
robotoff-net [#51] | closed | Debian 11 | 4 | 16GB (DB 4GB, Services 8GB) | 92GB | 0GB | Robotoff API + Schedulers + Workers, PostgreSQL DB |
mongo-dev [#45] | closed | Debian 10 | 2 | 16GB | 40GB | MongoDB running in Docker | |
off-net [#41] | closed | Debian 10 | 4 | 16GB (PO needs > 6GB) | 192GB | 0GB | ProductOpener frontend + backend, MongoDB, PostgreSQL, Memcached |
robotoff-dev [#40] | closed | Debian 10 | 4 | 8 Gb | 32 Gb | 100 Gb | robotoff, elastic search, tensorflow, postgresql |
Matomo [#24] | closed | Debian 10 | No idea. | No idea. | No idea. | No idea. | LAMP |
robotoff-org [#20] | closed | Debian 10 | 4 | 8 Gb | 32 Gb | 100 Gb | robotoff, elastic search, tensorflow, postgresql |