Open rhofer opened 5 years ago
The problem is that we don't only need generic storage layer as django-storages is, but we rely on filesystem on which we can store working copies of the VCS repositories and files to execute external tools (eg. git, gettext). What could work here is focusing on Git only and storing the Git objects in a database and perform the operations on that or using temporary working copies. This not something we can do in forseable future. The other part is fulltext index, which will be gone in the 4.0 release, see https://github.com/WeblateOrg/weblate/issues/2825.
For user uploaded content, it might be possible to use django-storages. It would certainly work for screenshots, but with fonts it would be again tricky, because the the rendering backend supports only loading fonts from a local directory. There seems to be a solution in C to load fonts directly, but that doesn't work out of box in Python as it touches too low level parts which are not exposed in the Python bindings.
TLDR: It's probably not impossible, but would need major rewrite and it's certainly not on roadmap for now.
Summary of what we store in the DATA_DIR and possible solutions:
avatar-cache
- this does not need to persist, can be rebuilt, not all workers need itbackups
- needed on the backup worker only for doing backupscache
- fontconfig local cache, this really should live inside /app/cache in the Docker, will be fixed in https://github.com/WeblateOrg/weblate/pull/8636celery
- celery beats schedule, should be addressed by https://github.com/WeblateOrg/weblate/pull/8635fonts
- fonts used by fontconfig/pango/cairo, these need to be on the actual filesystem and kept in synchome
, ssh
- configuration for command line tools invoked by Weblate, some files can be regenerated, but some need to persist (SSH keys, borg backup keys)media
- user uploaded screenshots, this one should be ready for using other storageprojectbackups
- these should be possible to rewrite using Django storage layerstatic
- is already using /app/cache in Dockervcs
- version control data, right now Weblate supports Git, Mercurial and git-svn, so solution for both is needed, for Git https://github.com/bgahagan/git-remote-s3 might be a solutionOther affected features of storage change:
In order to separate application layer from persistancy layer in Openshift, the persistant volume clame in Openshift should not be used (files store). Instead, an object store as AWS S3 should be used, as django-storages would allow.
Question: is Weblate already supporting this today?