Provide a remote worker for scan job

[kkovaletp]

Is your feature request related to a problem? Please describe. Photoview can be deployed on a low-performance server, as most of the time it doesn't need a lot of resources to run. However, scan job needs more CPU and preferable GPU acceleration to process source media. This job might run 'forever' on such low-performance servers.

Describe the solution you'd like I'd like to have a separate deployable part, which I can deploy on another system in the network that has more resources to run the scan job. Then Photoview server could use the results of the remote job run in the same way as if it is run locally. It should be possible to deploy as a Docker image or installable on the host.

Additional context This issue is to collect more opinions and ideas for this topic

[googollee]

The first thing in my mind is, how to transfer files between photoview and remote works. Here are some options:

• Remote workers provide APIs to upload/download files.
  • It makes a worker complex.
  • It requires a queue in the worker.
• NFS/Samba shares files across hosts and workers.
  • The file path must be the same between hosts and workers.
  • Hard to debug if file paths don't match, especially for users with low technical skills.

Any other ideas?

[kkovaletp]

The other option for files handling is to integrate 3rd-party file storage services (could be self-hosted, like OwnCloud/NextCloud, or cloud - a lot of *drive, Dropbox,...), Cloud providers (AWS, Azure, GCP), etc. + is relatively easy to integrate, as there are libs and frameworks, as well as a lot of ready implementations in other open-source projects + in the case of Cloud providers integration, Photoview will be 1 step closer to being ready for hosting in the cloud (however, this is currently not in our roadmap: nice benefit, but not a goal or significant advantage to consider when making the decision) - is the file transfer speed limit:

• for self-hosted services, it shouldn't be so critical, however, it will still be lower than NFS/Samba
• for Cloud services transfer will be to/from the internet to the nearest jump-server of the provider's CDN

[kkovaletp]

There is also the shared volume Docker compose architecture, which allows mapping the same physical folder to more than 1 compose services (containers). But it doesn't work for distributed over different hosts deployment: Docker Swarm or Kubernetes is needed. Also, it doesn't solve the case when either Photoview or a worker (or both) is deployed directly on the host