filecoin-station / zinnia

Runtime for worker nodes executing modules in a resource-limited sandbox
https://crates.io/crates/zinnia
Other
17 stars 2 forks source link

Storage API #192

Open bajtos opened 1 year ago

bajtos commented 1 year ago

Allow modules to persist data on the disk.

We must take into account how much space is available on the host's disk and ensure we leave some free space for other applications running in the system.

For the short term, we can start with a fixed limit on stored data, this limit should be configured from outside. (Station Desktop should use some reasonable default like 10GB, Station Core should let the operator decide how much space they are willing to offer.)

For the longer term, we should implement dynamic limits based on the actual available space at a given moment.

bajtos commented 1 year ago

If we want to enable modules for Filecoin data onboarding, we must provide at least 32GB of storage, as that's the typical data unit.

Modules should have a way to check if the host machine has enough storage to perform the module's next task.

However, we may want to look for ways to provide this information without leaking the actual free disk space or other potentially-sensitive details about the host computer.