search

FlowFuse / flowfuse

Connect, collect, transform, visualise, and interact with your Industrial Data in a single platform. Use FlowFuse to manage, scale and secure your Node-RED solutions.
https://flowfuse.com
Other
283 stars 64 forks source link

Add persistent file-system support #1779

Closed knolleary closed 6 months ago

knolleary commented 1 year ago

Description

We have built FF (in particular the k8s driver) to follow cloud native principles; not to rely on the local filesystem for storing configuring or runtime state.

We introduced the custom File nodes and the File service to follow this principle - allowing flows to read/write "files" that are stored remotely.

This approach is limited. It requires custom nodes to access the service. This means any 3rd party nodes that rely on the filesystem do not benefit and can lead to unexpected issues.

For example, UIBuilder stores a lot of runtime configuration in local files. If a project is suspended/resumed, this state is lost and cannot be regenerated from the flow file. This isn't a perfect example as UIBuilder stores configuration in those files, which means it won't get included in Snapshots, but the principle is there for flows that want to use the file system.

We need to evaluate how feasible it is to provide persistent file storage to the containers when running in k8s.

knolleary commented 1 year ago

@MarianRaphael we should keep this on the radar - I think it will become a bigger issue over time. I will include consideration of this in our HA design discussions as it has an impact there.

joepavitt commented 6 months ago

Am I right in saying there are undocumented updates with this @knolleary?

knolleary commented 6 months ago

@joepavitt I think https://github.com/FlowFuse/flowfuse/issues/3056 represents the more specific epic - one I clearly raised forgetting I had already raised this one. We don't need both.

joepavitt commented 6 months ago

In that case, will close this as now covered by #3056