FlowFuse / flowfuse

Connect, collect, transform, visualise, and interact with your Industrial Data in a single platform. Use FlowFuse to manage, scale and secure your Node-RED solutions.
https://flowfuse.com
Other
283 stars 64 forks source link

Add "Import Snapshot" Endpoint to the FlowFuse API #3458

Open anthonyjsargeant opened 9 months ago

anthonyjsargeant commented 9 months ago

Description

In order to be able to deploy an application across VPC clusters in Kubernetes, as a Developer, I would like an endpoint in the FlowFuse API that would allow me to import a previously exported application snapshot.

We’ve been working with FlowFuse for the last few weeks now, as part of an investigation into how it, and Node-RED could be used in our organisation.

We like the speed at which a developer can create an application using FlowFuse and Node-RED so naturally, we were interested in using the DevOps pipelines feature. However, due to restrictions placed upon us for security reasons, our non-prod and prod Kubernetes clusters are in differing VPCs and have no network connectivity between them.

This makes deploying Node-RED applications via a FlowFuse DevOps pipeline impossible as we’re not able to assign a production instance to an environment as it would exist outside of the k8s cluster that FlowFuse is running on.

A potential solution would be to use a third-party tool that can export a snapshot from an application in one instance of FlowFuse and then import it into application of another instance. For example, for our traditional use case, we employ Octopus Deploy to create deployment pipelines for artefacts into our non-prod and prod environments.

We have noticed that the FlowFuse API has an endpoint for exporting application snapshots, however, we cannot use this approach as currently there isn’t an endpoint for importing snapshots.

As I am not a member of FlowFuse, I'm not able to suggest an estimate, nor give any indication as to which customers this feature would be available to.

Which customers would this be available to

Other (See comments)

Have you provided an initial effort estimate for this issue?

I am not a FlowFuse team member

knolleary commented 9 months ago

This was discussed in Slack with @anthonyjsargeant

Our API does provide the endpoints to achieve this in two steps:

  1. https://app.flowfuse.com/api/static/index.html#/Snapshots/post_api_v1_projects__instanceId__snapshots_ - allows you to import a snapshot to an instance
  2. https://app.flowfuse.com/api/static/index.html#/Instance%20Actions/post_api_v1_projects__instanceId__actions_rollback - allows you to tell a instance to rollback to a particular snapshot

So the process would be:

  1. Import the snapshot and get back its id
  2. Tell the instance to rollback to that snapshop

@anthonyjsargeant was going to try this out and feedback if it meets his requirements.

robmarcer commented 9 months ago

This is for https://app-eu1.hubspot.com/contacts/26586079/record/0-2/7761278700

robmarcer commented 9 months ago

@hardillb this is what we are discussing in the meeting on Thursday.

From my discussion with the customer, they will be running two totally separate instances of FF and they need to be able to migrate NR instances from one to the other. Maybe I'm misunderstanding but @knolleary's solution sounds like it would work within a single FF instance rather than two.

hardillb commented 9 months ago

Nick's solution will work between 2 instances, it will just need 2 API keys

hardillb commented 9 months ago

Just to add, the snapshot from the first FF instance should be available with this API endpoint

https://app.flowfuse.com/api/static/index.html#/Snapshots/get_api_v1_projects__instanceId__snapshots__snapshotId_