Immersive Node-RED experience; merge interface into FlowFuse

ZJvandeWeg commented 1 year ago

Original Issue

Description

FlowForge manages Node-RED already, though when interacting with Node-RED, it's a button to go to its editor. That removes all things FlowForge from view except for a theme.

Value added to Node-RED through FlowForge is now at least multiple clicks away. It would be great if the added value was closer to Node-RED.

The proposal for this epic is to include the Node-RED editor as tab into the sidebar.

Edited

The core objective of this Epic centers around enhancing user experience by creating a seamless integration between FlowFuse and the Node-RED Editor. Currently, users face a noticeable disconnect due to the need for navigating through multiple browser tabs or windows to switch between these two environments. This not only hampers the fluidity of the user journey but also makes the interaction with FlowFuse and its features less intuitive when operating within the Editor.

By reducing the number of clicks required to access the Editor from within FlowFuse, we aim to bring these two environments closer together, making the transition smoother and more intuitive. Conversely, by enabling easier access to FlowFuse's diverse functionalities directly from the Editor, we aim to enhance the visibility and usage of FlowFuse features, thereby enriching the overall user experience.

This Epic seeks to bridge the gap between FlowFuse and the Node-RED Editor, eliminating the UX barriers that currently exist due to the separation into different browser interfaces. The ultimate goal is to create a cohesive, integrated experience where users can effortlessly navigate between the Editor and various FlowFuse menus, thereby promoting a more engaging and efficient workflow.

### User Stories
- [ ] https://github.com/FlowFuse/flowfuse/issues/3646
- [ ] https://github.com/FlowFuse/flowfuse/issues/3655
- [ ] https://github.com/FlowFuse/flowfuse/issues/3647
- [ ] https://github.com/FlowFuse/flowfuse/issues/3648

joepavitt commented 1 year ago

Was very cautious of this when it was first proposed due to restricted real-estate, but put this together quickly (more thoughts/iterations) on the way:

The left size bar (including FF Logo) would be at the FlowForge-level. Everything to the right of that would be the embedded Node-RED editor

joepavitt commented 1 year ago

Additional consequence of this type of embedding also means floating UI options for things like Node-RED logs. Not sure yet how to differentiate between a "click this to open a floating side version" vs. "click this to navigate away from the editor and see this in full screen"

knolleary commented 1 year ago

Just need to be aware of what gets iframed from Node-RED and what is from the FF platform. The proposed screenshots have the two overlapping in a way that isn't possible.

joepavitt commented 1 year ago

The proposed screenshots have the two overlapping in a way that isn't possible.

@knolleary it is possible - see my comment here: https://github.com/flowforge/flowforge/issues/2246#issuecomment-1580253712

The FF logo would be part of FlowForge. Within Node-RED, no icon, just the text.

knolleary commented 1 year ago

Ah sorry - missed the comment below the big screenshot 😉

Something to consider - a user can still point their browser straight at the url of the instance and get the editor without the wrapping. If we change the theme to not have the logo in the top right corner, then it won't be there when accessed directly.

joepavitt commented 1 year ago

If we change the theme to not have the logo in the top right corner, then it won't be there when accessed directly.

yeah, had considered this. I'm guessing (as no reason why this would be a thing) that we can't modify theme based on url params? e.g. ?theme= as a param of the UI for NR Editor

lgrkvst commented 1 year ago

I can see the appeal when you work with FF and NR on a daily basis, but to add some counterbalance I'm not overly excited about an "Immersive Node-RED experience". I for one want to hide everything related to Ops stuff away as my audience thinks that "Instances" refers to points in time. "Devices" means smartphones and fitbits, "Snapshots" has to do with cameras, and Environment Variables are carbon emissions and smog warnings. Here, Dev and Ops are different target groups, and to a new user Node Red is challenge enough.

Moreover I don't see it making sense from a UX perspective. What I need to accomplish in FF and NR respectively are often separate tasks, at separate layers of a tech stack with separate goals and audiences. Of course there are occasions when they touch – picking up environment variables, troubleshooting palette issues or creating a custom stack that plays with Arduino's serial interface, but in those cases you'll never be able to beat two separate tabs in terms of speed, or two windows side-by-side in terms of flexibility.

The FF interface is super fast and a breeze to work with – everything is light-weight and loads instantly. In comparison Node Red is a bit on the chubby side and thrives in its own, separate tab.

MarianRaphael commented 1 year ago

@lgrkvst Thank you for your thoughtful feedback. I appreciate the nuanced perspective you bring to the table, especially considering the differences in user experience between FlowFuse (FF) and Node-RED (NR). We'll certainly take your comments into account as we continue to improve and evolve our product. The user experience is essential to us, and input like yours is invaluable for making sure we get it right.

Thanks again for taking the time

joepavitt commented 8 months ago

@Yndira-FlowForge any updates you can share on the design work here?

Yndira-E commented 8 months ago

Hi @joepavitt, not yet. I had planned to start working on it today 😉 I'll share something as soon as I have it.

Yndira-E commented 8 months ago

Building upon the proposal from @joepavitt:

I thought we could bring back the levels in the sidebar. In the overview, since there are different sections, I added a chevron and made these sub-sections an accordion, so everything is not visible at once, which can be overwhelming in such a small space. I also added a visual cue next to the expanded list, so it's easier to see the beginning and the end of it.

For devices, I'm not displaying all the info, just a basic list of devices and status. I added a link to /instance/devices for more details.

And settings would take you to /settings/general.

So, there would be more than one link to the instance overview with all the tabs. This means that a user could click on one instance on the list /team/instances and go straight to the editor.

Here's an interactive prototype. It's not perfect, and only the overview, devices, and settings are interactive. I just wanted to share it as soon as possible to get feedback and check if you think this is the right approach.

joepavitt commented 8 months ago

My concern here is the amount of screen real estate lost from the Node-RED Editor to that sidebar now, given that 98% of the work a user will do on this screen, is in the editor

Yndira-E commented 8 months ago

That sidebar would be collapsed by default and only expanded if the user clicks on one of the list items. Clicking again on the same icon collapses it back, so it will only be taking space when the user is using it.

Maybe we could hide it entirely? and leave only a small button to expand it again? Something like this:

Default:

Collapsed:

joepavitt commented 8 months ago

The challenge we have to consider is from a structural perspective too when building this out. The Node_RED Editor and top navigation bar will be inside an <iframe />, so we need a single line cut off with the designs.

We could circumvent this a little by having a relative positioned sidebar with the icons, lined up to the FF icon top-left (which would render at the FF-level). Then, on hover (or with expand button to be clicked) expand the menu options as you've shared? That expanded content would then be absolute positioned above the Editor, rather than squashing the whole editor in the x-axis?

Yndira-E commented 8 months ago

I see your point, and I think that could work. Interacting with the sidebar options shouldn't have a visual impact on the editor, correct? Users wouldn't expect clicking on something in that bar to immediately reflect a change in the editor, so it should be fine if it's covered up momentarily.

joepavitt commented 8 months ago

Interacting with the sidebar options shouldn't have a visual impact on the editor, correct?

Not necessarily, but we need to have it be useful. If I want to see Snapshots, or create a snapshot, I should have easy access to do so. Similarly for the underlying Node-RED Logs (see the floating Logs approach above).

Maybe our approach here isn't a side navigation menu, but in fact a "quick view" of each of the tabs, e.g. a small window opens on hover (or click) which gives me quick FF-based actions and information?

ZJvandeWeg commented 8 months ago

Sorry for the branch in the conversation, but if I click an option in the menu does it ask me to deploy first to persist the changes?

Ninja edit: I assume that real time editing would resolve this as it must save all changes for users.

joepavitt commented 8 months ago

but if I click an option in the menu does it ask me to deploy first to persist the changes?

I think we should toy with the idea of not navigating away at all at the moment, and offer a "small" view of those tabs so you can stay in the context of the Editor, whilst also getting FF value. If you're navigated away, then there is no value in having a nested Editor.

Ninja edit: I assume that real time editing would resolve this as it must save all changes for users.

That'll be for @knolleary to work out

MarianRaphael commented 7 months ago

For me, it is currently hard to imagine how this design, with the navigation as a sidebar, can be integrated into the current structure where the navigation bar is at the top. Switching the navigation bar's positioning feels weird to me at the moment, but maybe I am missing something.

ZJvandeWeg commented 7 months ago

You'd almost want to think about immersive the other way around. How can we make NR more FF oriented.

What a user wants to do

Create snapshots
Access the logs (STDOUT STdERR)
Theming (seems broken in demos lately)
...?

Is there a way to make FlowFuse a first class citizen?

Note: I still believe embedding NodeRED into FF is a better move as it's easier to maintain and be Node-RED version independent.

joepavitt commented 7 months ago

What a user wants to do

Create snapshots

Access the logs (STDOUT STdERR)

Theming (seems broken in demos lately)

...?

my points here exactly.

ZJvandeWeg commented 7 months ago

Sorry @joepavitt , didn't mean to pass it off as my idea. It wasn't clear to me in your linked note where the integration would live.

joepavitt commented 7 months ago

It wasn't clear to me in your linked note where the integration would live.

No problem at all. My proposal would be as per @Yndira-E's layout and idea of having the information to hand, but rather than the options being in a nested sidebar, they float in small windows above the editor, pinned to the sidebar by default, but could then be floating windows as per here.

An example of the "Snapshots" view for example:

joepavitt commented 7 months ago

Structurally, everything in red here would be the iframe, everything else would live at the FF-level.

I'd probably even change the FF logo to be an explicit "<-" back button, and keep the logo next to the "Development" text as we have it at the moment, within the NR Editor.

knolleary commented 7 months ago

Ninja edit: I assume that real time editing would resolve this as it must save all changes for users.

My current thinking (but is open to discussion once I have a design proposal in https://github.com/FlowFuse/node-red/issues/7) is that real-time editing does not mean the flows are constantly being saved in the runtime - it means other user's changes are shown in real-time within your editor. The deploy button still needs to be clicked by someone in order to save and deploy those changes. This is where the Google Docs analogy breaks down - we aren't making changes to the running flows in real-time.

As this comment is off-topic of this issue, I will post it then hide it - we can follow up in the linked issue once we have a firmer proposal in place.

Yndira-E commented 7 months ago

Considering @MarianRaphael's comment:

For me, it is currently hard to imagine how this design, with the navigation as a sidebar, can be integrated into the current structure where the navigation bar is at the top. Switching the navigation bar's positioning feels weird to me at the moment, but maybe I am missing something.

Would something like this be too much?

It would offer the same view in the same position as in the platform, and same options.
It would be above editor (like Joe suggested).
It should be scrollable and there's a "drag button" at the top of it, so it can be expanded as much as needed.
It can be hidden entirely, leaving a button to open it again at the bottom, next to the search icon:

MarianRaphael commented 7 months ago

Would something like this be too much?

I like the idea. This would also be a fairly small iteration for features like Snapshots available in the editor.@joepavitt what do you think?

joepavitt commented 7 months ago

(a) Not sure we have access to modifying that part of the NR Editor without a plugin (cc @Steve-Mcl )

(b) What does this then offer that wouldn't already be available in the NR Tools Plugin sidebar, where you can create Snapshots?

(c) I don't think that's obvious enough for a user to see, so even if we added it, I don't think users would actually utilise it?

Yndira-E commented 7 months ago

(a) Not sure we have access to modifying that part of the NR Editor without a plugin (cc @Steve-Mcl ) (c) I don't think that's obvious enough for a user to see, so even if we added it, I don't think users would actually utilise it?

@joepavitt the UX/UI isn't entirely figured out, I'm sorry I didn't mention that. I was just presenting the general idea and checking if it was welcome, didn't want to spend too much time working on it if it wasn't.

It could work in the same way as the editor's right sidebar:

It's expanded by default (maybe only the first time in our case? So the user becomes aware of its existence)
You can drag the left side of the bar to enlarge it
You can also hide it entirely

And when it's hidden:

There's a button that appears when hovering the right side of the viewport to bring it back.

joepavitt commented 7 months ago

That space is already taken up by the Node-RED "tray" that opens when you double click any nodes to edit them.

We ideally constrain this to the borders of editor.

I recommend considering UX-first, it's better practice for Product Design to focus on UX, even if block diagram layouts, then worry about the aesthetics after.

Yndira-E commented 7 months ago

I meant keeping the FlowFuse tools at the bottom (like this), was just describing how it could work, replicating the behaviour of the right sidebar but at the bottom 😉

Steve-Mcl commented 7 months ago

(a) Not sure we have access to modifying that part of the NR Editor without a plugin (cc @Steve-Mcl )

The footer is not something exposed to hook into however since we have a custom theme we could, via jQuery, select something well defined (like the search button) and append our button.

joepavitt commented 7 months ago

But even then, we'd have no way of emitting an event out of the Node-RED iframe to FlowFuse, which means that we would end up having to rebuild the FF UI inside a Node-RED plugin, which is not something I fancy.

MarianRaphael commented 7 months ago

Not sure we have access to modifying that part of the NR Editor without a plugin

Isn't it possible to implement it the other way around? Implement the editor in FlowFuse as an iframe, with the menu at the bottom, or in other words, the editor on top.

Steve-Mcl commented 7 months ago

But even then, we'd have no way of emitting an event out of the Node-RED iframe to FlowFuse, which means that we would end up having to rebuild the FF UI inside a Node-RED plugin, which is not something I fancy.

Sorry Joe, I didn't read the spec (I am not aware of what needs to talk to what) I was only commenting on the button placement.

But if we were to need intercommunication, there are ways to do that too. For example, there is the channel messaging api that could be hooked up via the js scripts in the theme and/or the sidebar plugin.

Would need a better understanding of the intended UX to comment further.

joepavitt commented 7 months ago

Isn't it possible to implement it the other way around?

You cannot communicate in/out of an iframe within a browser. So, even if we wrapped the Editor in an iframe with a custom theme that adds a a new button, there wouldn't be a way of getting that button to tell the parent FlowFuse UI to open/close any tray containing the additional navigation options.

Steve-Mcl commented 7 months ago

Isn't it possible to implement it the other way around?

You cannot communicate in/out of an iframe within a browser

See previous message. This is how I implemented the SVG editor in node-red-contrib-ui-svg

joepavitt commented 7 months ago

Meeting Notes - 19th March, 2024

Attendees:

Joe, Marian and Yndira

Notes:

Joe did initial designs, but since then general layout/framing of FF UI has changed, Yndira asked to look at it fresh
In https://github.com/FlowFuse/flowfuse/issues/2246#issuecomment-2004390079 we're proposing that this would replace the entire "Instance" UI within FlowFuse
Joe: Why are we wanting an immersive Node-RED Experience? What is the value?
- FlowFuse features more accessible in Node-RED, without interrupting the NR development workflow
- We should be justifying with numbers what the user's path is.
- If they're all going to
Joe: What features are people going to want to use whilst having the Editor open?
- Marian: Snapshots & Node-RED Logs
Agreed proposal for UX:

"Instances" > Click Instance > Load Immersive Editor as per these designs

Minor Modifications to designs:

"Back" arrow alongside the FF icon in the FF tray
Still have "actions" and "Open Editor" options top-right of tray
Just to reiterate that the new /instances/ screens would n longer have a FF header or the side navigation

Engineering Considerations:

/layouts/Platform.vue drives all of your platform pages, and forces the inclusion of a <PageHeader>, we would likely need to add a new /layout/<Editor?> type that didn't utilise the <PageHeader>
Joe's sizing: 1-2 week estimation
Access Limitations - Protected Instances, RBAC?

MarianRaphael commented 7 months ago

Screenshot 2024-03-19 at 15 50 37 Proposal Node-RED Editor simply as a new tab as first Iteration

joepavitt commented 7 months ago

Proposal Node-RED Editor simply as a new tab as first Iteration

This defeats the point of the proposal though - if it's a tab, to then add snapshots, view logs, etc. I still need to navigate away from the Editor (tab) in order to do so

ZJvandeWeg commented 7 months ago

@joepavitt What's your proposal for an iteration? I think an editor is a quick way to test core hypothesis, would love to see an alternative.

joepavitt commented 7 months ago

I think the shortest iteration here is @Yndira's Proposal:

<iframe> the Editor on a new /editor endpoint
Make the FF tabs, etc. available in a floating tray from the bottom.
Wrap Instance endpoints with a permissions check to make sure they have Editor access, otherwise redirect to existing page.

Extras not in MVP:

Drag to resize the FF contents at the bottom

It's maybe 1 days work? 2 with the permissions piece possibly.

Would need to restructure the Instance view a little into componentise the views better so we can use them in two places.

The idea of adding the Editor tab is a moot point, we'd be measuring success of the feature through interactions, of which I can't see there be any after first use because it offers no value in that format

Yndira-E commented 7 months ago

@MarianRaphael Following up on our discussion from yesterday, here's a quick and rudimentary prototype of this idea.

Here are the screens:

I noticed that we have two top bars that look quite similar, the FF one and the editor's. I'm not entirely sure what we can do about this. Would it be feasible to make it lighter, perhaps white, but only when it's embedded in our UI?
In this proposal, I've introduced the option to collapse/expand the sidebar to create more screen space for the editor.

For potential future iterations, I've drawn inspiration from the design software I use (such as Figma and Adobe products):

In Adobe, the sidebar can be detached from the window; I can even move it to a second screen if needed, this could be particularly useful when working in the editor, not so much in other circumstances.
All these platforms offer a shortcut to hide the software's UI and utilize the entire screen for design. Could this be a viable alternative to having a separate tab for the editor? With a shortcut, the FF UI would vanish, leaving the window resembling the editor's current appearance when we click on the "open editor" button.

What I don't particularly like about this approach aligns with what @joepavitt mentioned earlier:

if it's a tab, to then add snapshots, view logs, etc. I still need to navigate away from the Editor (tab) in order to do so

Maybe the next step could be to consolidate all the tabs into a tray similar to the version I proposed previously? But making our sidebar collapsable, and adding a shortcut to hide the UI as I suggest above? It's essentially a blend of both ideas.

joepavitt commented 7 months ago

But making our sidebar collapsable, and adding a shortcut to hide the UI as I suggest above? It's essentially a blend of both ideas.

Don't think we need to, having a full-screen editor, with the tray as you have it here is a great iteration as far as I'm concerned.

Yndira-E commented 7 months ago

As per @MarianRaphael's requests, I'm attaching a quick draft of how the blend of both ideas would look.

I find it kind of overwhelming, but I'm not sure if it is because I'm not used to it. WDYT?

joepavitt commented 7 months ago

I'm not 100% against the sidebar approach with icons only, but with text it's too much.

My main fear would be around people navigating away and leaving changes undeployed/unsaved with the sidebar, which would have been a flaw in the initial design I did too

MarianRaphael commented 7 months ago

Thanks, @Yndira-E , for the design. From a design perspective, the issue is complete for now.

cstns commented 6 months ago

Quick update:

We somewhat hit a bump with the embedded editor which only affected the production environment due to the editor and ff app being hosted on different domains and CSP prevent loading the editor.

The solution is somewhat of an easy fix, we need to inject CSP frame-ancestors header directives and X-Frame-Options (legacy) headers to the editor which will whitelist specific domains to embedd the editor.

This is easily accomplished whith runtime config via the httpAdminMiddleware that nr-launcher can provide and node-red can apply.

The issue that I'm having at the moment is a call being made to the '/account/authorize' endpoint with a subsequent redirect to /account/request/${requestId}/editor which apparently refuses to include the CSP headers in the reply, crashing the iframe.

joepavitt commented 6 months ago

This is easily accomplished whith runtime config via the httpAdminMiddleware that nr-launcher can provide and node-red can apply.

Does this mean we can fix this without needing to change any source in Node-RED? Or would this feature consequently only be available to anyone using the latest/greatest Node-RED?

cstns commented 6 months ago

We don't need to change any Node-RED code, the httpAdminMiddleware allows us to inject any headers we might require. I just need to get to the bottom of what/how that redirect call is made and why it doesn't include the CSP headers. I have a feeling it's not Node-RED related but not 100% sure

FlowFuse / flowfuse