finos / FDC3

An open standard for the financial desktop.
https://fdc3.finos.org
Other
192 stars 113 forks source link

FDC3 For Web browsers #896

Open robmoffat opened 1 year ago

robmoffat commented 1 year ago

Enhancement Request

In a recent email on the FDC3 mailing list, @kriswest wrote:

... I also want to add that there is clearly significant interest in the community in enabling FDC3 use on the web. There is a strong use case in that it would enable better onboarding journeys with less drop-off (where you use an app on the web with others before adopting a desktop container or similar).

and:

But there are also additional challenges such as how to make the API available reliably without importing a proprietary module from a particular vendor into every app, how to deal with more than one implementation of API/Desktop Agent in the browser at once, how to do this reliably and securely within the browser sandbox etc.. Work needs to be done in the Standard to solve these issues and to make web browser use possible in a future FDC3 Standard version - which I believe is possible (and likely to involve using a vendor-agnostic FDC3 NPM module to detect and connect to API implementation(s)). However, we're going to need to do that work to enable the aforementioned API implementations to be compliant and if we fail to hold the line now on compliance with the current version of the FDC3 Standard, that may never happen.

Hence, I strongly encourage any vendors or firms working on API implementations that would fall outside current compliance requirements to work with the FDC3 maintainers and community, in general, to help move the Standard forward! The first step will be to raise an issue to bring to the Standards Working Group, which might then move to a Discussion Group to develop a proposal for additions to the Standard and its software. The maintainers meet again next week and will discuss getting such an issue on the books.

This is therefore a placeholder issue where we can begin to discuss how to solve this problem and potentially open up FDC3 to a wider audience.

robmoffat commented 1 year ago

This is a problem with some prior art:

As @kriswest says in the quote above, he thinks there is interest within the community. I have personally talked to a number of App vendors who are confused as to why this isn't a thing already.

I am hoping we can have a discussion on this at the next maintainers' meeting and move this forwards

robmoffat commented 1 year ago

Here is my thinking so far on the subject (more to seed a debate rather than anything else).

  1. Glue Core uses PostMessage to allow FDC3 to communicate within-browser but across tabs. This is a promising line of inquiry and I'm hoping we can learn more from that team about how this works. There are security concerns here that I would like us to think through.

  2. In discussing this with @kriswest yesterday, we agreed that it would be nice to rally around a well-written Use Case. Perhaps this is where we should start?

  3. A further point we discussed is the need for what I am calling an FDC3 "wire protocol":

    • FDC3 Sail implements a bus for FDC3 messages between apps, with the agent acting as router (and converting between FDC3 versions as needed).
    • Desktop Bridging is also essentially the same thing - providing a sockets-based messaging protocol between different desktop agents.
    • At some level, Connectifi must be implementing a bus also.
novavi commented 1 year ago

Hi @robmoffat thanks for raising this issue.

Related this this, on the point @kriswest raised (which you cited) about the possibility of using a 'vendor-agnostic FDC3 NPM module to detect and connect to API implementation(s)', I have something to share.

I have implemented this fdc3-installer library:

https://github.com/novavi/fdc3-installer

over the last few weeks. It aims to provide a solution to the problem in browser-based environments of tight coupling between FDC3-enabled apps and FDC3 Desktop Agents, by providing a mechanism to dynamically discover and install an agent within an app at runtime. This approach allows an app to run both in a desktop container (using the container's built-in agent) and also in a browser-based micro-frontend container (using a dynamically-installed agent) with zero code changes to the app. The library supports multiple strategies for the areas of agent discovery, creation and bootstrapping - with the ability to easily add and try out additional strategies based on any subsequent discussion / ideas.

The library does not claim to have all the answers by any means, but demonstrates some of the options that could potentially be used for a solution in this area. I hope this will help in the wider discussion that needs to be had within the FDC3 community about how we can best overcome the tight-coupling problem for browser-based FDC3 Desktop Agent implementations - and help move us towards standardized approach in the future.

I was originally aiming to publish this library more like sometime next month rather than now but after seeing that you'd raised the issue, I brought this forward in the interest of the discussion and just published what I have now. Because of this, I haven't actually even proof-read the stream-of-consciousness that I turned into a README yet, so parts of the description might still be a bit rough round the edges.

kriswest commented 1 year ago

@novavi many thanks for sharing your work on this issue.

While I suspect/hope we'll end up adopting a subset of what you propose, it nevertheless represents a tour-de-force in exploring the options and related issues. I look forward to reviewing (and hunting for the strategies I intend(ed) to propose within your implemented set).

K

On Mon, 23 Jan 2023, 10:08 Derek Novavi, @.***> wrote:

Hi @robmoffat https://github.com/robmoffat thanks for raising this issue.

Related this this, on the point @kriswest https://github.com/kriswest raised (which you cited) about the possibility of using a 'vendor-agnostic FDC3 NPM module to detect and connect to API implementation(s)', I have something to share.

I have implemented this fdc3-installer library:

https://github.com/novavi/fdc3-installer

over the last few weeks. It aims to provide a solution to the problem in browser-based environments of tight coupling between FDC3-enabled apps and FDC3 Desktop Agents, by providing a mechanism to dynamically discover and install an agent within an app at runtime. This approach allows an app to run both in a desktop container (using the container's built-in agent) and also in a browser-based micro-frontend container (using a dynamically-installed agent) with zero code changes to the app. The library supports multiple strategies for the areas of agent discovery, creation and bootstrapping - with the ability to easily add and try out additional strategies based on any subsequent discussion / ideas.

The library does not claim to have all the answers by any means, but demonstrates some of the options that could potentially be used for a solution in this area. I hope this will help in the wider discussion that needs to be had within the FDC3 community about how we can best overcome the tight-coupling problem for browser-based FDC3 Desktop Agent implementations - and help move us towards standardized approach in the future.

I was originally aiming to publish this library more like sometime next month rather than now but after seeing that you'd raised the issue, I brought this forward in the interest of the discussion and just published what I have now. Because of this, I haven't actually even proof-read the stream-of-consciousness that I turned into a README yet, so parts of the description might still be a bit rough round the edges.

— Reply to this email directly, view it on GitHub https://github.com/finos/FDC3/issues/896#issuecomment-1400091717, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM7PBGQKMDQ7S3S72MXK53WTZKALANCNFSM6AAAAAAUAFXPPQ . You are receiving this because you were mentioned.Message ID: @.***>

novavi commented 1 year ago

@kriswest no probs.

Yes tbh I'd be very much in favour of narrowing down to a subset of the options if this type of approach was to be standardized. The strategies you intend(ed) to propose might not even be amongst the ones I came up with actually! I'm sort of hoping that there might be more options on the table to choose from when standardizing this.

The main constraint I applied when listing out and implementing those discovery strategies was that the container-based ones must work on a cross-origin basis. I think it's sensible to make any standardized solution work for the more restrictive cross-origin scenario, and then it will automatically work for the (simpler) same-origin scenario. From my experience of large organisations who hope to provide a more unified customer experience for their existing brownfield apps, those apps are typically built by separate teams on separate tech stacks in separate repos with separate release schedules and deployed on separate subdomains. That is to say, if an organisation has apps that are commonly on the same subdomain as their container (and as each other) then they are quite probably heavily down the road towards a module federation approach, and may have less requirement for FDC3 interop in their apps in any case (except perhaps for third-party app integration).

There's more freedom on the app-based discovery strategies of course, because you don't have to worry about the cross-origin issue e.g. even reading from a previously-defined global variable would be possible. I did avoid a few other things like reading from meta tags e.g. that requires SSR of the host page in order to prevent it just being a pointless exercise in moving hard-coding from one place to another. That said, the general risk with app-based discovery strategies (as opposed to container-based ones) is that it could be quite easy to misuse them if one did not fully understand their purpose (e.g. again using an installer, but simply moving hard-coding from the app's code into the app's config).

In terms of the range of strategies supported, the whole concept certainly becomes less usable by casual FDC3 app devs and less supportable as a library if it aims too wide (indeed, supporting corner cases sometimes encourages requests to support further corner cases). Laying out the options in this library was partly intended as a mechanism to help test them and discuss them - because without a concrete implementation it can sometimes be hard to see the relative merits and pitfalls of different approaches.

Having said that, it's worth pointing out that we should really distinguish between a generic installer library that can load pretty much any arbitrary browser-based FDC3 Desktop Agent today (which is sort of what I just about implemented) and a future standardized installer that only needs to load a future FDC3 Desktop Agent adhering to a future spec version (2.1 or above?) which is clear about how agents are to be provided. For example, if a future version of the FDC3 spec mandated that an agent must be provided by a factory function and exported via an ES Module with a specific name, then one should be able to easily dispense with the different creation strategies - although one would still need to offer support to pass args to that function in a config of some sort (whether to provide a WebSockets url for a cloud-based approach, or simply just more mundane options applying to all browser-based implementations e.g. whether fdc3.open() should open apps in tabs / windows / iframes). The issue of bootstrapping is more involved than that of creation because as I alluded to in the README for this repo there is (currently at least) a relationship between agent bootstrapping and window.fdc3 installation in terms of the overall agent lifecycle... which is a subject for another email thread I think!

On Mon, Jan 23, 2023 at 12:42 PM Kris West @.***> wrote:

@novavi many thanks for sharing your work on this issue.

While I suspect/hope we'll end up adopting a subset of what you propose, it nevertheless represents a tour-de-force in exploring the options and related issues. I look forward to reviewing (and hunting for the strategies I intend(ed) to propose within your implemented set).

K

On Mon, 23 Jan 2023, 10:08 Derek Novavi, @.***> wrote:

Hi @robmoffat https://github.com/robmoffat thanks for raising this issue.

Related this this, on the point @kriswest https://github.com/kriswest raised (which you cited) about the possibility of using a 'vendor-agnostic FDC3 NPM module to detect and connect to API implementation(s)', I have something to share.

I have implemented this fdc3-installer library:

https://github.com/novavi/fdc3-installer

over the last few weeks. It aims to provide a solution to the problem in browser-based environments of tight coupling between FDC3-enabled apps and FDC3 Desktop Agents, by providing a mechanism to dynamically discover and install an agent within an app at runtime. This approach allows an app to run both in a desktop container (using the container's built-in agent) and also in a browser-based micro-frontend container (using a dynamically-installed agent) with zero code changes to the app. The library supports multiple strategies for the areas of agent discovery, creation and bootstrapping - with the ability to easily add and try out additional strategies based on any subsequent discussion / ideas.

The library does not claim to have all the answers by any means, but demonstrates some of the options that could potentially be used for a solution in this area. I hope this will help in the wider discussion that needs to be had within the FDC3 community about how we can best overcome the tight-coupling problem for browser-based FDC3 Desktop Agent implementations - and help move us towards standardized approach in the future.

I was originally aiming to publish this library more like sometime next month rather than now but after seeing that you'd raised the issue, I brought this forward in the interest of the discussion and just published what I have now. Because of this, I haven't actually even proof-read the stream-of-consciousness that I turned into a README yet, so parts of the description might still be a bit rough round the edges.

— Reply to this email directly, view it on GitHub https://github.com/finos/FDC3/issues/896#issuecomment-1400091717, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAM7PBGQKMDQ7S3S72MXK53WTZKALANCNFSM6AAAAAAUAFXPPQ

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/finos/FDC3/issues/896#issuecomment-1400279240, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2PIF2NEGCEAVBWP7JUIXDWTZ4B3ANCNFSM6AAAAAAUAFXPPQ . You are receiving this because you were mentioned.Message ID: @.***>

nkolba commented 1 year ago

@novavi - this is great work! Thanks for sharing.

Just want to note that this is a code project and exactly the kind of work that is vital to the ecosystem. It also underscores that there isn't anything missing from the standard to solve this problem. In fact, as @robmoffat alludes to, Connectifi already provides secure and seamless support across all environments including web and cross-origin in full conformance with the 1.2 spec.

Keeping the standard simple, accessible, and focused only on those problems that truly address interoperability is the best way to continue to drive adoption as well as the innovation of FDC3.

kriswest commented 1 year ago

It also underscores that there isn't anything missing from the standard to solve this problem.

That is not the case @nkolba - it demonstrates that the problem, of establishing standardized ways of retrieving a reference to an API implementation, is solvable. It absolutely doesn't eliminate the need to adopt a standard for how to do so that app developers can write a single way of doing so/single version of their apps. Further, @novavi's write-up on it clearly indicates several areas that would specifically benefit from standardization, or already do, in order to reduce complexity for the app developer. These include the installation of the API reference into window.fdc3, all 3 of the discovery, creation and bootstrapping strategies and the use of internal & external provider directories.

If we don't provide a clear standardized approach we are not keeping the Standard simple or accessible - quite the opposite as application vendors will still have to choose approaches, configure for individual providers, test widely etc. and hence we again encourage the ecosystem to become fragmented and factional. Whereas, with a little work we can likely reduce complexity for app developers significantly, aiming for as close to zero configuration and variation in code as possible. That is quite obviously addressing problems that affect interoperability and ease of adoption, where attempting to block the community from doing so is not.

nkolba commented 1 year ago

As discussed in the standards meeting last week, it may helpful to split this (and the previous email thread) into three separate topics:

Note: while points 2 & 3 are somewhat related, the global issue doesn't just impact web. It is a blocker for conformance support on other tech stacks such as Java, .NET, Python, and many others and forces FDC3 conformance into a very specific type of implementation.

kriswest commented 1 year ago

As discussed in the standards meeting last week, it may helpful to split this (and the previous email thread) into three separate topics:

As discussed at the meeting (and minuted), I agree that the 1.2 spec allows for the FDC3 API reference to be provided by other means (MAY), but does recommend (SHOULD) that it is provided as a global. The SWG voted to change this in 2.0 such that the global is required (MUST) as it was widely agreed that a standard way to retrieve the FDC3 API reference was essential to the Standard as a whole and its uptake with the industry.

Further, it is somewhat awkward that, in order to run the tests on a 1.2 implementation without the API being provided via window.fdc3, the test framework has to be modified with proprietary dependencies and bootstrapping code for each implementation that needs testing - providing an example of the type of challenge that this creates.

  • Whether the global requirement should be continued going forward in 2.x
  • What additional standards (if any) are needed for Web.

I think it an odd suggestion that these two items should be dealt with separately as both relate to the code an app developer must write in order to get at a copy of the FDC3 API. If we don't provide an answer that spans both the web and containers, then this fractures the ecosystem and is likely to result in app developers having to provide different implementations, deployments and appD records not just for browser/container deployments, but also for working with different FDC3 implementations. Whereas, if we work to move the standard forward we can provide a standardized approach that works for all cases. The value in doing so for app vendors, desktop agent vendors and FDC3 adopters is obvious - so I'll refrain from restating it here again. I have, however, yet to hear a decent argument for why not moving the Standard forward on this issue would benefit the community in any way.

Note: while points 2 & 3 are somewhat related, the global issue doesn't just impact web. It is a blocker for conformance support on other tech stacks such as Java, .NET, Python, and many others and forces FDC3 conformance into a very specific type of implementation.

This is not the case as those global requirement has always been scoped to web applications:

API access in 2.0 https://fdc3.finos.org/docs/api/spec#api-access:

The FDC3 API can be made available to an application through a number of different methods. In the case of web applications, a Desktop Agent MUST provide the FDC3 API via a global accessible as window.fdc3.

API Access in 1.2 https://fdc3.finos.org/docs/1.2/api/spec#api-access

The FDC3 API can be made available to an application through a number of different methods. In the case of web applications, a Desktop Agent SHOULD provide the FDC3 API via a global accessible as window.fdc3.

This is expanded on in the supported platforms section in both versions, e.g. in 1.2 https://fdc3.finos.org/docs/1.2/supported-platforms#native:

The FDC3 standard does not define wire formats for communication. Hence, for native applications to be FDC3-enabled, they need to make use of a library (e.g. a DLL in .Net or JAR file in Java) that provides them with an implementation of the FDC3 API. FDC3-enabled native applications are therefore specific to particular desktop container frameworks (or other suitable environments) that provide the necessary libraries.

Despite this limitation, implementing support for FDC3 in a native application can allow it to interact with a wide variety of FDC3-enabled web applications.

I.e. the Standard's documentation acknowledges that the lack of a standardized approach to getting an API reference in other languages is a significant limitation, linking the value that can still be provided without one to interop with web (in this case container apps built with web technologies) where there is an answer. We also lack interface definitions in other languages - although I'm aware of at least a couple of community members working on that problem.

Again, it's rather obvious that the lack of standardisation is what's limiting native implementations and making it difficult to test them for conformance. The assertion that it is the requirements placed on web (which enable its conformance testing) that limit native, is daft.

K

P.S. I'm back in the office tomorrow and will work on arranging the proposed meeting to further the discussion.

novavi commented 1 year ago

Just on the issue of the provision of the window.fdc3 global going forward, for the benefit of anyone reading this who didn't already see the separate (longer) 'FDC3 Desktop Agents vs. API implementations and using FDC3 on the Web' thread, I would like to reiterate here a few key points that I made as part of that discussion:

Of course, there may be some valid counter-arguments to the ones I made above which could make a compelling case for the removal of the global from the standard (and which actually outweigh the disruption caused by the removal). I'd be interested to hear of any. The main one I can think of off the top of my head is if there was a requirement to install more than one DA in a given environment (although I believe this would go beyond what the current FDC3 spec provides for, and I would also have thought that any potential need to do that would likely be significantly undercut by the availability of a Desktop Agent bridge a.k.a. Backplane solution for interop between apps running in different ecosystems / containers that used different DAs).

For reference, the full 'FDC3 Desktop Agents vs. API implementations and using FDC3 on the Web' discussion thread I was referring to is here: https://groups.google.com/a/finos.org/g/fdc3/c/jCvlLjokBLs

kriswest commented 1 year ago

Great summary @novavi

kriswest commented 1 year ago

At the last FDC3 Standards Working Group meeting (#899) it was resolved that a discussion group should be set up to discuss what changes need to be made to how the FDC3 API is accessed, to enable its use in a Web browser (rather than a container or browser extension that makes the API available at window.fdc3).

This is currently only possible through the import of an implementation from a proprietary module (allowed, but not recommended in standard version <=1.2, but not in 2.0), which has significant downsides for the FDC3 ecosystem as it requires an application to be adapted for each desktop agent that it may be used with (which is not the case when working with an injected window.fdc3). A lively discussion of possible ways forwards, and their impacts on the FDC3 ecosystem has already started over email, which should continue in the discussion group.

To help us organise the discussion group at a time suitable for all those who wish to participate, please indicate your interest and dates you could attend below (by clicking on the relevant emojis):


Poll Closed: result: image

novavi commented 1 year ago

Kris mentioned in the 'FDC3 for Web Browsers Discussion group' meeting today (https://github.com/finos/FDC3/issues/908) that people could raise here discussion points about the use of FDC3 in web browsers and/or points for the next meeting relating to any potential evolution of the standard.

I don't actually have any specific issues for the next meeting at this stage (I'm sure I'll think of something nearer to the time). But just to pick up on one point raised today about the alleged inability to use the postMessage API for native browser local cross-origin comms (sorry, I can't remember who mentioned it now). To be clear, postMessage API can in fact be used successfully for native browser local cross-origin cross-window interop.

The restriction in this case is that (unlike the same-origin-only Broadcast Channel API) postMessage API requires a target window reference and also (assuming you sensibly want to avoid reliance on asterisk wildcards for security purposes) a target window origin when posting messages. In the case of an iframe, it can perform limited discovery of sibling iframes because even though it can't access its container's DOM on a cross-origin basis, it can still use the container's window.frames collection on a cross-origin basis. However, even with those sibling window references, it cannot (to my knowledge) read the original or current location hrefs for those windows - or indeed discover the appIds of the apps currently running in those windows. Moreover, there's also the wider problem of interop with external tabs/windows - only the iframe which originally spawned the external tab/window would be able to directly interop with it (i.e. via the window reference returned by window.open() or by window.opener depending on the direction you're messaging) and anything that involved indirect interop by means of republishing messages across a chain of windows would obviously be too brittle to be worth considering - not least because one or more of the windows in the chain could be closed by the user at any time.

This all means that postMessage API can be used for cross-origin cross-window local interop, but that you simply need a solution to the above routing problem. One solution is to use some sort of lightweight 'hub' or 'router' in the original container window to help orchestrate things for the DAs in the app windows (e.g. route interop/channel messages, and open external windows on behalf of the iframe apps and track the references to those opened windows). One consequence of this approach (which is broadly what I used in a limited FDC3 DA implementation myself) is that it matters where a window is opened from. For example, if you opened an external window via the original container (i.e. an fdc3.open() invocation within an app window which delegated the actual window.open() to the parent container), then interop with the external window would work fine. But if you bookmarked that external window, closed it, and later reopened it - then you would have no interop because you'd have no window reference on either side.

In practice, I don't believe the extra hop via a 'hub' or 'router' in the container presents any problems for most real-world scenarios. Now, there might be some limited alternative solutions for peer-to-peer style iframe interop, perhaps involving some sort of handshaking to overcome the initial lack of visibility around the window hrefs (although I'm honestly not sure how you'd do this securely in a manner that would allow you to ever trust the other iframe window). The issue of peer-to-peer local interop with external windows seems intractable to me though - at least based on current browser APIs that I'm aware of . I'd actually be interested if anyone had any different patterns for cross-origin local messaging in general.

Anyway, the end result of all this is that for most real-world scenarios you could potentially use postMessage API for native browser local cross-origin interop - as long as you use a sensible approach for message routing. However, in the specific theoretical situation where FDC3 apps / windows were opened completely independently of each other (a so-called containerless scenario) you would then have to use a cloud-messaging WebSocket API approach, instead of being able to use a local-messaging postMessage API approach. And of course there are a few other scenarios where a cloud-based approach could achieve things that a local-based approach could not support e.g. direct interop with an app on a another device, without using a bridge.

nkolba commented 1 year ago

thanks @novavi - this an excellent summary of what is feasible as well as the challenges with the client only approach to interop in the browser. The reproducibility of a URL and the controls around cross-origin trust are fundamental building blocks of the web and not wanting to break these patterns lead us (at Connectifi) to the cloud approach.

My thoughts post the meeting are pretty brief:

Looking forward to hearing from the rest of the community on this.

novavi commented 1 year ago

thanks @nkolba - you raise a valid point about the reproducibility of a URL. It's certainly true that if a window with a given url behaved differently depending on where it was opened, this could cause confusion to end users. And I agree that a cloud-based approach certainly has the potential to eliminate that particular (undesirable) difference. Having said that - playing devil's advocate for a moment - one could argue that although ensuring reproducibility of a URL would be a good thing to aim for, it's possible that the cost might be paid in greater friction in non-federated identity scenarios, if a cloud-based approach ended up forcing an (undesirable) additional logon for the agent in the given app. This is something the user wouldn't see when opening the same app in either a desktop container or in a MFE container that used local interop. So it's possible that there could be trade-offs, depending on the specific setup of the institution and the ISVs involved. Of course, every setup is different - which is why I think a good result for the community would be to have a standard which encompassed all the common scenarios, and a range of solutions from agent providers to cater for those different scenarios.

I have to say that - based on my current knowledge of the FDC3 standard at least - I don't really agree with the assertion that removing the window.fdc3 as a MUST has no clear downside. I believe the downsides of removal would be the inability to use window.fdc3 to directly detect the presence of a pre-existing agent (in the case where a container-agnostic app could be run both in desktop containers and browser-based MFE containers) and also the fact that window.fdc3 is currently closely intertwined with the agent bootstrapping process and the standard fdc3Ready() behaviour. For these reasons, it seems to me that any removal would require (a) a new standardized mechanism for desktop containers to provide their FDC3 DAs (I've not yet heard anyone suggest what this could/should be); and (b) a change to the behaviour of fdc3Ready(). Point (b) in particular seems problematic for ISVs because an app that is expected to work across different FDC3 versions (e.g. where it only uses basic FDC3 functionality) could not easily adjust its usage of fdc3Ready() based on the version of the agent or version of FDC3. This is because the app shouldn't even be invoking fdc3.getInfo() to determine the version until after it has awaited fdc3Ready(). But the app also couldn't rely on fdc3Ready() if the behaviour depended on the version of FDC3. This feels like a chicken and egg situation - perhaps there's a way around it that's not clear to me at the time I write this. For reference, I also listed some of these points above (https://github.com/finos/FDC3/issues/896#issuecomment-1411280183)

Now that I think about it, I've yet to actually see a convincing upside for any proposed removal of window.fdc3.

Firstly, I don't particularly buy into the argument that static import declarations are necessarily better in every circumstance just because they are the more modern ES way. To me, a DA feels like a different category of import than something like a UI component (where an import declaration would always definitely be better) in that it's an API that can be injected into the environment in the case of a desktop container. For me, this makes it more analogous to a polyfill for a new browser API which hasn't yet been made available in all browsers. Additionally, it's not clear to me that a static import declaration is a good method for loading an agent in any case (since import declarations can't be used in conditional blocks, this would imply either fixing to a single specific agent at build-time or alternatively including multiple agents in the resulting script bundle even though only one was required at runtime). That's actually what caused me to look at the idea of an installer library in the first place.

Secondly, I think I'd probably need some convincing that there's any significant downside to having the agent installed in a global (instead of inside part of the main app module) from a security perspective. In particular, having the agent hidden in the revealing module pattern or similar doesn't seem to offer any meaningful protection from being tampered with by malicious code. For example, if someone wanted to write malicious code to use as part of a XSS attack to read all inbound and outbound interop messages, it would actually be far easier to directly target the native browser APIs that the FDC3 DA relied upon (postMessage or WebSocket, to attack local or cloud interop approaches respectively) than to target the layer on top i.e. the multiple intent/channel methods of the FDC3 DA itself.

However, I can believe there could be some genuine upsides which I'm not fully aware of - so I'm definitely interested in hearing any other arguments on the matter.

As for the question of open source vs FDC3 standard in terms of the strategy for an installer library or similar, my main concern with an open source approach would be the risk of untested/unmaintained implementations and unnecessary duplication/fragmentation. To this I would also add the possibility of confusion in the case of smaller ISVs where the developer knowledge of FDC3 may not initially extend much further than their boss having mandated some work with a deadline and a simplistic "Get FDC3 done" slogan. More specifically, if you think about the combination of multiple open source agent installer implementations with (over time) different versions of the FDC3 spec and (as the market grows larger) multiple competing desktop containers and web-based agent providers, it could easily lead to smaller ISVs building apps that don't do a good job of installing an FDC3 DA. At best, this might mean the app not working reliably or consistently across different containers. At worst, this might risk an approach containing security flaws that would not be limited to that particular app i.e. which could potentially compromise the interop of the whole ecosystem / MFE container that the app was loaded into (even when all the other apps in the ecosystem had done the right thing). This is why I would argue in favour of having some form of installer library or equivalent mechanism as part of the FDC3 standard itself - where it can be agreed, designed, implemented, tested and maintained in a manner that promotes confidence in FDC3 as an open interop standard and enables growth in FDC3 adoption amongst ISVs and institutions. I appreciate that others might take a different view on this, so I'd also be interested to hear from the rest of the community.

kriswest commented 1 year ago

Hi,

  • the window.fdc3 requirement is the only clearly identified issue with the spec relating to FDC3 in browser

Like @novavi, I heard no clearly articulated advantage to this proposal, nor much support for it at the meeting. We did however hear from a number of speakers with concerns about such a proposal, particularly with no clearly defined successor to the global. Many of us want to avoid creating a situation where:

As such, I continue to believe that it is in fact the inclusion and bootstrapping of proprietary libraries, without a standardized pattern or open/cross-vendor library support, that is the greatest issue facing FDC3 on the web. This is particularly the case as it will take away the ability for an app to be (Desktop Agent) vendor agnostic if it wishes to be (highlighted as an important property of FDC3 at the meeting multiple times).

Other interesting points that we heard at the meeting:

Hence, the overly simplistic proposal of dropping the requirement for the fdc3 global is a distraction from the real issues we need to work on. A solution to those may see the global superseded - but its existence does not block the use of FDC3 on the web.

kriswest commented 1 year ago

@api-forever

Note that the client-side integration with an IFRAME requires the ability of the application to communicate its content height to the parent window which hosts the IFRAME, so the protocol for this should be added to the FDC3 spec, too.

A good point (as most solutions to that I've seen use postMessage between the parent window and the iframe). However, not every layout will require that (iframes might internally scroll, or you might use full windows via window.open only). Nevertheless, it seems worth considering a solution.

Both patterns must continue to work if there is an injected window.fdc3, therefore a fail-fast algorithm for determining whether to wait for the fdc3Ready event should be added to the FDC3 spec. One possibility is to inject window.fdc3 = null as soon as this becomes possible during page loading and then replace it with the full implementation before firing fdc3Ready, or removing it before firing fdc3Ready or another event if the agent decides not to trust the application web page after additional security checks.

Also a good point. This advice on API Access in the Standard will almost certainly need adjusting to facilitate testing for the (impending) availability of an implementation:

The global window.fdc3 must only be available after the API is ready to use.

https://fdc3.finos.org/docs/api/spec#api-access

nkolba commented 1 year ago

FWIW I don't we're that far apart in what we're saying and I think we all agree that producing a common library is the best way forward. Perhaps the best course is to put the larger questions on the standard to the side for now and focus on that.

kriswest commented 1 year ago

Thanks Nick. That seems like a reasonable focus for the next meeting's agenda, see you there.

novavi commented 1 year ago

I assume we're talking about a common installer library here? If so, this sounds positive. I think to move forward it would be worth first defining clearly what should be in and out of scope for the initial focus on an installer library. Based on the discussion so far, there are many wider issues raised by the use of FDC3 in a browser-based environment which can probably be kept in mind but largely parked for discussion purposes (this includes the idea of running different agents in different apps, which was floated in the last meeting). So the idea of sub-dividing the standard as @kriswest suggested to allow an initial focus on API interface and methods of retrieving access to it seems sensible to me.

Clearly everyone will have their own views - and I'll be very interested to hear everyone else's - but in the meantime some of the things that spring to mind when I think about a common installer library now include:

I could go on further, but will just put the above thoughts forward for now. Look forward to discussing more in the next meeting.

novavi commented 1 year ago

For the record, I should correct something that I mentioned in one of my recent posts about browser-based local interop above (https://github.com/finos/FDC3/issues/896#issuecomment-1433771266) where I stated that postMessage API requires a target window reference - and that this always imposes limitations around app launching and requires a common 'hub' or 'router' window for messaging.

I had originally ruled out Broadcast Channel API completely on the basis of the same-origin policy restriction it imposes. And since postMessage API is (as far as I am aware) the only native browser local messaging API that works for the cross-origin scenario (which is necessary in most real-world situations) I had thought that one would just have to live with the requirement for window references. In practice, that would mean apps delegating interop and window opening functionality to a 'hub' or 'router' running at the micro-frontend container level. Moreover, it would mean that if a user launched an app independently (i.e. in manually opening a separate window or tab, and navigating to the app's url) then that app could not participate in local interop.

However, with a bit of lateral thinking, I can now see that there is a fairly simple mechanism to overcome the above problem. This mechanism effectively provides for cross-origin interop between isolated independently-launched windows. The solution I came up with uses postMessage API in conjunction with Broadcast Channel API, and it works along the following lines:

This goes further than I previously thought was possible with browser-based local interop. To be clear, based on a spike I just created, the above solution appears to work - but I have to confess I haven't (yet) tried it in anything beyond that limited spike.

Of course it's worth noting that for a local browser-based app to interop with a local desktop container or a remote (e.g. Citrix-based) container, then cloud-based interop is still absolutely necessary.

kriswest commented 1 year ago

Of course it's worth noting that for a local browser-based app to interop with a local desktop container or a remote (e.g. Citrix-based) container, then cloud-based interop is still absolutely necessary.

@novavi This is a case a Desktop Agent Bridge could/would solve for (as a browser page could connect to local websocket).

novavi commented 1 year ago

@kriswest Absolutely

novavi commented 1 year ago

Clearly everyone will have their own views - and I'll be very interested to hear everyone else's - but in the meantime some of the things that spring to mind when I think about a common installer library now include:

Standardization of agent provision - the may be a common ground where a balance can be struck between allowing

A couple of other things came into my mind since I posted that comment (https://github.com/finos/FDC3/issues/896#issuecomment-1439144348) above. This is because those thoughts were largely based on the experience of the fdc3-installer library I put together previously - which at the time I had only ever tested on a dummy DA implementation with a single getInfo() method. However, thinking a bit further about the requirements for an installer library in the context of a real DA implementation raises a few more considerations:

robmoffat commented 1 year ago

I'm a bit confused about how @novavi's Desktop Agents would know about each other in the scenario above.

But I think there is probably some value in establishing a common wire format for FDC3 messages to suit both postMessage and Sockets.

FDC3 Common Wire Format

I think @kriswest said in the meeting that his team were working on a JSON Schema for this. I would imaging that it could be reused by our postMessage-transport from DA to app and back as well as the DA-to-bridge WebSocket-transport.

novavi commented 1 year ago

Thanks @robmoffat.

I should have been clearer in the point I was trying to make about interop between isolated independently-launched browser windows. The Desktop Agents would know about each other because in my example they were from the same vendor, and would connect on a peer-to-peer basis (publishing intent and channel messages, along with details of currently-running apps, etc. to each other as and when each app's DA instance bootstrapped). So this was all specifically about the example of a single DA provider. In terms of bridging though, I do agree that there is probably value in establishing a common wire format for FDC3 messages. That would certainly open up opportunities for communication between different desktop agent providers (which goes beyond what I was talking about).

What I was trying to get across was that there is another pattern for comms that could potentially be used by a browser-based DA. This is irrespective of any future common wire format because - until you introduce bridging across different DA vendors into the equation - a given DA only needs to concern itself with its own comms pattern. On this basis, there are potentially three different approaches for implementing a DA in a browser-based environment (I mean a vanilla browser, not leveraging any browser extension) to route FDC3 interop messages between different windows:

Pattern 1 - via the cloud, utilising a WebSockets connection. This is the approach Connectifi takes. Crucially, this pattern works for isolated independently-launched browser windows.

image

Pattern 2 - via a main window. I used this pattern previously in a fairly-extensive-but-incomplete DA implementation to support intent and channel FDC3 interop. I suspect this is also broadly the approach taken by the browser-based Glue42 Core solution - but someone from Glue42 would need to confirm or refute that. This patern does not work for isolated independently-launched browser windows - it only works if all the app windows are spawned from the main window (either directly, or by the app's DA delegating window opening to the main window in order to allow tracking of window references for subsequent use in message routing).

image

Pattern 3 - via a local browser messaging cross-window cross-origin bridge (not using WebSockets, and not relying on a main window). In the example below, App 1 runs on port 3100 and App 2 runs on port 3200. Most importantly, the Isolated Window Bridge hidden iframe within both app windows runs on port 4000 (to satisfy the same-origin policy restriction imposed by Broadcast Channel API). My test used ports on localhost, but in a real-world scenario clearly the windows would use different domains / subdomains. The approach works for isolated independently-launched browser windows on a cross-origin basis (despite the fact that it relies purely on local browser messaging).

image

The main point is that it's technically possible to bridge between two isolated independently-launched browser windows using local browser messaging only. I was actually kind of amazed to find that a third pattern was technically possible - I only thought about it and created a successful spike on it a few days ago. Prior to last week, I was labouring under the illusion that cross-origin interop in a browser environment was only possible using Pattern 1 or Pattern 2. Hence in the interest of the overall discussion on FDC3 For Web browsers, I thought it was at least worth calling this out for the benefit of anyone else who also hadn't realised this was possible.

I was always very conscious that:

But what I hadn't tried until last week was using postMessage API in conjunction with Broadcast Channel API - together with judicious use of iframes on a common 'bridge' origin.

Note that in many (most?) circumstances there is still good reason to have some sort of main window or launcher window in any browser-based micro-frontend ecosystem for the purposes of discovering and launching apps and workspaces/layouts. The point is that for cross-window interop in a pure browser scenario, a single DA implementation used within individual app windows does not necessarily need to use a main window for interop message routing (this has the consequence that the main window could be closable by the user without breaking interop between the remaining app windows) and does not necessarily need to use WebSockets.

To be completely clear, none of this has any direct bearing on a common wire format or on the use of WebSockets. The common wire format idea makes sense to me. And there will always be a need for WebSockets for cloud-based DA vendor solutions and also for bridging across different DA tech stacks (e.g. into desktop containers and legacy native apps) and across different environments (e.g. Citrix or mobile devices). The above Pattern 3 DA does not preclude the use of a cloud-based bridge between a Pattern 3 DA and any other flavour of DA (all it does is provide the internal comms for itself).

novavi commented 1 year ago

One further thought.

Interestingly, the use of a hidden iframe might also have other potential usages in DAs - beyond the postMessage API <-> Broadcast Channel API <-> postMessage API pattern I floated above for a local-browser-messaging DA.

For example, it might be possible to use a postMessage API <-> WebSocket API <-> postMessage API pattern in a cloud-bassed messaging DA. One potential challenge for cloud-based DAs concerns AuthN/AuthZ across multiple apps from different ISVs in non-federated customer scenarios. In those scenarios there could be some friction involved in getting individual apps onto a service bus (this is on top of the existing requirement to get the user logged into each individual app). However, if a cloud-based DA library proxied each app window to a 'Service Client' hidden iframe for WebSockets interaction with its cloud-based service bus, this would have the potential to reduce that friction. In particular, the scenario would no longer require getting each individual app onto the service bus. Instead, it would require auth for the service bus on the basis of the user session instead of on a per-app basis (with the user <-> app auth continuing to happen as normal). Friction could potentially be reduced even further if the 'Service Client' hidden iframe could be configured to run on the same iframe as the ecosystem's container / launcher / entry point (meaning the auth for the service bus could happen at initial launch of the first component in the ecosystem, rather than on each subsequent app opening). This is all if the DA actually supported such a security model.

image

Overall this type of approach could make a cloud-based FDC3 DA browser app ecosystem operate more along the lines of the experience a user enjoys in a traditional desktop container (where each app does not have to separately auth onto the container's DA - and where trust is established firstly for user <-> desktop container DA, and secondly for user <-> app(s)).

Clearly there are many trade-offs in this area, so there's possibly a case for both patterns (i.e. direct WebSockets, and also 'Service Client' hidden iframe WebSockets). It's also the case that although this is something that could (optionally) be supported directly by a cloud-based DA if it was considered useful / appropriate, it would be a bad idea for an app to randomly attempt to use a cloud-based DA in a manner like this that was not intended / expected / supported by the DA and its associated service bus. Ultimately, a cloud-based DA might have a single security model which always required each each app to auth separately onto its service bus (i.e. requiring trust in the app at runtime, rather than just trust in the app within the appD and trust in the user).

I should also say that whilst the above patterns might not directly effect the use of all installer library proposals, it's probably worth at least keeping some of these possible DA implementation patterns in the back of one's mind when thinking about what things a standardized installer library might end up having to contend with (e.g. a DA that already used iframe proxying even before any potential additional proxying caused by some dynamic installer library solutions which themselves required an iframe to run the DA inside).

Additionally, it is also worth noting that the proposed common wire format could well be the most important part of an eventual DA bridging specification. This is because although many DA bridging scenarios will involve two different tech stacks or environments (which therefore require a cloud-based WebSockets solution), it looks like it's possible that a simple DA bridging scenario involving only browser windows with different DAs might be solvable with an implementation that relied on a postMessage API <-> Broadcast Channel API <-> postMessage API pattern (instead of requiring a full cloud-based WebSockets solution).

kriswest commented 1 year ago

@novavi just one item to pick up here:

This is because although many DA bridging scenarios will involve two different tech stacks or environments (which therefore require a cloud-based WebSockets solution)

Bridging as currently proposed uses websockets, but provided by a locally running executable (the bridge), with bridges themselves being able to cluster for multiple devices. It is therefore not exactly 'cloud-based' - some of those involved initially were keen to ensure that comms stayed on device or at least on the local network. I suspect that may be an important distinction for some InfoSec teams.

thorsent commented 1 year ago

I'm really looking forward to tomorrow's discussion. A lot of technology has been uncovered.

From my perspective, I think we should aim for a bootstrapper that is as small and simple as possible, with the prime objective of (1) identifying the desktop agent and (2) loading the desktop agent's proprietary JavaScript (in order to instantiate the fdc3 object).

To that end, I would like to suggest utilizing web manifests. A dedicated fdc3 section can contain instructions for the bootstrapper (this is similar to @novavi 's suggestion of using a file in the root but leverages an existing standard).

Instructions might then be one or more of the following (perhaps specifying precedence): a) use global fdc3 object (preload style) b) given load (iframe/pop-up style - e.g. Glue42) c) use finos.org iframe to obtain from running agent (runtime style) d) load - (deploy-time style - e.g. Connectifi)

P.S. @novavi, for the hidden iframe trick, a sharedWorker is another possibility, with the advantage that it is a singleton by design. A second possibility is simply to use localStorage. A running desktop agent could simply drop its url into a well-known slot in local storage where it can be picked up by any window wishing to connect. This last one has the advantage that the desktop agent need not even be running.

novavi commented 1 year ago

Ah, many thanks for clarifying @kriswest, that's an important distinction.

This makes sense - keeping the comms on the local environment (i.e. the same model that desktop containers follow) could be important for some institutions. The use of local comms could also reduce potential friction around auth, and perhaps also questions over which ISV / ecosystem / institution owned/operated the bridge service bus.

From an ISV's operational perspective, I imagine the trade-offs with using a local executable include potential challenges around reliability (e.g. handling bridge installation / runtime / shutdowns / restarts / updates and any associated corner cases across range of different end-user environments) and difficulty for operations / support teams in easily inspecting log files if a problem requires diagnosis. Those are issues that ISVs / institutions likely wouldn't face if using a cloud-based solution (whether for bridging purposes, or using a cloud-based DA vendor). I'll be very interested to see how this area develops (the operational aspect as much as the technical specification and solution really). And I wonder whether for some customers, the option of running a self-hosted bridge (e.g. on their own cloud infra) as an alternative to installing a local executable on end-user machines might be attractive. I mean in the same way that for a cloud-based DA vendor, providing customers with different options (in that market, SaaS and self-hosted infra models) is likely to be important to satisfy diverse technical, operational and legal requirements.

novavi commented 1 year ago

Thanks @thorsent - this all sounds good to me, and very much chimes with a lot of the conclusions I've reached in my exploration so far.

From my perspective, I think we should aim for a bootstrapper that is as small and simple as possible, with the prime objective of (1) identifying the desktop agent and (2) loading the desktop agent's proprietary JavaScript (in order to instantiate the fdc3 object).

I agree - small and simple will be the best way to encourage ISV apps to benefit from it. I always hoped that if an installer lib could be done as part of the FINOS FDC3 standard itself (rather than as an external lib) there would be a great opportunity to make the standard for web-based DA provision clear enough that a lot of potential installer configurability would become unnecessary. In particular, this could completely avoid the need to the type of creation and bootstrap strategies I previously POC'ed (which themselves largely served to highlight how this area would clearly benefit from some standardization).

To that end, I would like to suggest utilizing web manifests. A dedicated fdc3 section can contain instructions for the bootstrapper (this is similar to @novavi 's suggestion of using a file in the root but leverages an existing standard).

I like this idea - leveraging an existing standard would be better. Also on this subject - when I paused the work I'd done on the fdc3-installer POC lib, I'd come to realise there was a potential overlap between installer config and DA config. I mean in the sense that I approached this using static values specified in a config file to pass as DA creation (e.g. factory function) arguments, but it would be equally valid for the installer not to pass any static values and have the DA read the config/manifest itself in order to obtain the static values. I believe Glue42 Core currently allows config to be specified as an object literal on the global window object (above the DA script include). This also points to the potential benefit for standardization in the area of configuration more generally. Perhaps there is even a case for three categories of FDC3-related instructions in a manifest:

Beyond the values that could be specified in a manifest, I think there's likely still a need for an app to be able to pass non-static values (e.g. callback functions), which could perhaps be provided to the installer method and passed through to the DA at creation time (see further down for more thoughts on this).

Instructions might then be one or more of the following (perhaps specifying precedence):

Yes, this is good - I like the idea of being able to specify an order of precedence.

b) given load (iframe/pop-up style - e.g. Glue42)

I'm guessing this is possibly along the lines I described in the 'Pattern 2' diagram in my previous post (https://github.com/finos/FDC3/issues/896#issuecomment-1446871544) but I'm not 100% sure?

c) use finos.org iframe to obtain from running agent (runtime style)

Having a common, trusted DA domain makes a lot of sense. This also allows app developers to quickly get up and running and to test this pattern without the need to setup an ISV- , institution- or ecosystem-specific common DA domain. However, I would add to this that from a production deployment perspective, there are possibly a few additional considerations. These include:

For the reasons above, it might be worth considering providing finos.org as a default for the common DA iframe but also allowing some configurability in this area. And of course there's an interesting question of which DAs a common DA domain would allow to be loaded. If this was not locked down (e.g. with no CSP restrictions) it could open up a potential attack vector. On the other hand, if it was a closed set of DAs then that would imply the common DA domain having some role in curating which DAs could be loaded (which for finos.org would likely be problematic for a number of reasons, not least the ability to test an unreleased version of a DA or a non-public DA, which would likely require more than just a single common domain e.g. dev.finos.org / qa.finos.org / uat.finos.org or similar).

There's also another dimension to this. One of the examples commonly talked about is where a browser-based ecosystem and a desktop container ecosystem need to interoperate. But if browser-based FDC3 becomes more commonplace, it's not difficult to imagine a scenario where an institution uses two different browser-based micro-frontend container ecosystems (there are different permutations here, but it doesn't really matter whether it was one in-house with one ISV, or it simply two different large ISV solutions both having their own ecosystems).

So in this scenario we could have MFEC X hosting App X1 / App X2, and MFEC Y hosting App Y1 / App Y2. MFEC X might use DA A, whereas MFEC Y might use DA B. Let's leave bridging out of the equation here, and just concentrate purely on the DAs that the individual apps need to discover, install and use. If a single common DA domain (e.g. finos.org) was used by the installer lib, it could prevent these apps using the iframe approach for installing their DAs - because the apps would not be able to disambiguate the DA at runtime.

This implies it might be better for apps to be able to specify a preferred common DA domain. However, if we're not careful then this could lead us back down the path of app <-> DA lock-in that the installer lib is supposed to be solving - so I suspect it would be better to allow an app to infer a common DA domain at runtime from the container that launched it (rather than to have the app always hardcode the common DA domain). That would allow App X1 / App X2 and App Y1 / App Y2 to discover and install their appropriate DA at runtime.

Without wanting to muddy the waters too much, it might also be worth considering what would happen if you mixed container-launched apps like those in my example above with containerless apps. If App Z1 (developed by a partner ISV) was independently launched in an isolated window, it's not obvious that it could directly (i.e. without bridging) participate in interop with either App X1 / App X2 or App Y1 / App Y2. That might mean App Z1 would need hardcode to either DA A or DA B (which would be undesirable) or it might mean App Z1 could use a different mechanism to discover an appropriate DA (e.g. perhaps using something along the lines of the AppOrigin or AppSessionStorage discovery strategies I laid out in the fdc3-installer POC lib, but with the ability to use the discovered DA in a hidden iframe for interop with the other apps already using that DA).

a) use global fdc3 object (preload style) b) given load (iframe/pop-up style - e.g. Glue42) c) use finos.org iframe to obtain from running agent (runtime style) d) load - (deploy-time style - e.g. Connectifi)

One question that follows on from this is whether the standard would require a DA that was loadable via techniques (b) or (d) to also be loadable via technique (c). The question really is really whether that list merely specifies different mechanisms to load DAs, or whether the choice of discovery/load mechanism would be based on (or at least strongly influenced by) the particular flavour of DA implementation? I say this because (without modification) a DA that runs successfully via technique (b) might well break at runtime if loaded via technique (c). Therefore it's probably worth an installer lib being clear on the boundaries between DA discovery/installation and DA implementation.

A related issue of course is window.fdc3 installation. In particular, who does it - the DA or the installer library? Or would you see the standard potentially taking a different path here (with all that entails)?

P.S. @novavi, for the hidden iframe trick, a sharedWorker is another possibility, with the advantage that it is a singleton by design.

I like this idea a lot! I curtailed my exploration in this area before I got quite that far, but this makes a lot of sense to me. It would allow the DA to run in a single location and avoid race conditions, with an installer just providing a surface API layer at the app window level to proxy to the sharedWorker running in the iframe.

A second possibility is simply to use localStorage. A running desktop agent could simply drop its url into a well-known slot in local storage where it can be picked up by any window wishing to connect. This last one has the advantage that the desktop agent need not even be running.

This is also an intriguing idea, and could potentially be a good solution.

On the iframe technique more generally, there is one more point I should make - which is to reiterate the issue of non-static DA arguments. I gave an example in a previous post on this thread about the possible need for apps to pass callback functions to customise the behaviour of a DA (e.g. logging, Resolver UI, or anything else). Clearly, unlike the case of static values, this is something that cannot be handled in a config or manifest. If an installer lib was simply importing the DA directly in the app window, then the implementation is trivial (the app would pass the callback function args when invoking the installer, which would in turn pass them to the DA's factory function). However, if the installer lib was importing the DA in a hidden iframe and providing a DesktopAgent interface in the app window to proxy the app's FDC3 API calls through to the hidden iframe via postMessage API, then it's not immediately obvious how callback function args would be provided to the DA implementation. One way of handling this would be for the installer lib to create reverse proxy functions in the hidden iframe at runtime which used postMessage API to call back to the (parent) app window, with an event listener that app window receiving the messages and invoking the callback functions that the app originally provided to the installer lib. Of course, that would add an extra degree of complexity, so it would be worth thinking through any other solutions.

It seems to me that a lot of these issues are at the intersection of MFE containers, installer libs, DAs, DA bridging, and the overarching issue of interop security in web browsers. Unfortunately, lack of standard for MFE containers isn't something we can solve as part of FDC3, so I think the best we can achieve is a solution which supports a range of common MFEC implementation patterns. It does feel like a standardized solution is within reach, so some of this comes down to striking the right balance between overcoming the app <-> agent lock-in problem (in order to preserve FDC3's spirit of openness) but not suppressing innovation in the DA vendor space.

nkolba commented 1 year ago

Hey all,

@kriswest @robmoffat @novavi @thorsent Unfortunately, I won’t be able to make today’s meeting. There are a lot of great thoughts and insights in this discussion - so, definitely looking forward to seeing the output from it.

-Nick

robmoffat commented 1 year ago

I have a very simple "straw man" design that you could shoot down for me. This is likely a small subset of everything @novavi has tried.

The upshot of this is that a Desktop Agent could be written in a web page to talk to apps using this. It could open windows, or child iframes to contain FDC3 apps. It communicates with these applications using the standard FDC3 wire protocol running over postMessage. But: it would only be able to communicate with apps it has opened itself.

novavi commented 1 year ago

Hi @api-forever.

I suggest sending postMessage to the opener/parent first rather than waiting for one, for the following reasons:

  • the opener/parent can set up a listener in advance, without a race condition

I must admit I haven't fully processed @robmoffat's straw man in the context of the wider discussion yet. But I think what you said makes sense to me, if I've understood what you and @robmoffat were getting at here. There's definitely a need to avoid race conditions. I'm guessing you perhaps mean using postMessage in both directions e.g. first setting up a listener in the parent window, then once the child window is actually loaded it could postMessage to the parent window, and then finally the parent window's listener could process the request and use postMessage to send a response back to the child window. All with a uuid stamp on the messages to effectively simulate a higher-level async request/response API that can be used by the child window.

Just to pick up on a couple of the other points you raised:

Furthermore, it would be useful to have standardized means of running the application without an FDC3 agent, so that one does not have to wait for a timeout when it is known in advance that an FDC3 agent is not going to be used.

I must admit, I don't fully understand the use case here - so it would be useful to know more about it. If the app in question knew in advance that it did not need to use an FDC3 DA, then it could simply avoid importing and invoking the installer library. This approach ought to be straightforward, because the idea behind the proposed installer library would likely be to use it on a route-specific basis inside only the apps or components that actually require FDC3 (i.e. immediately above where the app/component currently invokes await fdc3Ready();) rather than using it on a universal basis inside an index.html host page or similar (where that host page could theoretically be used to load FDC3-enabled apps or non-FDC3 apps, depending on the route).

Alternatively, if an app wanted to leverage FDC3 when available, but use graceful degradation so that it was still functional even without FDC3 (albeit requiring a more manual user workflow) then the app could simply use conditionality on the result of the FDC3 installer invocation and around any subsequent FDC3 DA API calls (to be fair, this might require try/catch in the case where FDC3 DA acquisition timed out - this would depend on the exact installer implementation and the desired app behaviour).

One way of doing this is adding a special URL parameter, such as "$fdc3", which would indicate that the FDC3 agent will be provided.

If I've understood this correctly, I'd be fairly cautious about an FDC3 installer lib (or indeed an FDC3 DA) requiring a specific url parameter on an app to dictate behaviour. In the days of classic web apps (i.e. pre-SPAs) one could often simply add additional url querystring params and be reasonably confident of no side effects if those params were uniquely named. For modern web apps though, there are now a variety of different url patterns supported by competing JavaScript frameworks and open source routers which involve hash-based and push-state client-side routing, along with a wild west of other weirdness still in use (e.g. when Angular 2 came out, it revived a little-used semicolon-delimited IETF url param standard dating from nearly 20 years earlier). I've also seen apps (unwisely) mix one than one param pattern in a single url, which makes it even more unreliable for code external to those apps to make assumptions about how a url param should be provided (unfortunately we cannot legislate against stupidity). Therefore to make an FDC3 installer lib handle the widest range of ISV FDC3-enabled apps I think it would be best to avoid reliance on url params if possible. (Or to put this less strongly, if url params were to be used then it would be best if that represented simply one option for providing args rather than the only option.)

Finally, I would add that another problem with using url params to determine FDC3-related functionality (whether for this purpose, or to define which DA to use at runtime) is that an ISV cannot prevent a user from bookmarking a url. Therefore the use of url params in this area has the potential to waste the time of an ISV's support team if it allows users to launch apps independently in ways that were not originally intended by the ecosystem - or in ways that are simply no longer appropriate following future updates of the app (or its installer lib dependency).

robmoffat commented 1 year ago

My current solution for this is a URL parameter for enabling FDC3.

Given @novavi's caution against this, could this be handled as a DesktopAgent-implementation constructor argument?

novavi commented 1 year ago

Hi @api-forever

Thanks for explaining - this is an interesting case. It's one that I hadn't considered before, and haven't seen discussed by others (yet) in the limited time the 'FDC3 for Web Browsers' subgroup has been running. I had taken the view until now that an app using an FDC3 installer library would be expecting to use FDC3 in every circumstance. I can see now that the render time and rerendering issue mean that it is undesirable for your apps to wait for FDC3 DA acquisition timeout.

The use case consists of apps that support a number of optional UI-level integration protocols, including FDC3.

Whether or not this falls under the scenarios that a standardized installer library could/should handle is hard to say at this point. @kriswest has certainly been keen to solicit opinions of ISVs (and I'm sure has also taken representations from ISVs directly) in order to ensure that the proposed standardized installer library will satisfy real-world use cases and provide a solution to the most common problems - the primary one being DA discovery and install in the case where use of a FDC3 is a given. One of the trade-offs with a standardized solution is that it's unlikely to handle every single every corner case. This is because there are undoubtedly another half dozen more highly-specific scenarios that other ISVs could require, and supporting every single might add too much complexity to the library. That said, I think all use cases should at least be put on the table for debate by the interested parties - and I'm sure this one could be raised at the next meeting.

However, there ought to be relatively easy way of solving your problem even if your use case wasn't directly handled by an installer lib. An installer lib will handle selection of DA vendor (this might include pre-injected, already-running on a given SharedWorker, inferred and dynamically imported, etc. as @thorsent suggested) so it seems like most of what you want to achieve could be satisfied by adding an extra level of conditionality (whether to use FDC3 or not). More correctly - given what you said about your apps supporting a number of optional UI-level integration protocols - perhaps it's not just conditionality around the use of FDC3, but more selection of interop type (FDC3 being just one of these types). The fact this relates to a wider issue of handling multiple interop types might also point to this being something that could fall outside the scope of a standardized FDC3 installer lib.

To avoid re-implementing this conditionality or interop type selection in all your individual apps, theoretically you could have your own ecosystem-wide common handler which wrapped a standardized FDC3 installer lib and extended it with the additional conditionality that is specific to your use case. And if you also had other third-party ISV apps running in your ecosystem, you'd likely want to put your common handler in a package they could consume (one could argue that the more ISVs required this pattern, the stronger the potential case would be for handling it directly in a standardized installer - as opposed to handling it within a package specific to your ecosystem).

My current solution for this is a URL parameter for enabling FDC3. While it could be different for each application or application vendor, it would be easier to use if it had a standard recommended name.

In terms of using a url parameter within a library which needs to cater for different app implementations and avoid clashes with existing app param names (ideally keeping this optional because of some of the issues I mentioned previously in the area of url params) one solution to this is to make the param name configurable rather than have any standard recommended name. I did play around with something along these lines in the fdc3-installer POC I worked on recently - see https://github.com/novavi/fdc3-installer#appquerystring for an example of this. That was of course related to selection of a specific DA, rather than conditionality around whether to use FDC3 or not. Note also that it was just a POC - it has not been defined yet whether a standardized installer lib would even (optionally) support querystring params.

A final point to make would be that you could potentially make use of window.name projection from your container to your individual apps as an alternative to using url params. I don't think this is a particularly widely used technique for passing params to windows, but it is something I have used successfully myself on a previous project. This has the advantage that the param (the details or behaviour of which might change over time) can never be inadvertently snapshotted in a user's bookmark. This projection from container to app is possible by means of the one-time binding for an iframe's name attribute, a window.open()'s target argument, or an anchor's target attribute. In practice this technique is only feasible in a scenario where you have complete control of iframe and child window creation (since the binding is one-time rather than one-way, it needs to be set upfront at window creation time). Based what you said about your portal, you probably do benefit from this degree of control - so it might be something you could consider for your app-specific use case (even though the technique is a harder sell for use in a standardized lib which couldn't make those sort of assumptions about attribute projection on window creation events by MFE containers and DAs it doesn't control). See https://github.com/novavi/fdc3-installer#discovery-strategies-explanation and https://github.com/novavi/fdc3-installer#appwindowname for some notes on this (of course you wouldn't have to use ordinal position values, that's just how I happened to encode the data in that example).

It might be possible to skip this parameter in the window.fdc3 injection use case if the agent discovery protocol could be modified along the lines described in my previous post

I think that window.fdc3 as a concept is still under discussion, but your use case might perhaps feed into that discussion.

Working for an ISV myself I possibly have a limited frame of reference, so I'll let @kriswest, @robmoffat, @thorsent or others chime in on your use case anyway - they likely have more insight on the scenario, whether it's something they've seen used by other ISVs, and how it might fit into the big picture.

novavi commented 1 year ago

Ideally, the apps should work with any opener/container/portal supporting FDC3, from any vendor. Also, the apps themselves can be from any vendor.

Ah... well if this is something you would expect to work even outside of the portal you control (as well as within individual apps you also don't control) then that changes everything - and I can see why you ideally would want this to be part of the standard in some fashion.

In general, as I mentioned, I'm personally quite wary about having things like MFE containers and generic libraries making decisions based on url formats in apps that they don't directly control, apps which are built using arbitrary frameworks / routers, and apps which the container or library can't guarantee to understand - unless this is absolutely necessary for an important and common use case (and done strictly on an opt-in basis). But others may disagree, and perhaps I am unduly pessimistic - so I believe it's worth a discussion.

Using a URL parameter works, and is FDC3-compliant (one just specifies such a parameter for each URL in the app directory)

How are you specifying this in the appD entry though? Doesn't this mean you currently need two entries for each app in the appD though i.e. one with the fdc3 param and one without? Or does your portal simply not use an appD entry when it opens the non-FDC3 variant of the app?

I suppose we'd need to know more about how and when your portal actually determines whether to use FDC3 interop for an individual app. For example, I'm not clear whether this is an ecosystem-wide decision made upfront for the entirety of a given user session, or alternatively whether you are mixing and matching FDC3-enabled app instances and non-FDC3 app instances within the same container during a user session.

Encoding the integration capabilities in the window/element name will work, as long as this is specified in the standard.

This was more of a technique I was suggesting you could potentially explore within your own portal, when I had assumed that portal was the only container your apps ever would run from. I'd be reasonably surprised if something like window.name projection made it into the FDC3 standard though, especially off the back of the current installer lib discussions - because doing so would likely require every DA to support this as part of their fdc3.open() method implementations. And even if this was agreed, it could only be used in future DAs adhering to a future version of the FDC3 spec - which would make it a tough sell to use as part of a standardized installer lib that is intended to be capable of handling current DAs / current versions of the FDC3 spec.

novavi commented 1 year ago

One further thought @api-forever - it's probably worth noting that if your ecosystem's decision on FDC3 interop or no FDC3 interop was made upfront for a user session, there is one other mechanism at your disposal beyond simply url params and window.name projection. You could pass the flag through to your apps by means of a hidden iframe on a common domain inside your app windows (as alluded to earlier in this issue, leveraging @thorsent's SharedWorker or localStorage ideas).

So regardless of whether your app runs in an iframe within the portal's main window or whether it runs in an external window spawned from that main window - your app (or more likely a library within it) could create it's own hidden iframe on a common domain which it could postMessage with in order to read an fdc3Enabled or similar flag. Of course, this approach only works either if you directly control the container (where you could set the flag on the common domain yourself) or if your apps were running within another container which used a standardized library that allowed this level of configurability at runtime (ultimately also setting the flag on the common domain as required).

So for the specific case where the flag only needed to be set upfront for a user session, the use case could potentially be handled by a standardized installer lib (although possibly at some cost). But for anything more ad-hoc than that, I suspect it would be tricky.

thorsent commented 1 year ago

@robmoffat The wire protocol is attractive in its simplicity but I think that it could turn out to be non-trivial. As an example of how this can get complicated, imagine a web page that is connected to a desktop agent. Then imagine that the desktop agent itself is rebooted (perhaps it's running in a different tab, or on a server). The client would have to reconnect (if it could) but then its state would no longer match the server (i.e. context listeners, intent listeners, joined channels).

FDC3 is not prescriptive about how this condition should be handled, which is good because it's complicated! But I think that vendors should be in a position to address these and other issues. Our hands are tied if we're not able to place code into the app's process, which is concerning on a practical and philosophical level.

I'm also concerned about being too prescriptive around the transport. I think there are use cases where we can't rely on the desktop agent code itself living in the browser.

robmoffat commented 1 year ago

Hi @thorsent,

The wire protocol is attractive in its simplicity but I think that it could turn out to be non-trivial.

Scrolling back up, I talked about a common wire format (i.e. a JSON schema), that could be used over different protocols (e.g. web sockets). I think that part is fairly easy. I agree that building a protocol on top of that is more tricky because of the issues you raise (plus others like delivery failures, etc.)

I'm also concerned about being too prescriptive around the transport. I think there are use cases where we can't rely on the desktop agent code itself living in the browser.

Yes, I hope I'm clearing this up with the first part of this comment. If you go back to the diagram I posted in, it showed a couple of different transports in action: Desktop Agent Bridging is clearly standardizing one of them, using web sockets. I believe a postMessage based protocol is what we need to standardise for FDC3 On the Web (WebC3 for short).

I believe WebC3 is only relevant to the "desktop agent code itself living in the browser" use case. Other use cases should do what they need to do.

thorsent commented 1 year ago

@robmoffat I think you're on to something with a stadardized postMessage protocol. It can be baseline functionality, ignoring edge cases such as the reconnect problem I was concerned about. Web based desktop agents would be required to speak the protocol to achieve certification.

But the good part is that, desktop agent suppliers could still optionally supply their own npm modules which could themselves be certified so long as they also spoke the standard protocol. But they would be free to add features or upgraded capabilities. For instance, a Glue42 replacement library could support its own proprietary protocol for speaking with the Glue42 desktop agent, so long as it also spoke the standard protocol when connecting to other agents.

This arrangement would foster competition among desktop agents while still providing app vendors with a WORA guarantee.

novavi commented 1 year ago

@api-forever

The first supported protocol that the app asks for is used. So, there can be a mix of FDC3-compliant and other apps that interoperate. The protocol the app asks for can be set with a URL parameter (if supported by the app), thus such apps can be used without code, config or deployment changes in a variety of UI containers, or standalone/isolated.

It seems to me that trying to do this in a generic way both for apps your don't control and for containers/portals you don't control is likely to be challenging. Whereas perhaps having the apps work with multiple interop protocols when run specifically in your own container/portal - but having them require FDC3 when run in another container/portal - is probably more achievable. Even if you don't directly control all the apps, perhaps you have enough influence over them to get the vendors to use a thin layer on top of a standardized installer lib to handle your required specialized url parameter?

Arguably, trying to standardize runtime interop protocol selection (as opposed to simply FDC3 DA selection) could be difficult/contentious if attempted as part of the FDC3 standard. And for anything beyond the simplest of scenarios, it could rapidly descend (metaphorically speaking) into questions about what should happen for FDC3 usage when a black cat is seen during a full moon - the answer to which is likely to be different for each organisation/ISV delivering interop-enabled apps. I suspect @kriswest and others will have some views as to how your scenario might or might not fit into the picture for a standardized FDC3 installer lib, and if it did fit in then where it would sit alongside the main goal of FDC3 DA selection.

It could be non-mandatory (should vs. must). The DAs that do not do that would require more work for setting up the URLs in the app directory for correct interop. This could be fully automatic instead.

Yes, technically it's possible that window.name projection for fdc3.open() could be made part of the FDC3 standard if it was seen as a sensible way forward and received a majority vote. You'd really have to get a view from those involved with the evolution of the FDC3 standard. That said, I can envisage a few potential problems here:

This is similar to postMessage to an opener/parent. Furthermore, IFRAMEs are resource-intensive, IMO it is best to avoid doubling their number. It would be useful to get the FDC3 enablement flag without slowing down the app UI.

Yes similar, but hidden iframes provide more overall comms options than opener/parent windows - not least for the case of apps that can be launched both from a container and independently of a container.

You do raise an interesting point about the use of hidden iframes doubling the number of window objects in this type of FDC3 app ecosystem though. This is definitely worth bearing in mind when testing any solution which relied on hidden iframes (even if the code running in those iframes was minimal). And in the context of its potential use by an installer lib, it's probably fair to say this should also be borne in mind when considering whether a hidden iframe approach should be an optional pattern or a mandatory pattern. I assume your concern is specifically related to the act of iframe inclusion within app windows, rather than the total number of browser windows the user has open (which would generally be beyond an ISV's or an app's control).

novavi commented 1 year ago

Hi @robmoffat @thorsent

I think you're on to something with a stadardized postMessage protocol. It can be baseline functionality, ignoring edge cases such as the reconnect problem I was concerned about.

I like this idea a lot. Specifying a standardized JSON schema wire format and decoupling this from DA implementation details like reboot / reconnect / delivery failures / proprietary protocols makes a lot of sense to me. Additionally, it would support bridging implementations that could include not only the currently-envisaged local WebSockets service but potentially also hidden iframe local browser messaging (e.g. in the case where a bridge needed to span two different browser-based MFE container ecosystems but where no desktop container was involved).

The wire protocol is attractive in its simplicity but I think that it could turn out to be non-trivial.

@thorsent - related to this issue of simplicity that could be non-trivial in practice, I've had similar thoughts about another area - the hidden iframe postMessage <-> Broadcast Channel <-> postMessage DA installation idea we discussed previously. You might well already have worked through these things (and beyond) during the course of a POC or other work and/or have been thinking about this in a different way from me in any case - in which case, feel free to ignore this. But I will list my thoughts below in case they are of interest.

Creating a cross-window cross-origin proxy for an API with async methods returning object literals seems fairly trivial on the face of it. And of course there are many different implementation patterns ranging from compile-time code-gen proxy classes, to (at some cost) runtime typescript-rtti interface-based code-gen proxy classes, to a range of other options including runtime postMessage-based DA class instance introspection / proxy class construction.

However, once you have to go beyond object literal return values then I suspect there's likely more complexity - two examples of this would be:

Unless I'm missing something this implies any cross-window cross-origin DA proxying solution would need to not only proxy the methods on the DesktopAgent interface, but it would need to also proxy every return object value from some of the DA methods (for everything beyond those methods whose return values which are simply object literals). This is more involved than the potential need I cited previously to create reverse-proxy functions in the case where an app needed to provide callback function args to a DA implementation running in a hidden iframe (because there would be a finite number of those args, and they would be known upfront at installer lib configuration time).

Of course this is a solvable problem, but it would seem to add a degree of complexity if implemented as part of an installer lib. To be fair, it's probably more achievable to implement this type of postMessage-based method return value object proxying in a robust manner now that ES2021 has given us FinalizationRegistry - because this would allow a cleanup callback on proxy object reclamation in the app window in order to consequentially postMessage into the hidden iframe to release the real objects). But I do wonder whether the hidden iframe approach is best used as:

In particular, one thing that stands out to me is that if the hidden iframe approach was used by as a DA implementation pattern then the DA would have the following options available to it:

My point is that there isn't necessarily a single solution or right way for a DA to leverage a hidden iframe window. Different DA vendors might want to make different choices and different trade-offs - and for some vendors this might even involve consideration of aspects beyond the FDC3 standard itself (e.g. how well a given approach fitted in with other parts of their ecosystem, which could involve a browser-based MFE container or potentially other services). Additionally, even if there was a single optimal approach for how much of the DA code to run in each window today, the assumptions that decision was predicated upon might change over time e.g. based on DA functionality required in future versions of the FDC3 standard.

Another point that should probably be borne in mind is that unless you make the proxying fully dynamic to handle any DA at runtime (along the lines I mentioned above using object introspection or equivalent) then an installer lib which used a cross-window cross-origin proxy would either:

Anyway, I would concede that haven't thought this through fully - but based on my initial thoughts I just thought I'd raise the question of how much a hidden iframe approach could/should be a core part of an installer lib (with potential implications on the DAs it could support) or how much it might be more of an DA implementation pattern. It's probably the case that a POC would tease out more issues than just the ones I thought of above.

novavi commented 1 year ago

One small addendum to the issue of proxying - support for DAs with opt-in non-standard behaviors.

Specifically, the implications for this if a generic cross-frame proxy was implemented in an installer lib (rather than as part of a FDC3 DA itself). One example of this might be support for apps to join multiple channels, which I believe one of the desktop container's FDC3 DA already supports. Other cases might possibly present more complexity still, and I would have to assume that this type of opt-in DA behaviour could be a feature going forward for web-based DAs as well as for desktop container DAs. I mean because as long as the core behaviour under default config satisfied the FDC3 standard, individual DAs might choose to innovate in some areas (like the multiple channel example) in advance of the standard's ability to catch up. In this scenario, the last thing an ISV would want is an installer lib which either failed to expose a DA's complete range of functionality properly, or failed to make it clear in which scenarios the installer lib can and cannot be used.

Is it possible that opt-in config settings for specific DA implementations could perhaps break some of the assumptions made by an installer lib's generic proxy about the runtime behaviour of some of the DesktopAgent interface methods? To be clear, I'm not saying this is definitely the case. And you could also make the argument that any app making use of non-standard behaviours might weaken the case for that specific app using an installer lib in the first place (it might be case-by-case because it would likely depend on what the specific non-standard behaviours were, and whether the app was capable of gracefully degrading in the absence of those behaviours). But in summary I think that - alongside the issue of an installer lib supporting multiple versions of the FDC3 standard - the question of opt-in non-standard behaviors should probably also be borne in mind when considering installer lib implementation options in this area.

kriswest commented 1 year ago

@novavi we've done a limited amount of work on exposing optional feature flags to apps using a desktop agent, via the fdc3.getInfo() call that returns ImplementationMetadata. See: https://fdc3.finos.org/docs/api/ref/Metadata#implementationmetadata

At present, multiple channels is not included, despite it being used in production by a couple of large institutions. The relevant issue #242 remains undecided - and TBH is incomplete. I'm aware that both institutions (that I know of) not only use multi-channel but allow (or wish to allow) users to join apps to channels in one direction (as opposed to both). I believe an update issue will be raised for us to consider this in due course. I expect that it will remain an optional feature and gain an optionalFeatures flag.

Non-standard features can give rise to issues in conformance tests. Finsemble, for example, supports multiple channel membership but has to allow it to be disabled to reproduce expected behaviour in one or two of the conformance tests (i.e. when you join a second channel you leave the first). This may remain an issue if this becomes an optional feature, as a DA should probably be able to demonstrate that it can produce the default expected behavior if the option is disabled.

In that particular case, the non-standard behaviour is rarely an issue for a user (they can see what channels the app is on). Other cases may be less simple, where they cause issues for application code. Hence, you are right to raise this for discussion!

If we're looking at ways that an app vendr might limit or choose the DAs they work with, should they be able to decalre that they will accept any certified conformant agent?

novavi commented 1 year ago

Thanks @kriswest for the information and suggestions.

I agree that particular non-standard behaviour likely doesn't present an issue. In terms of non-standard behaviours more generally, optionalFeatures seems a good way for an app to test the functionality of the DA it's using at runtime. One point I wonder about though is that under-discussion-but-not-yet-standardized FDC3 behaviours are unlikely to be pre-assigned optionalFeatures flags in the @finos/fdc3 ImplementationMetadata interface in advance of any DA implementing them (as is the case for the multiple channels example you cited). So does this mean that a DA such as Finsemble would likely only provide this flag via the fdc3.getInfo() call after the ImplementationMetadata interface has been updated in a future published version of the @finos/fdc3 package? Or is there a case for ImplementationMetadata also supporting some sort of [key: string]: boolean or similar (albeit at the cost of losing strong typing) to allow DAs to expose flags even in advance of those flags becoming well-known enough to be defined in the optionalFeatures object?

Whatever is done in terms of optionalFeatures for this type of scenario to make life easier for app developers using a directly installed DA, it does seem that there are interesting implications for any proxying undertaken outside the scope of a DA implementation.

In terms of app vendors limiting or choosing the DAs they work with, I assume you mean when using an installer library and specifying this in a web manifest or config? One of the things I picked out in the Outstanding Work for the fdc3-installer POC I did previously was:

At the time I was sort of leaning towards semver, which I guess would give slightly more control than could be achieved using @finos/fdc3's versionIsAtLeast() - and would also allow this to be specified at installer config/manifest level rather than only within an app's code.

Your idea of allowing apps to declare they accept any certified conformant agent is an interesting one. This gives rise to couple of questions:

kriswest commented 1 year ago

To provide some structure for today's FDC3 for Web Browser meeting, we've prepared the following proposal (borrowing heavily from the discussion and proposals above, credit to @novavi @robmoffat and others).

Proposal Goals

Key goals for the proposal should include:

Use cases (ways a Desktop Agent might be used)

An installer library will need to allow for several different formats of FDC3-enabled environments built with web technologies. Terminology for these is defined below:

  1. Container or Browser Extension: A web app running in the context of a container or browser extension that provides an implementation of the FDC3 API (the traditional FDC3 use-case).
  2. Web container (Window.open / window.opener): A web app running in a window opened by another window with a Desktop Agent running in it or otherwise associated with it.
  3. Web Container (iframe): A web app running in an iframe inside another window with a Desktop Agent associated with it
  4. Independent Web App: A web app running in an independent window, not opened or embedded in any other window.
  5. Micro-frontend Container (Single DOM): Multiple separate 'apps' running in the same DOM, that could communicate with each other and other windows through the desktop agent.
    • N.B. 2, 3 and 5 could all be referred to as a Microfrontend Container - as they are related to a root application or container that could be in charge of providing or indicating which Desktop Agent should be used. However, the overlap in terminology will be problematic and we need to agree concrete names for each case. I recommend Web container for the window.open and iframe cases to help disambiguate.

We may also wish to update the terminology for the API instance itself, for example to introduce an alternative term for "Desktop Agent", e.g.:.

"Web agent" - The equivalent to a desktop agent, but which is accessible within a web container. For instance, a given FDC3 compatible desktop agent might also provide an equivalent web agent.

Alternatively, we a more generic term may be useful, e.g.: "FDC3 Agent".

Note: changes to terminology/naming do not significantly affect javascript code compiled from Typescript sources (the content rather than the name of the type matters). However, the impact on the TypeScript sources and impleentations in other languages is more significant and should perhaps be avoided.

Meta Proposal

In this section, an outline of a proposal to support the use cases defined above is provided, as a number of steps that should be followed in sequence. Each step will require selection of a technology option to implement it; proposals for technology options are provided in later a section.

Packaging

An installer function will be made available in the "@finos/fdc3" library. Applications will be able to obtain an fdc3 object from the installer function (as an alternative to using the global fdc3 object directly). The resolution of the fdc3 object will be asynchronous. The use of the installer function will introduce a new preferred mechanism for obtaining the fdc3 object (but will continue to support the global fdc3 object when available via a container - see below).

import { installer } from "@finos/fdc3"

const fdc3 = await installer(); // New standard methodology for accessing fdc3

// the `window.fdc3` global may be also set by an app for backward compatibility
//  Note this may have already been set by a container or browser extension
window.fdc3 = fdc3;

[Edited 20/03/23 to show an app setting the global itself, rather than the installer doing it]

Apps should no longer wait on the "fdc3Ready" event (though it will continue to be used internally - see below).

Note: This is not a breaking change. Existing FDC3 apps can continue to work within containers using the same global fdc3 object and same fdc3Ready event. To work in multiple agent types they will need to upgrade to the installer instead.

The installer library must address different strategies for establishing which FDC3 Desktop Agents (of different types) are available, before returning the fdc3 object.

Step 1: Container preloads and Browser extensions

Establish whether the web application is running in the context of a container or browser extension that will provide an implementation of the FDC3 Desktop Agent. Resolve to the instance of that implementation provided.

Step 2: "Web Containers"

Establish whether the web application was launched into a window or loaded into an iframe by a 'Web Container', which provides details of an FDC3 implementation to bootstrap or otherwise connect to. Resolve to an instance of a client to that implementation.

Step 3: Independent web pages

Establish that the web application is an independent web page (loaded directly rather than launched within a container or by a Web container) and, hence, no information is provided externally on what FDC3 implementation to use. Load or defer to a library, identified by configuration provided by the application itself, directly imported or otherwise connected to.

Step 4: Micro Frontend Container instance sharding

If the web application contains multiple independent 'micro-frontend' or 'widgets' that wish to communicate with each other, and potentially other applications, then a method of 'sharding' or creating sub-instances of the Desktop Agent interface, itself retrieved in any of the previous steps, should be provided. This function should ensure that each instance is assigned a unique appId (perhaps derived from the parent appId) and that messages are routed between those sub-instances as they are between applications running in separate DOMs (broadcasts, raised intents etc. are not normally resolved or delivered back to the instance that raised them.

Technology options

Step 1: Container preloads and Browser extensions

Option 1: Race

Current implementations of FDC3 rely on an fdc3 object to be made available globally by a container, most often using a preload (example container technologies include Electron, browser extensions, and webviews). The new installer function must detect when a global object is available. If it is available, then it will resolve with the global object.

This process is complicated by the current spec which does not require that the global object be made available immediately.

If the global object is not immediately available, then installer implementations must wait for an "fdc3Ready" event to be triggered.

The following algorithm will therefore be used to resolve the fdc3 object:

Option 2: Require preload

Containers generally rely on preload scripts to deliver a copy of the FDC3 API as a global (window.fdc3) - this is currently the dominant use case for FDC3. It is important not to break existing app implementations (where possible), to avoid requiring a new major version of FDC3 or causing issues for firms that have already adopted FDC3.

The FDC3 Standard currently states that:

The global window.fdc3 must only be available after the API is ready to use. To enable applications to avoid using the API before it is ready, implementors MUST provide a global fdc3Ready event that is fired when the API is ready for use. Implementations should first check for the existence of the FDC3 API and add a listener for this event if it is not found.

https://fdc3.finos.org/docs/api/spec#api-access

This bakes in a possible delay before the API is ready and hence may cause issues for the detection of a container preload in an installer library - leading to slower bootstrapping and race conditions/conflicts.

However, as 'preload' scripts can run before page content, they should be able to set the window.fdc3 global before an installer needs to get going. Hence, a modification can be made to the above advice. I.e.:

Containers using preload scripts to setup the FDC3 API should set the global window.fdc3 as soon as possible after start-up, to facilitate detection of the implementation by apps and other FDC3 libraries, and MUST fire the fdc3Ready event as soon as they have done so. As the FDC3 API is asynchronous, such implementations MUST use queuing if they are not immediately ready to handle API calls, allowing them to respond when they are ready._

Other types of FDC3 Desktop Agent implementation MUST only provide the window.fdc3 global when it is ready to use and MUST provide a global fdc3Ready event that is fired when the API is ready for use. _

<advice for app implementation removed and should be replaced with advice on using the installer>

Step 2: "Web Containers"

Option 1: Specified client library bootstrap

A web container specifies to an applicaiton it launches which FDC3 implementation client library to load and bootstrap.

The installer function will use the bootstrapping rules to determine the url location of a "FDC3 web agent client library" (client library). The client library must be an ES6 module that exports a function called bootstrap() which resolves with an fdc3 object conforming to the DesktopAgent interface. The installer function will dynamically load the client library, run the bootstrap function and resolve with the fdc3 object.

The installer function should attempt to retrieve the app's web manifest. If a manifest link can be found in the current page, then that location will be retrieved, defaulting to manifest.json in the same folder as the app's main page. The web manifest will be expected to contain an "fdc3" section which may provide bootstrapping rules to the installer function.

Bootstrapping rules will be an ordered array contained within the fdc3 object in the web manifest. Each rule is associated with a supported discovery mechanism (additional discovery mechanisms can be added as needed by maintainers). If no bootstrapping rules are provided, then the discovery mechanisms should be used in an unrestricted manner to attempt to load a client library specified by a web container.

This example provides two discovery mechanisms. The "queryString" discovery mechanism looks for a query string field (proposed field "fdc3_web_agent") and, if it's available, uses that field's value to resolve the client library url. The "url" discovery mechanism simply resolves to the specified url as a fallback.

{
   fdc3: [
      { queryString : {
             "vendorX" : "[https://vendorX.com/etc.js](https://glue42.com/etc.js)",
             "vendorY" : "https://vendorY.com/etc.js"
         }
      },
      { url: "[https://etc.etc.etc/etc.js](https://etc.etc.etc/etc.js) }
   ]
}

Further discovery mechanisms might include:

Certification requirement for client libraries:

Pros:

Cons:

Option 2: App to agent wire protocol

The installer function will provide its own fdc3 object (an implementation of the DesktopAgent interface) which will act as a thin proxy, communicating to web agents via a standardized wire protocol. (Vendors may provide alternative client libraries - see below).

postMessage will be used as the transport because it is the only guaranteed cross-origin transport available in browsers. Location of the web agent is potentially complicated because the web agent may be separated by any number of intermediary parent frames (window.parent) or windows (window.opener). Reliably passing data across this chain is potentially complicated by the fact that the intermediary windows may be cross origin and may not have included the installer.

This proposal suggests that the installer should set up a race, simultaneously posting a handshake message to several possible locations to discover a web agent. The first location to respond becomes the agent of choice. Those locations would be:

  1. window.opener - if it exists
  2. window.parent - if it exists
  3. window.top - if it exists and is not the same as window.parent or the window itself

A compliant web agent would only need to support establishing connectivity with windows or frames that communicate back to it with postMessage, and may ignore requests from windows that it did not itself open (by comparing message origins, enclosed appIds and a list of origins for windows it has launched and have yet to close).

The wire protocol itself will be aligned as closely as possible with the proposed Desktop Agent Bridging wire protocol and FDC3 Desktop Agent API functionnames, parameters and return types.

Option 2b: App to agent wire protocol over a common domain

A Hidden iframe may also be used provide more reliable or secure connectivity (by passing all communication to/from an iframe on a predetermined domain), at the cost of a more complex implementation. This would also require a production dependency on that domain (e.g. a subdomain of finos.org) for all implementations for hosting the relay code. However, the content itself would be static and can be backed by CDN and aggressive caching.

This approach has the advantage that an agent may register itself with the relay as available, facilitating connection without specification from a Web container. This feature will be particularly useful in step 3.

Proprietary client libraries

To support alternative methods of connectivity, vendors may provide their own client libraries. A client library may then be certified if it supports the mechanisms and other requirements adopted for an installer. It may also then support proprietary mechanisms the standardized ones, so long as the standard mechanisms are implemented with the priority specified in this proposal when deployed in a competing web agent. A client library may extend the FDC3 desktop agent so long as it passes the certification suite.

Certification requirement for client libraries:

Pros

Cons

Step 3: Independent web pages

Independent pages are those which are those which are not directly opened by a web agent. An independent page's discovery of its web agent depends on the technology options outlined in step 2 above. Neither of these approaches compromise write once run anywhere (WORA)

Option 1: (unspecified) Client library bootstrap

The same approach as defined in Step 2 option 1 is used to load client library providing an implementation, but without reference to outside specification of which library to use. Hence, a fallback option must be provided by the application's configuration in its webmanifest and/or the implementation of the installer library.

A web agent is free to implement any connectivity mechanism in their client library. They would need to convince app deployers to make their client library. They need only convince app maintainers to point an instance of their app to a web manifest that points to their client library. For instance, an app developer could dynamically assign the web manifest location in the app's code, or an app deployer might deploy the same app in a different folder with a different web manifest

Option 2: App to agent wire protocol

Vendors are free to provide proprietary client libraries and convince app developers to choose their library. To be certified, the client library must interoperate with other desktop agents according to the standards defined in Step 2.

Option 2b: App to agent wire protocol over a common domain

By implementing the App to agent wire protocol communication over a predetermined domain, a Desktop Agent running in another window/DOM can register itself with a relay running on that domain, which itself can use a sharedWorker (singleton) to enable communication between instances of the iframe in different windows. This will enable resolution to a running agent without reference to specified agent library or a single default client library.

However, in the event that multiple agents are registered, some form of resolution mechanism (e.g. a UI) will need to be exposed to the user (which can be filtered via config in the web manifest). This may prove a better option than an app specified default.

Step 4: Micro Frontend Container instance sharding

Note: Sharding should probably be considered via a separate proposal. There are potentially deep implications. However, it's being placed in this proposal because it could have implications for a standardized installer, particularly that it adds further complication to the implementation of a DesktopAgent, particularly vis a vis standardizing a wire protocol.

After receiving an fdc3 object from the installer, apps should be able to create new instances of fdc3 objects which are unique to the desktop (or web) agent. This approach can be used to support micro-frontend architectures and other use cases where multiple instances of fdc3 are required within the same JavaScript process.

fdc3.createShard(subAppId: string, subAppDRecord ?: AppDRecord): Promise<DesktopAgent>

Shards should behave as if they are different windows, i.e. they can be used for context messaging and raised intents within the window. Each shard should be treated by the desktop agent as an individual instance of the parent application, associating it with the parent's AppD record. A shard can optionally provide its own AppD record. Desktop agents may choose to accept or ignore that record (e.g. for security purposes).

robmoffat commented 1 year ago

Couple more options for you:

Step 1/Option 3: Override

// the window.fdc3 global is also set by either a container or the installer for backward compatibility

I think it's a mistake for the installer to set window.fdc3. For four reasons.

  1. If you are using the installer, then you are bought into the new way of using FDC3 at least within the scope of this piece of javascript. There's no need for backwards compatibility.
  2. By doing this, it prevents you from being able to instantiate the desktop agent multiple times within the same window, as otherwise they're both setting the global and you won't know which one you'll end up with.
  3. You're actually reducing backwards compatibility, because containers will still be injecting into window.fdc3 and people aren't necessarily getting the same instance as they would have before the introduction of the installer.
  4. You're introducing a race condition around who is setting window.fdc3 which otherwise wouldn't exist (probably the biggest problem).

Instead, the installer should act as a proxy where window.fdc3 is available and set.

Since this specifically avoids a race condition, you could call this Step 1/Option 3: Override

Step 2/Option 2/Proprietary Client Libraries 2

Instead of having the proprietary client library specified in the app (making each app contain proprietary code 😭 ), after handshaking the desktop agent could give the app the details of some javascript library to load and include (perhaps conforming to some api).

This would potentially allow apps to be "branded" (a la connectifi) with a logo / channel information related to the desktop agent they are running on.

nkolba commented 1 year ago

Here is the project mentioned earlier today. Now open source. https://github.com/connectifi-co/fdc3-web-portal

This takes a stab at:

@robmoffat @kriswest @thorsent @novavi @hampshan would be great to get your feedback.

kriswest commented 1 year ago

@robmoffat

I think it's a mistake for the installer to set window.fdc3. For four reasons.

  1. If you are using the installer, then you are bought into the new way of using FDC3 at least within the scope of this piece of javascript. There's no need for backwards compatibility.

That depends on the availability of developer time to update. Where it's limited and there are many apps to update, the lightest touch is often preferred. I base this on experience of doing these updates with multiple institutional clients with large inventories over previous versions of FDC3.

  1. By doing this, it prevents you from being able to instantiate the desktop agent multiple times within the same window, as otherwise they're both setting the global and you won't know which one you'll end up with.

This isn't currently a use case for FDC3 - even in the microfrontend use case (as proposed) there is a single parent Desktop Agent with others sharded from it. In fact, it's potential enabler in that case as it means it doesn't have to passed down to each widget when created, rather they can reach for it and request their own instance from it (as proposed).

Further, there is, at present, a single appId, relating to a single Desktop Agent instance at present meaning multiple instantiations within a window doesn't make sense currently. Happy to discuss if there are use cases, but we've specifically touch on this in multiple meetings now (I've explicitly asked if there were use cases) and so far the answer has been no, apart from the microfrontend use case (where the shared are derived from a single instance).

  1. You're actually reducing backwards compatibility, because containers will still be injecting into window.fdc3 and people aren't necessarily getting the same instance as they would have before the introduction of the installer.

Thats not the case give any of the options for step 1 so far. If it gets set that is the instance that the installer resolves to - granted it wouldn't make sense to set it in this case as it's already set (but even it did that would effectively be a no-op).

  1. You're introducing a race condition around who is setting window.fdc3 which otherwise wouldn't exist (probably the biggest problem).

No we're not as that race has to be specifically dealt with in step 1 (in any of the proposed options).

All that being said, I don't think it matters whether the installer sets it (for steps 2+), the installer sets it if a config option is set or an additional line of code sets it after the installer returns. I do see a use-case for it being set - particularly during migration.

Re: Step 2/Option 2/Proprietary Client Libraries 2: I don't fully understand the suggestion - loading some javascript lib is basically option 1, negating the purpose of option 2? That said I would tend to agree on preferring the use of an independent OS library. However, it was pointed out to me that IFF proprietary libs conform the defined steps (and can be tested for conformance to them!) many of the problems go away. Steps 1 and 2 in particular need to be closely adhered to. Step 3 is where you might be most likely to see problems or deviation as a vendor is likely to prefer their own version when there are multiple to choose from - where in an OS version we might offer the choice to the user (if more than one is already running)... something to think on further.

branding - the fdc3.getInfo() function can provide some details on the DA connected to. It could be extended to include a logo URL and other data easily enough.