Duplicated add-ins from store and sideloading even though it's the same add-in, "poisoning" files

wh1t3cAt1k commented 1 year ago

Your Environment

Platform [PC desktop, Mac, iOS, Office on the web]: Office on the web
Host [Excel, Word, PowerPoint, etc.]: Excel
Office version number: latest
Operating System: Win 11
Browser (if using Office on the web): Edge

Steps to reproduce

Clear all browser cache for all time and re-login
Create a new Excel workbook
Insert a new add-in from AppSource store called Velixo

Download another manifest from https://nx-master.velixo.com/addin/manifest.xml
Insert -> Upload my add-in that manifest for sideloading

Actual behaviour

Two add-ins are displayed in the file, which is very confusing.
Effectively only one of them registers custom functions, but otherwise they are in conflict with each other.
File is "poisoned" with the dev add-in for all other users, especially if it's shared one, produces inconveniences and support requests.
When someone else without the dev add-in opens the file, they will see "this add-in is no longer available" error for one of the add-ins.
Overall, very poor user and dev experience with this strange approach.

This may be related to https://github.com/OfficeDev/office-js/issues/2776 - in that case, it was not a development add-in conflicting with store add-in, but it was an org add-in conflicting with the store add-in.

Expected behavior

We'd really like the add-in to be recognized as the same instead of registering multiple times in the workbook.

In addition, we would like clear product rules around the priority of such recognition.

Proposal:

For example, "sideloaded" has priority Top, "admin managed" has priority Medium, and "store installed" has priority Low. In particular if the user has a sideloaded dev add-in registered in his machine, this version would be preferred to the store / admin managed version from the document.
At the same time, if someone with a sideloaded dev add-in opens the document that uses a production add-in from the store, we want it not to "poison" the file with the dev version. We want our clients (who share workbooks with us to troubleshoot) and the corporate exec team not to see pesky notifications about "this development add-in is no longer available".

I am not sure if any changes need to be made to the registration mechanism of the add-ins in the document, maybe they do - but we clearly want only one add-in to be loaded in these scenarios.

I don't have a clear architectural recipe for the fix, what I can only say is that the current mechanics look broken for our clients and warrant a change for better UX.

Context

This problem negatively affects the customer perception of the modern add-in framework and our new cross-platform offering.

ghost commented 1 year ago

Thank you for letting us know about this issue. We will take a look shortly. Thanks.

gmichaud commented 1 year ago

I’ve been affected by this problem so many times that I now completely avoid sideloading dev versions on my main machine and use a separate set of workbooks.

RuizhiSunMS commented 1 year ago

Does it belong to your area? @AkhileshShah-MS

4tti commented 1 year ago

Our customers are affected by this as well. The issue is not only with side-loading but in combining multiple deployment methods. It will happen with centralized deployment, shared folder as well as store... and then nightmare with the custom functions registration will appear.

This is extremely painful... and for various reasons customers start with one deployment and later they will decide to use different one... and instead of transitioning smoothly there are problems which can be later resolved only with significant effort.

However, I am not sure about the approach dev/admin/store prioritization; I'd prefer rather some better approach that will user ideally allow to pick which one is preferred and so eventually giving possibility to switch between various versions, like e.g. test vs production environment in On Premise scenarios - without having to always undergo painful ritual of cleaning cache/inspect document, etc.

RuizhiSunMS commented 1 year ago

Hi @wh1t3cAt1k, sorry for the late reply. After some investigation, we find this issue is appsource related(not owned by us). Here are some explanations from appsource team guys (and rephrase by my understanding): From the base design, expect Office regards two add-ins as the same is not supported. There is one id property for the application to identify same or different add-ins. If the add-in is on the Appsource, then the id is generated by the Appsource. I f the add-in is central deployed, the id will be generated either. However, if the add-in is installed by the manifest, there is no such id. The application can't judge two add-ins as the same only by ui or guid. Wish this explanation could help.

gmichaud commented 1 year ago

Thanks @RuizhiSunMS for the follow-up. This looks to be a core design flaw with how deployment of add-ins is managed. I understand the technical challenge, but would like to point out that end-users are generally not nice and will simply complain that "things are not working".

Here's another example of the "poisoning" behaviour that @wh1t3cAt1k described here. I don't know if it has anything to do with sideloading or Appsource yet -- it's a file I had on my machine and that I had not used in a while.

Screen recording: https://www.loom.com/share/531f433484a848cba805551b2df1b163

I wanted to remove and re-add the add-in from this file because I was seeing unexplained issues, and now I can't get out of this loop. It even crashes Excel at some point, and I can reliably replicate this by opening the file again.

Whenever that happens I just open the .xlsx file to look at the internal XML to understand what's going on. We can't expect users to have to do that. In this particular case, my workbook contained two references to our addin.

Reference 1:

Reference 2: Affected file can be downloaded [here](https://velixo-my.sharepoint.com/:u:/p/gabriel/EY8h29p0w8ZFqq3SfeVdvvUBnhFY6dzz5p0ygMbuiTgTVQ?e=QqvLdH) in case you want to take a look - I zipped to make sure it wouldn't open in Excel Online. NOTE: I was only able to get out of this by [clearing the Office cache manually](https://learn.microsoft.com/en-us/office/dev/add-ins/testing/clear-cache)

wh1t3cAt1k commented 1 year ago

Probably this is what I have minimal repro for in the related issue: https://github.com/OfficeDev/office-js/issues/2776#issue-1340252421

These two problems may be linked.

I feel like it's a very serious problem that is deserving of a much higher priority on Microsoft's end, both these issues have remained unaddressed for a long time.

4tti commented 1 year ago

.... If the add-in is on the Appsource, then the id is generated by the Appsource. I f the add-in is central deployed, the id will be generated either. However, if the add-in is installed by the manifest, there is no such id. The application can't judge two add-ins as the same only by ui or guid.

Just to be sure what ID are you referring? Cause manifest contains ID but I guess you are talking about "we:reference id" and not about the manifest one.

Nevertheless to me this seems to be critical design flaw. Adopters should be able to control this behavior and if they wish, they should have a way to tell to Office Apps that "hey, this particular Addin is same" and eventually even have options how to resolve it... or ideally allow users to switch between the addins (versions) as it would unblock some usecases.

wh1t3cAt1k commented 1 year ago

Just to reiterate the key invariants of a potentially better architectural solution, as I see them:

When a developer has a sideloaded add-in installed, it is safe to assume that they want to use it in favour of the store version. It might also be reasonable to assume that admin-managed channel may take precedence over individual AppSource installations. This naturally produces a priority chain sideloaded > admin managed > store installed.
Our customers get confused and don't want to see warnings about sideloaded add-ins in their files after a developer / member of support team edits the file.

One potential solution would be to allow "logical groups" of add-ins with the same ID; in other words, allowing multiple registrations of the same add-in in the file - but handling it more reasonably than now by picking only one according to the priority rules.

If any link in the priority chain is missing (e.g. the user does not have the sideloaded version), we should not display the warning about the add-in missing - and simply gracefully take the next available channel.

It is also important to highlight that storage should be shared in the logical group of add-ins (e.g. OfficeRuntimeStorage)

4tti commented 1 year ago

@wh1t3cAt1k but this won't help with the 'prod vs test environment for OnPrem solutions' scenario, cause usually same approach of 'installation' is used (e.g. shared folder/centralized deployment). The proper solution should be that these logical groups ALLOW user to pick what they want to use (at least when they have permission to do so), i.e. I can imagine that actually there is 'Available' or 'Installed' group under the addins which could allow to pick version to be used (if accessible):

But still, my feeling is that the manifest approach (with all the things being 'static') is the root of all evil :)

Btw. yesterday I had a escalation call with the customer and they encountered even more weird file poisoning - for whatever reason there are 2 buttons of our addin pointing to same version/location... while investigating the file I did not notice anything wrong (single webextension file with single reference, etc.) - seems really weird. The customer is on current channel.... I'll investigate more.

wh1t3cAt1k commented 1 year ago

I agree with @4tti that having an option to choose between "channels" within a logical group would be a cleaner way to handle this.

The key invariant even in this structure for us, though, would be that the "unavailable" channel, such as the sideloaded add-in, should not produce any warnings in case other channels are available.

@ElizabethSamuel-MSFT @jakobpn @RuizhiSunMS copying you on this... I see that a "discussion" label was added here, but I would like to reiterate that it's the opposite of idle talk 😅 this here is probably one of the biggest pain points that we have with our clients, and a major generator of support requests just like @4tti mentioned above.

IMO it calls for prompt action.

We sometimes have a hard time persuading people to buy a product that, every now and then, requires one to open the file on Excel for Windows (!) and use Inspector Alakazam to clean up before reinstalling the add-in.

In contrast, our legacy XLL add-in with dynamic registration just works out of the box with none of the pain.

P.S. Regarding two completely identical web extensions recorded in the file, which @4tti mentioned in his latest post, I think it's reasonable to expect Excel to "distinct" away any completely duplicated instances upon file opening.

ElizabethSamuel-MSFT commented 1 year ago

Adding @JuaneloJuanelo to see if this can be raised with the AppSource folks.

4tti commented 1 year ago

Hello all,

Is there any update please? Thanks in advance!

Wenjun-Gong commented 1 year ago

Hi @wh1t3cAt1k, @4tti, and @gmichaud, thanks for sharing the details of the issues and your thinking on the end user experience. I would like to add a few comments:

If the manifests of the two add-in instances come from different catalogs (such as App Source v.s. sideloading), we take those as different add-ins from security perspective. For an example, for manifest sideloading or manifest central deployment, or manifest from SP catalog, the end user / admin is able to make changes to the manifest by their own and we are not able to make sure they are identical to the one downloaded from Microsoft App Source, although they share the same add-in names and same CF names.
Well received the feedback for the painpoints of the end users when they run into this "add-in not available" situation. I am wondering how frequently this problem is happening to your customers, with the scenarios you described (such as got a file from your support team which was dirtied by the developers, the different deployment appoaches were adopted for the same user - we'd like to understand what are the deployment approaches and why customers would choose multiple ones)?
If there is a magic button allows users to remove the add-in from the file when they see the "loading failed" message, do you think this will address this issue?

Hope this helps and appreciate if you can share more information for the 2nd and 3rd bullets.

wh1t3cAt1k commented 1 year ago

Hi @Wenjun-Gong , I understand the concern for security, but I do think the user should be able to take control and say: "hey, I know what I am doing, this is definitely the same add-in, we just changed the deployment model. Please use this from now on".

From this perspective, a control enabling smooth transition between "store types" is much better than the hypothetical "magical button" that would simply remove the add-in from the document. Removing the add-in is a hassle because all workbook-scoped state from web extension XML is lost.

In our case, losing this state can change the per-workbook behaviour of the add-in, which generates support requests. It would be suboptimal UX as the user needs to reconfigure everything after reinstallation.

As for the example of problems that may occur, also see https://github.com/OfficeDev/office-js/issues/3004 . This is yet another example of issues caused by what we consider a flawed deployment model.

There, it looks like the customer used admin-based development. They shared the file with an external party (in this case, us) - upon opening the file, we receive message saying there's a new add-in to install but it fails to load. Error message displayed is not helpful, clicking Retry obviously will not work, and "See Details" just sends us to the store home page. Complete blocker.

If, there, we could just say, OK, switch to this Velixo add-in that comes from the store (or developer sideloading!) instead, it would solve all our problems. And the customer would be able to do exactly the same on their end whenever a developer add-in accidentally "poisons" their file.

The important point here is the ability to switch sources, not "remove and reinstall".

Wenjun-Gong commented 1 year ago

Hi Pavel,

We recommend our partners only use the sideloading as part of the development or testing process. To distribute your add-in publicly to the users, we recommend methods of AppSource or central deployment through M365 admin center. More details can be found here: https://learn.microsoft.com/en-us/office/dev/add-ins/publish/publish

So, if the issue your customer reported is caused by sideloading, we suggest they stop using this method. Instead, they should use your add-in published in AppSource or central deploy your AppSource add-in.

If the issue has nothing todo with sideloading, we should look deep into the root reason.

wh1t3cAt1k commented 1 year ago

@Wenjun-Gong I agree that the clients should not use side-loading, but poisoning happens regardless of the distribution method, and with sideloading / catalogue insertion, it's often accidental - please consult @4tti 's considerations above.

Wenjun, based on the discussions that we had on a recent call, it felt like you had some action plan in mind regarding this issue? Perhaps something is already cooking?..

Another sanitized sample with two instances of Velixo add-in:

poisoning-repro.xlsx

@gmichaud observed that all formulas stayed in #BUSY indefinitely on this file, on Safari in Excel Online. I couldn't replicate, but I suspect the problem is related to poisoning, maybe the functions bound to the incorrect add-in? (unzip the XLSX - maybe Office picked webextension2.xml instead of webextension1.xml? Does the structure of webextension1.xml look OK to you?)

wh1t3cAt1k commented 1 year ago

@Wenjun-Gong @jakobpn @4tti we just ran again into an ugly situation with a client where no sideloading occurred at all, but it took a couple of man-hours to resolve. It happens with different clients at some point of time.

What happened is either of the two things:

either they always used centralised deployment but someone sent them the file where a store-based add-in was registered.
or they simply have changed from store-based deployment to centralised deployment at some point of time.

The result is that they had two add-in buttons on the ribbon, two task panes, two contextual tabs with the same name. Completely identical, unable to distinguish visually.

Now, we have custom functions that collect some data and then you can upload it using a button from the task pane.

So when the user invoked those custom functions, they were invoked in add-in instance 1. Then, when they invoked the data upload, the button was from add-in instance 2, so the customer created a support request that no data was uploaded.

Since the user was on a Mac, the only workaround was to unzip the xlsx, manually remove the webextensions folder content to get rid of the store-based add-in registration in the document, and zip it back again into an xlsx.

The feedback I got is "terribly poor user experience" 😔

Overall, I understand the security concerns mentioned earlier, but I'm afraid the current model is simply not working! IMO:

The users need to see only one of the add-ins, and if they have the same name, to select which one takes precedence.
Once a certain add-in explicitly takes precedence, the other add-in's registration should be removed from the document.

I wish to reiterate the fact that in this support request, the sideloaded add-in was not even in the equation, it was a duplication of the same add-in across the store-based and admin-based channels.

Overall, a huge pain point for us, I plead with you guys to address this as soon as possible.

Wenjun-Gong commented 1 year ago

Hi @wh1t3cAt1k,

We can take further look into the end user scenarios which do not use sideloading manifest which is for dev testing purpose.

For the cases you called out above, for the central deployment, is the source of the add-in manifest coming from Office app store?

wh1t3cAt1k commented 1 year ago

@Wenjun-Gong not sure I understood your question correctly, but yes, it's one and the same add-in, Velixo, available in the AppSource store.

Individual store-based version was recorded in the file, and the second, indentical, add-in instance came from centralised deployment by the organisation admin.

4tti commented 1 year ago

Hello all (@wh1t3cAt1k, @Wenjun-Gong )

sorry for the delay (too many things on the table). Here are few points from my side:

Magic button removing the add-ins is not good idea as already explained by Pavel. To be honest I am not sure how it will help and what is the difference to 'inspect document' -> it is just shortcut.
We are not necessarily talking about side-loading but let me pause here. Our customers do USE side-loading because simply AppSource nor Centralized deployment do not meet their needs in OnPrem solutions. I already explained that scenario above - i.e. you have e.g. N servers with various versions and you want to use same report for all of them. It can be test server vs prod server but it can even be region based servers. Excel is excellent that you can build fully dynamic report so then depending on what environment you use (as a source server) you will get corresponding data. Tell me please - how would you handle with e.g. Centralized deployment the Prod vs Test environment (e.g. version 3.5.1 vs 2.9.3 where Custom Functions and Ribbon have changed significantly). Basically same issue with AppSource (at least from my understanding). As I already mentioned - all of this is consequence of the initial design of the manifest (not allowing dynamicity).
With one customer I've encountered exactly same issue - having two task panes and some weird behavior. Fortunately the customer was extremely nice and patient! However, I already have feedback from multiple users (customers) that the user experience is poor ... and all such complaints were related to this poisoning!!

Please, don't focus on just 'deployment' but really take time and come with improved design to allow:

dynamicity in the manifests (i.e. we can have single frontend that will be able to handle multiple versions of Ribbon/CustomFunctions), or at least
ability to switch between multiple versions of the add-ins like proposed above (no matter the deployment) - user should be able to decide what he wants to use.

Argument with security is a bit funny. Why should Office/IT admin of the company change the manifest file or how does it influence security? Office/IT admins can do more dangerous things. Or I am simply not understanding this point - would be nice to get more details.

Side note - editing of the Manifest could actually be sometimes very useful, especially when there's some extremely important emergency escalation. In combination with ability to e.g. switch from 'AppSource' add-in to "local add-in" could give us powerful feature.

zpfaltersack commented 1 year ago

I have a very similar situation going on. I was going to open my issue for it but ultimately, while the trigger is distinctly different than anything listed here, the outcome is so similar that I figured I'd chime in and then open my own if directed to after the fact.

We have an add-in that is not in the app store and all of our customers use centralized deployment (we help set this up during onboarding so we know it's consistent across our customers).

For us, this poisoning occurs nearly every time a workbook is open in Excel Online and the feature to "Open in Excel Desktop" is used to open the workbook locally. In some of the cases listed on this thread it sounded like there were more ambiguous times the issue was seen so I wonder if it was due to this workflow. If an instance of Excel Desktop is already open locally then this makes the problem manifest basically 100% of the time.

I understand the whole dilemma about extension IDs and all that being generated and distinct based on where the add-in is installed from, but in this context, it's all from the same distribution mechanism with the only difference being this transition from editing in Excel Online to Excel Desktop.

This is really affecting our users quite a bit because it turns out this is a common workflow for reasons that aren't worth getting into.

gmichaud commented 1 year ago

One more example showing how problematic the overall user experience is, this time on video: https://www.loom.com/share/50f4cda390124766ba483aab9f8767cb

Users sharing files with people outside their organization is a pretty normal and common scenario, and I don't ever expect users to be able to figure out how to work around error like the ones on the video above.

Given the fact centralized deployment works by picking an add-on from the store I think it is reasonable to expect Excel to recognize the referenced add-on can also be found in the store.

And if we're talking about custom add-ins deployed centrally (which is NOT the case here), I would expect a clear warning or error message, ex: "Warning: add-in XYZ, referenced by this file, is unavailable in your organization.". Right now you get a "Could not connect to store" message with a Retry button, and the page that opens when you click "Get Details" does not provide any information.

zpfaltersack commented 1 year ago

After continued poking around I've identified with a little more clarity the differences between my issue and this one. Ultimately, even though we have only a single deployment mechanism that our users use, Excel Online seems to associate the add-in with a workbook in multiple ways. This results in our workbooks/Excel ending up in the "same" state as those defined in this issue. It's distinct enough I went ahead and created a new issue #3076. This note is just to close the loop for anyone that finds this but is experiencing something more akin to my workflow.

wh1t3cAt1k commented 1 year ago

Thank you @zpfaltersack for #3076, it likely affected us, too, without us knowing it.

Still, I would like to return here to the fundamental pain of add-in identity and poisoning, and again to emphasize how ugly/confusing it is for the users, who have no way of resolving the situation themselves in a satisfactory way.

We literally just had another crazy support request around this. Initially Velixo was deployed centrally for this user's organisation.

Though the person claims to have accessed the document using his work account, they started seeing an "add-in error" (Excel Desktop for Mac, latest version) right after switching to a new computer.

So, as a workaround, they disabled centralised deployment and reinstalled the product from the AppSource store.

All their old documents stopped working with an add-in error 🤡 because they were still "attached" to the centrally deployed version of the add-in.

We spent an hour (!) with him figuring things out on a call; even then, he couldn't self-resolve the issue, because there is no Document Inspector on a Mac where he could remove the add-in. However, even using Document Inspector cannot be seriously considered a satisfactory workaround, because the add-in loses all its internal add-in scoped state (including the defined connections and user settings that we store there...).

The user is a CIO with years of technical / coding experience, very smart guy, and he could not figure it out himself. Imagine a non-technical Excel user who our product is oriented towards.

This, I think, is another data point that illustrates our opinion that the user experience around the way add-in registers its presence & identity across multiple deployment channels (central, manifest, store) is fundamentally broken.

We desperately need consistent add-in's identity that works seamlessly when the document is passed between, say, developers (using a manifest), customers (using central deployment OR store add-in), and our support team (using the store version of the add-in).

If you think about it, the flow between customers (using central deployment in their organisation, or store) and support team (using store or central deployment in Velixo's organisation) is very common.

wh1t3cAt1k commented 1 year ago

This is not what our users expect - with XLL, it just worked for them, plain and simple.

The ideal scenario for us is a state where, regardless of the distribution channel, a person can open a shared document and continue working in it regardless if he's

a developer using sideloading,
or a member of a different organisation that deploys the same add-in centrally
or a random guy who installed the add-in from the store,

no matter what channel was used by the document's author.

4tti commented 1 year ago

Hey @wh1t3cAt1k,

I totally agree with your last statement. I'd like to emphasize that actually the whole manifest design and embedding in documents/workbooks/... is flawed. 1. Why should it matter from what source the add-in is coming? 2. Why could not user pick what source s/he wants to use if there are more available (and so version)? 3. Why custom functions/ribbon/keyboard shortcuts must be part of manifest and so cannot be introduced dynamically?

There are many questions and no answers. We stated here multiple times HOW PAINFUL it is for our customers... and still we don't even get feedback. I have mentioned several times that almost every On Prem customer we have is using/wants to use 2 environments - Test and Prod. The test one is for testing new version (features, that everything works as expected, etc.). And it is horrible exercise they need to undergo to switch the environments, in case of reports they already learnt (hard way) that they need two sets of reports, etc. Usually these testers are some advanced users WHO actually do the finance reporting in the Prod as well!!

And no response, no update from Microsoft again for month/s 😞 I am not sure how to make you aware how critical this is and how crucial the communication with your "adopters"/users is!

gmichaud commented 1 year ago

@4tti amen! 🙏 this is a big problem, which causes a DISPROPORTIONATE number of support requests and consumes a lot of our time. Even the most advanced users are unable to figure this out without assistance, and fixing the "bricked" workbooks is a pain.

4tti commented 1 year ago

Moreover, @Wenjun-Gong you mentioned this:

We can take further look into the end user scenarios which do not use sideloading manifest which is for dev testing purpose.

Can you give me any instructions how we can support (without side-loading) the test vs prod environments scenarios? I.e. single user wants to use "Addin A" on version X with the reports and then s/he wants to CHECK same reports on "Addin A" but version Y? How would you achieve that with centralized deployment or with AppSource? There are workarounds and the easiest is side-loading - but still really painful. I've asked multiple times and never got answer because it is scenario which is totally ignored.

Everything would be just ok if we would have capability to dynamically load things (ribbon, custom functions, ...) because then the WebApp could actually serve as 'crossroad' where user can pick what version to use, etc. and it would work perfectly fine with AppSouce/Centralized deployment (if done properly). Or if this is not OK for whatever reason - possibility for the user to pick which version of add-in should be used (but then e.g. centralized deployment would have to support deploying multiple versions of same add-in) would be just fine as well - but without stupid cache cleaning and document inspecting!

parched commented 1 year ago

@4tti The workaround we're using is to load the custom functions at runtime. It's not documented, but it works. Within the taskpane, we have a "backend" setting that can be changed from prod/test/localhost. I do the development of the custom functions with it set to localhost. I don't know if there's a way to load the ribbon dynamically though, unfortunately.

4tti commented 1 year ago

@parched thanks a lot! Good to know! Yes, I knew ScriptLab is using dynamic registration and I've asked why it is not available. Did not know how to achieve that though, thanks for giving detailed description. That's actually what I am looking for from Microsoft, to officially support this (or similar) approach; ideally with ribbon and shortcuts.

Unfortunately, afaik this is not officially supported and so can be subject of change. We cannot adopt it :(

wh1t3cAt1k commented 1 year ago

Hey @wh1t3cAt1k,

I totally agree with your last statement. I'd like to emphasize that actually the whole manifest design and embedding in documents/workbooks/... is flawed. 1. Why should it matter from what source the add-in is coming? 2. Why could not user pick what source s/he wants to use if there are more available (and so version)? 3. Why custom functions/ribbon/keyboard shortcuts must be part of manifest and so cannot be introduced dynamically?

There are many questions and no answers. We stated here multiple times HOW PAINFUL it is for our customers... and still we don't even get feedback. I have mentioned several times that almost every On Prem customer we have is using/wants to use 2 environments - Test and Prod. The test one is for testing new version (features, that everything works as expected, etc.). And it is horrible exercise they need to undergo to switch the environments, in case of reports they already learnt (hard way) that they need two sets of reports, etc. Usually these testers are some advanced users WHO actually do the finance reporting in the Prod as well!!

And no response, no update from Microsoft again for month/s 😞 I am not sure how to make you aware how critical this is and how crucial the communication with your "adopters"/users is!

Inviting @Wenjun-Gong and @jakobpn , it seems like a small crowd is gathering, all with the same problem.

I do agree that an ideal picture would be:

At least an opportunity to delimit "channels" (pre-release / GA) in the manifest, with an opportunity for the user to switch between channels. And a possibility for the developer to do staged roll-outs.
Consistent add-in identity regardless of the distribution source. So that a report can safely be passed between:

a developer (using dev manifest),
Velixo support (using store add-in or Velixo-org centrally deployed manifest),
and our customers (using store add-in or "their-org" centrally deployed manifest).

Without crazy magic of cleaning up with document inspectors, or separate copies of add-in attaching to the document.

On the surface, it seems like these two concerns may be addressed separately; for my organisation, the second is more urgent.

At the same time, the first one becomes increasingly important, too: right now the key stakeholders of the organisation have no chance to try on new features and provide early feedback before they become GA. (because if they used dev manifest sideloading, they'd spoil the real-life reports due to poisoning 💩)

Wenjun-Gong commented 1 year ago

Hi all,

Thanks for all the feedback and discussion regarding the experience when a single user could have installed an add-in from multiple sources (side-loading, central deployment, and store). We have identified a few different issues/asks for the scenarios and will need to involve different teams to investigate different ones. To make user we can track all the issues easily, I would like to list all the issues discussed as the following. We probably will need to separate several issues to other GH issues for better tracking.

Add-ins from sideloading and store (or central deployment) are treated as different add-in, thus user sees “duplicate” tabs/buttons on the ribbon
Add-ins from store and central deployment (the same add-in from store) are shown as two different add-ins, thus user sees “duplicate” tabs/buttons on the ribbon
Add-in from central deployment only (a Custom App instead of from store), when a workbook is opened from Excel Online and the add-in is added, people see duplicate tabs when they open the file in client. Tracked in #3076. Thanks @zpfaltersack for opening this issue separately.
Add-in is central deployed by uploading a customized manifest instead of from store, incorrect waring shown as "Could not connect to store" message with a Retry button when the file is opened by a user outside of the organization
Add-in was central deployed first (for store or local?) for a user, people saw add-in error of “You don’t have permission to use this add-in” right after switching to a new computer (Mac). They removed the central deployed add-in, installed the store one, but the add-in stopped running.

@wh1t3cAt1k , @gmichaud , @zpfaltersack, @4tti , would you please check whether I missed any scenario in the above list?

I also saw @4tti asking the recommendation on add-in installation approaches for the Testing environment and production environment. Thanks @parched for sharing the best practices for CF. We will take a further look and share back. We may update our document as well.

Thanks again for all the discussion and feedback so far. We will have follow-up based on all the issues we have identified, so I would appreciate it if you can callout anything I missed above.

4tti commented 1 year ago

@Wenjun-Gong Thanks a lot!

In case you need more details feel free to reach out to me and I can describe our problem and painful workarounds.

zpfaltersack commented 1 year ago

@Wenjun-Gong That looks like a good summary to me. One minor point on the ticket that I opened: it is centralized deployment but the manifest was loaded as a Custom App, not from the store. Based on the other behaviors people have discussed I would be surprised if that was relevant but I wanted to clarify just in case. I believe the issue I opened has that detail but it might be lost in the wall of text. Thanks!

Wenjun-Gong commented 1 year ago

Thanks @4tti and @zpfaltersack for the confirmation.

@zpfaltersack yes that's a different issue comparing with others. Let's track that one in #3076 .

@wh1t3cAt1k and @gmichaud please feel free to add anything I missed.

I will get back to this thread shortly.

Wenjun-Gong commented 1 year ago

Here are some updates:

- We will continue to use this GH issue to track # 1 & # 2 listed above:

Add-ins from sideloading and store (or central deployment) are treated as different add-in, thus user sees “duplicate” tabs/buttons on the ribbon
Add-ins from store and central deployment (the same add-in from store) are shown as two different add-ins, thus user sees “duplicate” tabs/buttons on the ribbon

- We will track # 3 in GH issue #3076 and decouple from this GH issue.

- We need to track # 4 separately as the root reasons are different. @gmichaud , as you shared this issue initially, would you please create a new GH issue then we can reroute to the right SME? Thanks.

Add-in is central deployed by uploading a customized manifest instead of from store, incorrect waring shown as "Could not connect to store" message with a Retry button when the file is opened by a user outside of the organization.

- We will also need to track # 5 separately as this sounds like an issue for central deployment. @4tti, as you reported this issue initially, would you please create a new GH issue then we can reroute to the right SME? Thanks.

Add-in was central deployed first (for store or local?) for a user, people saw add-in error of “You don’t have permission to use this add-in” right after switching to a new computer (Mac). They removed the central deployed add-in, installed the store one, but the add-in stopped running.

If you see any other new issue different with the above 5, appreciate if you can create another GH issue. Thanks all.

gmichaud commented 1 year ago

@Wenjun-Gong I just created the issue you requested -- https://github.com/OfficeDev/office-js/issues/3196

4tti commented 1 year ago

Thank you @gmichaud!

@Wenjun-Gong should I also create GitHub issue for the scenario I described? You wrote:

I also saw @4tti asking the recommendation on add-in installation approaches for the Testing environment and production environment. Thanks @parched for sharing the best practices for CF. We will take a further look and share back. We may update our document as well.

I am not sure that updating documentation will help because Dynamic registration of CFs is not the only missing piece but also there is no way how to dynamically register ribbon/contextual tabs or keyboard shortcuts (or I am not aware of any). Unless you provide best practices how to handle Test vs Prod (version 1 vs version 2) of same add-in via various deployment methods (centralized deployment, store, ...). Thank you very much in advance!

Wenjun-Gong commented 1 year ago

Hi @4tti , yes we are also looking at whether there is any easier way to handle the Test & Prod environment. As for testing purpose we recommend sideloading, we will track this one together # 1 and # 2 in this GH issue. Thanks for your patience.

4tti commented 1 year ago

@Wenjun-Gong Thanks. Just clarification, by testing - are we talking about Add-in creator testing OR Add-in customer testing?

Wenjun-Gong commented 1 year ago

Hi @4tti ,

When we say testing, we mean the test and debug stage during an add-in development lifecycle: https://learn.microsoft.com/en-us/office/dev/add-ins/testing/test-debug-office-add-ins

@Wenjun-Gong Thanks. Just clarification, by testing - are we talking about Add-in creator testing OR Add-in customer testing?

Wenjun-Gong commented 1 year ago

And a few updates:

Add-ins from sideloading and store (or central deployment) are treated as different add-in, thus user sees “duplicate” tabs/buttons on the ribbon.

For this case, we are treating the two manifest as different add-ins. For sideloaded add-in, the manifest is customizable by the end user. We are not able to identify whether it is identical to the one submitted to App Source. Thus we do not dedup these two.

And for the add-in installed from Store (or central deployed from store), we have a capability to quickly withdraw add-ins that have turned malicious and so if we have a store add-in and can't talk to the store, we err on the side of caution. @slabereemsft also comented in 3004 as well: https://github.com/OfficeDev/office-js/issues/3004#issuecomment-1442425568

Add-ins from store and central deployment (the same add-in from store) are shown as two different add-ins, thus user sees “duplicate” tabs/buttons on the ribbon.

For this case, on Win32, we have already been dedup the add-in. Please verify whether it works. However, for WAC and Mac, we are doing some further validation. I will update this thread once we have further conclusion.

For the Test & Prod environment best practices, especially for the de-dup asks mentioned in this thread, we are doing some investigation for a better experience. I will get back.

gmichaud commented 1 year ago

@Wenjun-Gong I haven't seen the duplication on my side in a while, however the problem mentioned here https://github.com/OfficeDev/office-js/issues/3196 still holds -- is anything being done to address this problem?

wh1t3cAt1k commented 1 year ago

Add-ins from sideloading and store (or central deployment) are treated as different add-in, thus user sees “duplicate” tabs/buttons on the ribbon.

For this case, we are treating the two manifest as different add-ins. For sideloaded add-in, the manifest is customizable by the end user. We are not able to identify whether it is identical to the one submitted to App Source. Thus we do not dedup these two.

@Wenjun-Gong I think there still may be a way to handle this in a reasonable way. Almost 100% of our complaints in this scenario is a developer poisoning the file for the customer or another team member.

If the manifest XML differs from the store manifest only in URLs (localhost vs velixo.com for example) and version, it can probably be automatically deduped in favour of the store-based add-in installed for the customer profile. Because you already know that the store version is safe,

And vice versa, for the developer opening a customer document with store / centralised add-in who has a sideloaded manifest installed, the sideloaded version can be resolved and preferred using a similar strategy. Again, you know that the user themselves sideloaded this manifest, so you know that he considers it safe.

Overall still a big pain if this duplication is left unaddressed.

4tti commented 1 year ago

Hi @4tti ,

When we say testing, we mean the test and debug stage during an add-in development lifecycle: https://learn.microsoft.com/en-us/office/dev/add-ins/testing/test-debug-office-add-ins

@Wenjun-Gong Thanks. Just clarification, by testing - are we talking about Add-in creator testing OR Add-in customer testing?

But that is the issue... you are missing IMPORTANT scenario. I am talking whole time about CUSTOMER testing scenarios. It is same as Office channels - monthly channel vs. semiannual channel vs enterprise monthly channel, etc.

Enterprise companies are TESTING the updated versions BEFORE they consider it safe to update "globally", this testing is usually done by some smaller group and then provisioned/deployed to the big group of users. This testing usually means comparing new and old version that everything works as expected, it is not just "checking" new features.

So please consider this for the centralized deployment and store.

4tti commented 1 year ago

And still, I think it would be MUCH better to give option to the User to select action like source of Addin they prefer - ideally admins could customize this for the users (i.e. completely disable this for them, etc.).

ad1) you are able to find out as @wh1t3cAt1k wrote. You have original and new manifest and you actually CAN decide if the changes are malicious or not and then even give OPTION to the user. It is much better than to just silently duplicate the addins 😕

Anyway, thanks a lot for the progress and that you will check the testing scenario :)

wh1t3cAt1k commented 1 year ago

@Wenjun-Gong just wanted to attract your attention to the pretty bad UX of number 1 issue from your list above.

Here is what I am seeing after opening the same file where I installed an add-in from the store, sideloaded from the web and then ran testing with sideloading from the shared folder in Excel Desktop:

Three! icons and three conflicting "add-ins" registered in the file even though it's logically exactly the same add-in.

I am seeing three prompts when I open the file, it's a mess and I strongly feel we need a way out...

gmichaud commented 1 year ago

Also confirming that I'm experiencing the same behaviour on my side. It is very confusing. The add-in toolbar icon also appears multiple times.

I am constantly having to use the Inspect File button to clear add-ins, and we have to be careful not to open any file that we will be returning to a user when working with a side-loaded (dev) version of the add-in.

JEGUOCEAN commented 9 months ago

Hello, Our customers are also facing this issue that I though to be resolve since March. Is MS still working on a solution?

OfficeDev / office-js