Closed sean-mcmanus closed 2 years ago
Colengms
commented
2 years ago If a user experiences an issue with a prerelease build, they may want to manually switch to the immediately previous build, which might be a release build. Currently, that release build is not immediately obvious to them, and they are likely to switch to the previous insiders build instead, which may predate the latest release build.
Colengms
commented
2 years ago @isidorn @sandy081 Our team has been discussing the prerelease extension experience in VS Code's marketplace UI and have questions and some feedback regarding the current design.
If a user reverts to an older prerelease version, will they still be updated to the latest prerelease when it is released? I assume no, and that any time a user selects a version other than the most recent one they will not be moved from that version automatically.
Is it required that we always publish a prerelease build whenever we publish a release build, to ensure a (more recent) prerelease is always available? Our understanding is that some extensions are doing this. This seems necessary, currently, for the marketplace experience to make sense to the user. We would prefer not to have to have to publish superfluous pre-releases.
We had previously prompted opening of this issue: #139528 Our intention was to be able to subscribe users to prerelease builds (programmatically) despite the last published build being a release build. As a result, my understanding is that we now have a state in which the user should be receiving 'the prerelease experience' (in which they will be automatically updated to the next prerelease build) despite not being on a prerelease build currently. Can you provide any information on what happens in the UI when in this state? Is the user provided an option to unsubscribe from prereleases while in this state, if they decide they want to remain on the current release build after all? My understanding is that they would only be presented with ‘release’ UI as the currently installed version is ‘release’.
Recently, VS Code split the 'install another version' menu into 2 menus, one for releases and one for prereleases. To see a list of release versions, you need to be using a release build. To see a list of prerelease versions, you need to be using a prerelease build. (And there is an option to switch back and forth between release and prerelease). We’re concerned that this approach does not make it obvious to a user how to revert from a prerelease to an immediately previous (release) build. (This bug) For example; Say we release 2.1 prelease, 2.2 release, then 2.3 prerelease. The user has an issue with 2.3, and wants to install the immediately previous version. However, 2.1 is the next most recent prerelease displayed. They would need to switch to release to even determine if a 2.2 exists.
It seems like it would be preferable if the UI were independent of whether a release or prerelease build is currently installed. Suggestion: Rather than have the UI tailored to whether a release or prerelease build is currently installed, tailor UI to whether they are subscribed to an ‘insider experience’ for that extension (the state that would seem necessary to support the fix for #139528). It would display both release and prerelease versions in an ‘install another version’ menu instead of just prerelease versions and would continue to upgrade to new prereleases even when the user is currently using a (most recent) release build. It could provide UI to unsubscribe from prereleases (switch to ‘release experience’), even if on a (most recent) release build. The ‘release experience’ could also potentially provide UI to switch to the ‘insider experience’ despite no prerelease currently available (or not yet more recent).
isidorn
commented
2 years ago @sean-mcmanus @Colengms thank you for your feedback. Let me respond in order:
sean-mcmanus
commented
2 years ago @isidorn For our extension, our releases will almost always have a higher version than the previous pre-release version, except in the very rare case where we have to ship a "patch" fix release after a higher versioned Pre-Release has already been released.
bobbrow
commented
2 years ago @isidorn it is item 3 that we have issues with. We feel that it should be unnecessary to post a pre-release that is identical to the latest stable release. This doubles the work required by extension publishers for every stable release if folks subscribed to pre-releases don't get the highest available version installed. They should always be on the newest version whether is it a stable or pre-release. When they want to downgrade, all published versions should be available to choose from.
The way we see it, anyone who opts in to pre-releases should basically see all versions all the time. There are four states a user can be in at any time and users can transition between states by upgrading to latest/downgrading to specific builds or switching to and from the pre-release channel:
| on latest | not on latest | |
|---|---|---|
| subscribed to pre-releases | This user sees all published versions and auto-updates to the newest version (stable or pre-release) | This user sees all builds, is given the prompt to update to latest stable or pre-release in the Extensions panel, but does not auto-update |
| not subscribed | This user only sees stable versions and auto-updates to the newest stable | This user only sees stable builds, is given the prompt to update to latest stable in the Extensions panel, but does not auto-update |
sean-mcmanus
commented
2 years ago @isidorn Also, after we released identical 1.8.2/1.8.3 (pre-release) we've gotten a significant amount of "noise" along the lines of "it also repros in 1.8.3" because users expect 1.8.3 to be different and possibly have the fix for the bug they're hitting in 1.8.2. We'd prefer to avoid this "noise", and avoid wasting user's time upgrading and testing an identical version when it's unnecessary. For example, https://github.com/microsoft/vscode-cpptools/issues/8760#issuecomment-1031745964
isidorn
commented
2 years ago @bobbrow let me clarify the current behavior, there seems to be a misunderstanding.
As for your table I am sure the on latest current behaviour is as you describe. For not on latest I will let @sandy081 comment.
Long story short: I think you are good. You do not need to release subsequent pre-release versions if you do not mind your pre-release users shortly being on a release version of the extension. Our recommendation to release subsequent pre-release version is only for those extensions that would like to avoid their pre-release users getting the stable version - which is not the case for you it seems.
sean-mcmanus
commented
2 years ago @isidorn I think there is still a misunderstanding. We already understood the behavior in your "Tom" scenario in regards to how updating will occur, because I believe Colen filed a bug directly on that which got fixed. Our concern is about the "downgrade" options that are available to Tom after upgrading to the release version 1.7.3 -- we want the "previous version" to always appear in the downgrade list, which doesn't happen unless we ship identical pre-release versions for every release version.
bobbrow
commented
2 years ago @sean-mcmanus I think we're all back on the same page. The ability for subscribers of pre-releases to see all versions when they want to downgrade is the subject of this issue. Instead of seeing "Install Another Pre-Release Version" in the context menu, it should just be "Install Another Version" and all available releases should be offered. @isidorn seems to agree in bullet number one of his first comment.
There was some side discussion about the UX and whether it is confusing for folks "subscribed" to pre-releases to be offered stable releases (@Colengms's last bullet). If we want to discuss that further, I think it would be best to open another issue about it.
sandy081
commented
2 years ago Apologies for not commenting on this until now.
Thanks @isidorn for taking the 🔥 and answering the questions. I completely agree with @sean-mcmanus @bobbrow that the root cause or the single core problem you are seeing is with Downgrading. I am summarizing the points and actions to be taken here:
We used to show all versions for Install Another Version... action just like how @Colengms mentioned here - Suggestion: Rather than have the UI tailored to whether a release or prerelease build is currently installed, tailor UI to whether they are subscribed to an ‘insider experience’ for that extension. It would display both release and prerelease versions in an ‘install another version’ menu instead of just prerelease versions. But we switched this behavior after this issue - https://github.com/microsoft/vscode/issues/140262. It seems this change is not welcome. Root cause is that, showing all versions is making the dropdown overpopulated and not easy to find the recent release. But I agree that old behavior is more reasonable. I will revert this behavior and reopen the other issue. - Action Required
Reverting to an older version will stop auto updating. User has to manually update the extension. But currently installing an older release version is clearing the pre-release flag. I agree that retaining the flag in this case makes sense. - Action Required
Is it required that we always publish a prerelease build whenever we publish a release build, to ensure a (more recent) prerelease is always available? Answer is No. Once you publish a new pre-release, extension gets auto updated to pre-release if user has opted to installed pre-release before.
When the user is on release version which is the newest version and user has opted to install pre-release before, UI does not show any indication of pre-release. But I agree that it makes sense to indicate user that user has opted to install pre-release and if there is a new pre-release version available, user will be updated to that - Here is the issue tracking this - https://github.com/microsoft/vscode/issues/141282
@bobbrow - Your comment here very well documented what is needed to be supported that aligns exactly with my above points. In short, we always install/update to latest version of release/pre-release depending on what user has opted to. It is not mandatory to have a newest pre-release version always (which I also mentioned in above points) and it is up to the extension. In addition to what we supported now, supporting 1 & 2 points will fulfill the table you mentioned.
Actions Required: Points 1 & 2.
Updating the issue summary accordingly.
gjsjohnmurray
commented
2 years ago @sandy081 regarding point 1, I think a user who has opted in to pre-release should get "Install Another Version" and be offered both kinds (pre-release and ordinary ones), but a user who hasn't switched to pre-release should only be offered ordinary releases (i.e. not shown the pre-releases). Indeed, I think that is what @alexr00 originally requested in #140262
sandy081
commented
2 years ago @gjsjohnmurray You are 💯 right. I was in the impression that I was supporting the behavior requested here before #140262 was implemented. And with #140262 I thought @alexr00 is requesting to provide an option to filter pre-release versions from the versions list when user clicks on Install Another Version... on pre-release extension. It was completely my mistake to mis-understand that requirement.
I would be implementing the following requested behavior for Install Another Version... action
I think a user who has opted in to pre-release should get "Install Another Version" and be offered both kinds (pre-release and ordinary ones), but a user who hasn't switched to pre-release should only be offered ordinary releases (i.e. not shown the pre-releases)
@alexr00 Please confirm if your ask in #140262 is also same as above?
sean-mcmanus
commented
2 years ago Ah -- our extension only releases a small number (like 5 at most) of pre-releases between a release version. The problem of having a hard time finding the release version in a giant list of pre-release versions could maybe have some alternative fix for the subset of users who ship a lot of pre-release versions.
alexr00
commented
2 years ago I think a user who has opted in to pre-release should get "Install Another Version" and be offered both kinds (pre-release and ordinary ones), but a user who hasn't switched to pre-release should only be offered ordinary releases (i.e. not shown the pre-releases)
Yes, I would still be very happy with this. My issue was primarily around stable users seeing tons of pre-release builds. That is resolved by your original change @sandy081 and would still be resolved by the proposed change. I do not have strong feelings either way for whether pre-release users see stable builds.
sandy081
commented
2 years ago At present, downgrading to any release version will opt out of pre-release and user has to explicitly switch to pre-release version to opt in again. IMO this makes sense because, by choosing to install a release version, user has opted out and user can always use the action Switch to Pre-Release version to opt in back. I also think that the current UI represents this flow very well and is simpler to understand.
May I know the reason of not opting out when installing an older release version? I think this makes UI more complicated for the user to understand.
Please assume that this flow is considering that Install Another Version... action shows versions as mentioned here
sean-mcmanus
commented
2 years ago @sandy081 Consider this...
Our team wants "release" vsix's to be fully usable by "pre-release" and "release" users without causing any opting out. From the user's perspective, they're just downgrading to the previous verson they were using (while in "pre-release mode")...same as when the user doesn't get opted-out of pre-releases when they're auto-upgraded to a release version.
sandy081
commented
2 years ago @sean-mcmanus
What is the UI you are expecting after downgrading to 1.8.4 release version? Generally, when you downgrade a version we show Update to
sean-mcmanus
commented
2 years ago If the user has opted into pre-releases, we want them to see "Update to 1.9.0" or "Update to 1.9.1" (when that is released), regardless of if they downgraded to 1.8.4 (release) or a pre-release (1.8.3 or 1.8.5).
sandy081
commented
2 years ago But as a user I would argue that if I switch to a release version either using Switch to Release Version action or through Install Another Version... action, I choose to be on released versions. I do not want this update action in my face that takes me back to pre-release. I would use Switch to Pre-Release Version action to go back to pre-release again if wanted.
I think the problem we are trying to solve here is following:
I am on a pre-release and I see it is buggy, so I would like to go back to a release version (using any of the above actions). How does I know there is a new pre-release that is not buggy? I do not think showing the action Update to 1.9.0 or Update to 1.9.1 helps because I do not remember versions. IMO this needs some UX thinking and user feedback.
bobbrow
commented
2 years ago Our interactions with customers on GitHub tell us that our users do remember which versions are the bad versions that impact their work.
I think the sticking point here is the semantics of "I'm a pre-release user" vs "I'm a stable user." I don't think we should be using these two buckets. The buckets we've defined for the C++ extension are "I can see all available releases" vs "I only want to see stable releases". The difference is subtle. In the first case you could argue that the user only wants to see pre-releases, but I don't think they care if the latest is a "stable" vs a "pre-release". They just want to be on the latest version. And if the latest version has a regression that impacts their work, they downgrade until a new release comes out (pre-release or stable, it doesn't matter to them). We've been doing it this way for several years now and have not had any reports of confusion from our customers.
So coming back to this:
I think the problem we are trying to solve here is following:
I am on a pre-release and I see it is buggy, so I would like to go back to a release version (using any of the above actions). How does I know there is a new pre-release that is not buggy? I do not think showing the action Update to 1.9.0 or Update to 1.9.1 helps because I do not remember versions.
Our "pre-release" customers don't care if the version they go back to is a stable or a release version. They just want to roll-back by one. And when the Extensions Panel offers them a new one, they can go try it out (again, they actually do remember the buggy version numbers). If rolling back by one takes them out of the "I want to see all releases" bucket just because the version they rolled back to happens to be a "stable" version, I think that is actually the more confusing behavior.
sandy081
commented
2 years ago @bobbrow Thanks for the detailed explanation and I see the validity of the use case. More importantly, it is helpful to know that c++ extension runs this model already and has user feedback.
I already fixed the bucket that a pre-release user can see all available releases.
As this issue is getting longer and has multiple topics, I would like to create a separate issue for retaining pre-release flag on downgrading and continue discussion there - https://github.com/microsoft/vscode/issues/142659
Lets close this issue for fixing Install Another Version... action
bobbrow
commented
2 years ago As this issue is getting longer and has multiple topics, I would like to create a separate issue for retaining pre-release flag on downgrading and continue discussion there - #142659
Sounds good. Thanks @sandy081!
sandy081
commented
2 years ago To verify:
Does this issue occur when all extensions are disabled?: Yes
Steps to Reproduce:
For example, 1.9.0 (insiders) ships with some regression, so a user wanting to downgrade to the latest version without the regression would want to downgrade to the previous release, which ideally would be 1.8.2 (non-Insiders), but due to this bug, they'd only be shown 1.8.1 (Insiders) unless we simultaneously published a 1.8.3 (insiders) that is identical to 1.8.2. We'd prefer users to not ever have to switch off the Pre-Release "channel" in order to get access to those versions, because that makes it harder for them to see when 1.9.1 (Insiders) is released (i.e. with the fix they want), once they switch to a non-Insiders -- to say it another way, we'd want Pre-Release users who switch to a non-Pre-Release version to still be counted as Pre-Release users, the same as if all our non-Pre-Release vsix's were also counted as Pre-Release.
The "reverse" scenario of showing the Insiders versions in the non-Insiders version selection list doesn't really matter to us either way.