flcdrg / au-packages

David's Chocolatey Automatic Packages
MIT License
18 stars 52 forks source link

[microsoft-teams-new-bootstrapper] Can we download the MSIX? #163

Open flcdrg opened 7 months ago

flcdrg commented 7 months ago

The bootstrapper.exe can download the MSIX. Is it possible for chocolateyInstall.ps1 to do this instead?

https://learn.microsoft.com/en-us/microsoftteams/new-teams-bulk-install-client#option-1b-download-and-install-new-teams-using-an-offline-installer

QuimaxW commented 7 months ago

My hunch is that these links will be stable to the current version of the msix, regardless of it's version on a given day.

x86: https://go.microsoft.com/fwlink/?linkid=2196060&clcid=0x409 x64: https://go.microsoft.com/fwlink/?linkid=2196106 ARM64: https://go.microsoft.com/fwlink/?linkid=2196207&clcid=0x409

I don't think the chocolatey package could be versioned off of those though.

QuimaxW commented 7 months ago

Redistribution rights shouldn't be an issue as the machine installing the package is pulling everything from Microsoft's servers at the time of install. Only folks who are internalizing the package for their organizational use would need worry about redistribution. However, as Microsoft has instructions for doing exactly that, it's being expected in that scenario that the redistribution is happening.

Another workaround for my case would be that we build a custom package for this that would have everything in the package.

flcdrg commented 7 months ago

Those links resolve to

Those URLs at first glance don't look like they probably change when the binary files change, which is disappointing. Will have to take another look when a new version is released.

flcdrg commented 7 months ago

For reference, the AppManifest.xml file inside the x64 msix contains this, which might be useful as a version marker.

  <Identity Name="MSTeams" Version="23320.3021.2567.4799" Publisher="CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US" ProcessorArchitecture="x64" />
flcdrg commented 6 months ago

A new version of the package was generated overnight. Interestingly the msix files haven't changed (and are a bit outdated compared to the version of Teams I have installed locally).

I'm going to create a separate package for the MSIX files, with the intention that this package can depend on that one.

flcdrg commented 6 months ago

Interesting to compare the bootstrapper version numbers to the ones reported in Teams.

image

image

teknowledgist commented 6 months ago

Maybe this is an ignorant comment, but I don't understand why the bootstrapper and MSIX are separate Chocolatey packages. The goal here is to have a known version of (New) Teams installed, right? Do either of the bootstrapper or MSIX make it possible to know which version of Teams is ultimately installed?

If not, then is it even possible to know there is a one-to-one match between the Teams version and either of these download versions? How can we know what version of Teams, Chocolatey will be installing?

If the Teams version can be identified, then why not just make the package download both the bootstrapper and MSIX files and keep the package management simple? The chocolateyinstall.ps1 code might not be so simple, but the user will have a lot less confusing time of tracking it. Look at the Adobe reader package, for example. It might have to download/install the "base" version and then an updater, or it might just need to install the updater. In the end though, the user doesn't need to worry about all that.

If I'm misunderstanding something, feel free to explain (or not) and I'll shut up.

flcdrg commented 6 months ago

It is a good question and one I thought about (and one that could be reversed in the future). I decided to go with separate packages as the two parts do seem to be versioned separately, and updated on a different schedule.

Even then, the MSIX downloads don't always appear to be the latest version that you can get if you check for updates within Teams, but maybe Microsoft are just being a bit conservative when they update the download URLs.

Also worth noting - this package is only good for pre-installing Teams on a machine the first time.. It isn't any good for updating Teams once you are using it (as far as I know). So the choco version numbers would also quickly be outdated once the client starts self-updating.

flcdrg commented 6 months ago

The "two package" solutions is just in preview, so now is a good time to discuss if it makes sense or not. Feedback welcome!

QuimaxW commented 6 months ago

@teknowledgist - My goal was to have a self-contained package that would internalize properly and not require a separate download which would slow down deployments. This would also have a benefit of bringing in a fairly current version of Teams as well.

However, as @flcdrg mentioned, unless one has a way of turning off the auto-updating of Teams, this package becomes quickly out of sync with Microsoft's versions. Just keeping with the current version could be a full time job to do it manually. We'd need a script that would download the bootstrapper and the msix, extract the version number, update the nuspec file(s), and upload those back to the community repo on a near nightly basis.

What I'm likely to do for our internal repo is have a script run weekly to pull the MSI file down to a local http server. Use the bootstrapper package, modified to use this http location as the source for the MSIx. This would follow the same model we use for Office365, where the package version is only bumped when we need to force a reinstall of the product.

flcdrg commented 6 months ago

@QuimaxW I don't use Chocolatey for Business, though I have heard of the internalising stuff. Does the internalising thing mean you'd prefer a single package, or is having two packages fine?

As far as I know, this "install it just once" is pretty much the same behaviour as the old Teams Enterprise installer. Once the bits are on the machine then Teams will self-update for each user on that machine.

teknowledgist commented 6 months ago

That Chocolatey package versions can be used "just for install" and the app upgrades itself is not terribly unique. Look at Firefox or Chrome. I do think Teams is different from those in that the self-updates are per-user.

The "Old Teams" enterprise installer was actually an installer-installer. It would install the per-user installer so that said installer would run on login of each user. The per-user installer would either install Teams or would upgrade Teams, but the Chocolatey package version would still match the enterprise installer version. I don't know if the per-user installer would install the "installed" version on first run and then update to the latest (whether immediately automatically, immediately if incompatibilities were discovered, or the next time the user logged in) or would download the latest version to install. I do know that the "install on login" slowed down new user logins and was tremendously annoying in classrooms and labs where Teams might be wanted, but only on demand. The per-user nature of it also meant that a choco uninstall would only remove the "installer-installer", but all existing user profiles would continue to have (and update) Teams.

I am assuming the "New Teams" installer is a similar kind of bastard, and thus, I still think the package should be focused on the version of Teams that will result on a clean install (i.e. first login). I should clarify that to the version of Teams before it updates itself. For example, the version of Teams that a new user would see if there was no Internet access for Teams to update itself. Like "Old Teams", the version any user has/uses will not match the Chocolatey package version, but the "primed" installer should continue to match the Chocolatey version.

My issue with having two packages is that they each seem meaningless without the other. Who cares if they have version xyz of the bootstrapper (or MSIX) if they don't know what version of Teams will result from a fresh install of it? How are you going to know what versions of the two downloads are compatible with each other? I think a "mono-package" is better for internalizing too. You are basically "freezing" the installer version and don't need to worry about the versions of the individual downloads.

As for how you determine the version of Teams to name/version the package... I don't know. MS is really terrible about some software download and/or versioning. I have to troubleshoot the OneDrive package all the time because they change the URL without changing the version or the reverse, or sometimes change both without updating the announcement page.

Anyway, that's my $.02. Thanks for the discussion opportunity.

flcdrg commented 6 months ago

You're bringing me around to the idea of having a single package - but yes as you hint at, then the question is, how do I version that package?

The MSIX version seems like it would be useful (as it corresponds with what you'd see in the UI - see screenshot above), but it is already a 4-part version, so there's no way to safely increment it when a new bootstrapper.exe was released but the MSIX was not updated.

The version numbers is probably the stumbling block to having a "mono-package"

teknowledgist commented 6 months ago

I suspect it is unlikely that the bootstrapper would update without the MSIX changing. However, given that this IS Microsoft...

OneDrive is also a "full", 4-part version, so when MS plays their three-card monte game and changes download URLs without changing the version (or other disorganization), I just add one to the last number. I don't know any software anywhere that both uses a full version number AND releases single increments on the last number. Take your example above. How likely is Microsoft to release version 23320.3027.2591.1506 after version 23320.3027.2591.1505? There is no Chocolatey rule that says the package version has to match the software version except for a possible date string in the build version slot.

flcdrg commented 6 months ago

Unfortunately Microsoft have done just that. The MSIX was last updated early December, but bootstrapper was updated as recently as a week ago.

The date string in the build version slot is an interesting idea. It would be helpful to see how Microsoft do change the MSIX versions over time. If they always rev the 3rd number, then we could mess with the 4th safely.

flcdrg commented 6 months ago

Found a published Teams client version history here - https://learn.microsoft.com/en-us/officeupdates/teams-app-versioning?WT.mc_id=DOP-MVP-5001655#windows-version-history

The current MSIX download version is 23320.3021.2567.4799 which doesn't match any of the recent Windows versions, but seems to be between the two December releases

Release year Release date Teams version Slimcore version
2023 December 27 23320.3027.2591.1505 2023.44.01.03
2023 December 07 23306.3315.2560.6525 2023.43.01.22

The 4th number does change, so that is promising

ItzLevvie commented 5 months ago

I work for Microsoft but I do not work on the Microsoft Teams engineering team.

Also worth noting - this package is only good for pre-installing Teams on a machine the first time.. It isn't any good for updating Teams once you are using it (as far as I know).

teamsbootstrapper.exe can be used for downgrading and upgrading Teams 2.1 using .\teamsbootstrapper.exe -p -o "C:\Users\$env:USERNAME\Downloads\MSTeams-x64.msix" which will take advantage of defer registration but it is not an officially supported scenario by Microsoft.

Its main purpose is to bulk deploy the new Teams so there could be unattended side effects for using it for downgrading and upgrading your Teams 2.1.

Those links resolve to

Those URLs at first glance don't look like they probably change when the binary files change, which is disappointing. Will have to take another look when a new version is released.

Even then, the MSIX downloads don't always appear to be the latest version that you can get if you check for updates within Teams, but maybe Microsoft are just being a bit conservative when they update the download URLs.

Those URLs will always point to the Last Known Good (LKG) installers, and they are only updated to newer versions depending on our telemetry in different cloud environments such as Public Cloud, GCC, GCC High, DoD, and Gallatin.

There are essentially three release states:

As of today for Ring 4 - all three are the same now:

I could create an aka.ms URL just for Chocolately to always download latest GA version of Teams 2.1 instead of the LKG version but that requires manual work from my side as well as setting up automation to make sure the aka.ms URLs are continuously updated to always point to the latest GA version of Teams 2.1.

We'd need a script that would download the bootstrapper and the msix, extract the version number, update the nuspec file(s), and upload those back to the community repo on a near nightly basis.

You can just parse the JSON from https://config.teams.microsoft.com/config/v1/MicrosoftTeams/49_1.0.0.0?environment=prod&audienceGroup=general&teamsRing=general&agent=TeamsBuilds to get the latest build version for Teams 2.1, and then make a version-specific URL by replacing lkg with that build version, or just use the version-specific URLs from there.

A lot of what you see in the JSON is used by Teams 2.0 and Teams 2.1 where it periodically checks for updates for the frontend (WebView2) and for the backend (React Web Client + CDL Worker).

BuildSettings.Desktop is Teams 1.0 (enterprise). BuildSettings.DesktopMsix is Teams 1.0 (enterprise) converted to MSIX using MSIX Packaging Tool. BuildSettings.WebView2 is Teams 2.0 (consumer). BuildSettings.GamebarWidget is the Xbox Game Bar widget for Teams 2.0 (consumer). BuildSettings.WebView2Canary is Teams 2.1 (enterprise). BuildSettings.WebView2Canary.MTRW is Teams 2.1 (enterprise) for Microsoft Teams Rooms.

That is what powers Microsoft Teams under-the-hood as well as what I am currently doing: https://raw.githubusercontent.com/ItzLevvie/MicrosoftTeams-msinternal/master/defconfig2


Regarding teamsbootstrapper.exe - newer versions of teamsbootstrapper.exe will only contain bug fixes, new features, and improvements to the commands available in teamsbootstrapper.exe. You can use an old version of teamsbootstrapper.exe to install the latest version of Teams 2.1 but using the latest version of teamsbootstrapper.exe is always recommended to make sure the deployment of Teams 2.1 is smooth.

How likely is Microsoft to release version 23320.3027.2591.1506 after version 23320.3027.2591.1505?

It is very unlikely for this to happen on our public release trains but it does happen to our internal release trains especially between Windows and macOS.

Those internal release trains are on our internal environment which is separated from our production environment so if you really wanted to do this then there will be zero issues and zero versioning conflicts between public release trains and internal release trains.

I have to troubleshoot the OneDrive package all the time because they change the URL without changing the version or the reverse, or sometimes change both without updating the announcement page.

I have taken a look at your OneDrive package, and it looks like most of the issues stems from scraping build versions from support.microsoft.com which closely resembles what is happening on Microsoft Docs. See https://www.reddit.com/r/sysadmin/comments/11tr33k/onedrive_build_newer_than_production_ring_build/

When the go.microsoft.com links change that is mostly down to our engineers because manual work is required to keep the links updated. Sometimes our engineers will update the same go.microsoft.com link to point to the latest version, and other times our engineers will create a brand new go.microsoft.com link to point to the latest version. Some go.microsoft.com (and aka.ms) links have automation in place to make sure that these types of errors are as close to zero as possible.

I feel like it would be much better to use solutions from https://github.com/aaronparker/evergreen/issues/268#issue-1059501235 since that is automatically updated from our flighting system and will result in much less errors.

The date string in the build version slot is an interesting idea. It would be helpful to see how Microsoft do change the MSIX versions over time. If they always rev the 3rd number, then we could mess with the 4th safely.

We use the YYDDD.XXXX.XXXX.XXXX version format for Teams 2.0 and Teams 2.1.

As an example - 24034.800.2673.6675 is broken down into the following: 24 (YY) is the year of the build. 034 (DDD) is the day of the week for that year. 800 (XXXX) is automatically generated by our CI system (Azure DevOps). 2673 (XXXX) is the total amount of builds we created excluding servicing builds. 6675 (XXXX) is automatically generated by our CI system (Azure DevOps).

There is a lot more to this such as the usefulness of the 2nd part number (800) as well as how we do servicing but I do not want to go into too much detail for something that most people will not care about.


We use the 1.X.00.DDDXX version format for Teams 1.0.

As an example - 1.7.00.3316 is broken down into the following: 7 (X) is 2024 which is the year of the build. 6 is 2023; 5 is 2022; and vice versa. 33 (DDD) is the day of the week for that year. 16 (XX) is automatically generated by our CI system (Azure DevOps).

We have two build flavors for Teams 1.0. One is Stable (S) and the other one is Experimental (E). Any build version ending with 1 to 49 is a daily Experimental (E) build; and any build version ending with 51 to 99 is a daily Stable (S) build.

Found a published Teams client version history here - https://learn.microsoft.com/en-us/officeupdates/teams-app-versioning?WT.mc_id=DOP-MVP-5001655#windows-version-history

The current MSIX download version is 23320.3021.2567.4799 which doesn't match any of the recent Windows versions, but seems to be between the two December releases

A lot of the Microsoft Docs pages, including the ones for Windows and other Microsoft products, are usually delayed by a few days to a few weeks because of priority work so some build versions will be missing.

Reliability on these pages for scripting purposes are not always guaranteed to work due to typos which tend to happen from time-to-time but sometimes those are quickly fixed before we receive complaints from consumers and commercial customers, and sometimes those typos will be unnoticed by our engineers (or copywriters) where those typos would stay there for a weeks to a few months to a few years until someone spots the error and makes a pull request to fix that.

flcdrg commented 5 months ago

Thanks @ItzLevvie for that detailed info. That will be helpful in figuring out a version for the package.

flcdrg commented 4 months ago

I've just published a preview version of a combined package - https://community.chocolatey.org/packages/microsoft-teams-new-bootstrapper/1.0.2402201.24004-pre. The version number uses the first 3 parts of the bootstrapper version and the first component (the YYDDD bit) of the Teams 2.1 Enterprise version.

choco install microsoft-teams-new-bootstrapper --version=1.0.2402201.24004-pre --pre 

Questions:

  1. I added the MSIX version to the title, but is that too much (it is also documented in the description)
  2. Anything else you think needs including (description updates etc)?
  3. The package will be updated on new bootstrapper versions. Would you expect it to be also updated for every MSIX release, or just the "lkg" versions?

thanks!

michael-franck commented 2 months ago

Mind if I pitch in on point 3. I would suggest to also update on MSIX releases.

Getting the version from the file should be fairly straightforward in Chocolatey.

Set-Alias -Name 7z -Value $Env:chocolateyInstall\tools\7z.exe
7z e <msixBundle> AppxManifest.xml
[xml]$manifest =  get-content .\AppxManifest.xml
$version = $manifest.Package.Identity.Version
flcdrg commented 2 months ago

@ItzLevvie do you happen to know if there are version-specific URLs for teamsbootstrapper.exe?

I'm not sure if there's inconsistencies in the CDN, but while working on updating this package, I noticed I'm getting different versions being offered when hitting https://statics.teams.cdn.office.net/production-teamsprovision/lkg/teamsbootstrapper.exe - sometimes the older version, sometimes the newer one. Obviously this doesn't play nice with Chocolatey's requirement to compare a checksum hash.

eg. image