404 error on msstore endpoint resulting in `Failed to update source: msstore`

mdanish-kh commented 2 months ago

Brief description of your issue

winget list reports me Failed when searching source; results will not be included: msstore. Looking in the CLI verbose logs, found that it gets a 404 when hitting msstore endpoint. Performing a source reset didn't help either.

Relevant log snippet

``` 2024-09-03 23:43:28.731 [REPO] Sending http GET request to: https://storeedgefd.dsx.mp.microsoft.com/v9.0/packageManifests/9NMW3C7204PX?Market=PK 2024-09-03 23:43:28.732 [REPO] Http GET request details: GET / HTTP/1.1 Content-Type: application/json User-Agent: winget-cli WindowsPackageManager/1.8.1911 DesktopAppInstaller/Microsoft.DesktopAppInstaller v1.23.1911.0 Version: 1.6.0 2024-09-03 23:43:28.974 [REPO] Response status: 404 2024-09-03 23:43:28.975 [REPO] Response details: HTTP/1.1 404 Not Found Access-Control-Allow-Credentials: true Access-Control-Allow-Origin: Access-Control-Expose-Headers: * Cache-Control: max-age=0, no-cache, no-store Connection: keep-alive Content-Length: 248 Content-Type: application/json; charset=utf-8 Date: Tue, 03 Sep 2024 18:43:31 GMT Expires: Tue, 03 Sep 2024 18:43:31 GMT MS-CV: fIMNwxgG4022fgfs.1 Pragma: no-cache Server: Microsoft-HTTPAPI/2.0 X-OSG-Served-By: Torus-EastAsia_LEGACY000AJ7_1.0.0.0 X-Segment-On: true {"code":"DataNotFound","data":[],"details":[],"innererror":{"code":"DataNotFound","data":[],"details":[],"message":"Product 9NMW3C7204PX is not present","source":"StoreEdgeFD"},"message":"Product 9NMW3C7204PX is not present","source":"StoreEdgeFD"} 2024-09-03 23:43:28.975 [FAIL] C:\__w\1\s\external\pkg\src\AppInstallerCommonCore\HttpClientHelper.cpp(187)\WindowsPackageManager.dll!00007FF8397F68E2: (caller: 00007FF8397F5BC9) Exception(1) tid(6e80) 8A150044 2024-09-03 23:43:28.975 [FAIL] C:\__w\1\s\external\pkg\src\AppInstallerRepositoryCore\CompositeSource.cpp(1127)\WindowsPackageManager.dll!00007FF8399009DA: (caller: 00007FF83983CE73) LogHr(1) tid(6e80) 8A150044 Msg:[C:\__w\1\s\external\pkg\src\AppInstallerCommonCore\HttpClientHelper.cpp(187)\WindowsPackageManager.dll!00007FF8397F68E2: (caller: 00007FF8397F5BC9) Exception(1) tid(6e80) 8A150044 ] 2024-09-03 23:43:28.975 [REPO] Failed to search source for correlation: msstore 2024-09-03 23:43:28.975 [REPO] Did not find Id [9NMW3C7204PX] in tracked source: msstore ```

Full log: WinGet-2024-09-03-23-43-25.211.log

Steps to reproduce

Not aware of any. Happens from time to time on my machine. Could be a CDN / my region issue

Expected behavior

Msstore source is updated successfully

Actual behavior

Failed when searching source; results will not be included: msstore

Environment

Windows Package Manager v1.8.1911
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.22631.4037
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.23.1911.0

Winget Directories
-------------------------------------------------------------------------------------------------------------------------------
Logs                               %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\DiagOutputDir
User Settings                      %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\settings.json
Portable Links Directory (User)    %LOCALAPPDATA%\Microsoft\WinGet\Links
Portable Links Directory (Machine) C:\Program Files\WinGet\Links
Portable Package Root (User)       %LOCALAPPDATA%\Microsoft\WinGet\Packages
Portable Package Root              C:\Program Files\WinGet\Packages
Portable Package Root (x86)        C:\Program Files (x86)\WinGet\Packages
Installer Downloads                %USERPROFILE%\Downloads

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale

Admin Setting                             State
--------------------------------------------------
LocalManifestFiles                        Enabled
BypassCertificatePinningForMicrosoftStore Enabled
InstallerHashOverride                     Enabled
LocalArchiveMalwareScanOverride           Disabled
ProxyCommandLineOptions                   Disabled
DefaultProxy                              Disabled

github-actions[bot] commented 2 months ago

Hi I'm an AI powered bot that finds similar issues based off the issue title.

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it. Thank you!

Open similar issues:

Failed when searching Source (#3419), similarity score: 0.81

Closed similar issues:

Failed when searching source; results will not be included: msstore (#4676), similarity score: 0.80
Winget source down - 0KB cache file from CDN (#3744), similarity score: 0.80
winget source update fails (#3721), similarity score: 0.79
Failed when searching source; results will not be included: msstore (#3579), similarity score: 0.79

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

mdanish-kh commented 2 months ago

Feel free to mark as https://github.com/microsoft/winget-cli/labels/Area-External if there's nothing WinGet can do here

Trenly commented 2 months ago

I believe it is package specific, as these are the relevant lines -

2024-09-03 23:43:28.730 [REPO] Finding available package from installed package using system reference search: Query:[none] Include:PackageFamilyName='42458pdfiumapp.freepdfreaderforadobeacrobat-freepd_41tabtrnk92zy'[Exact]
2024-09-03 23:43:28.730 [REPO]  ... searching source: msstore [StoreEdgeFD]
2024-09-03 23:43:28.730 [REPO] Performing search: Query:[none] Include:PackageFamilyName='42458pdfiumapp.freepdfreaderforadobeacrobat-freepd_41tabtrnk92zy'[Exact]
2024-09-03 23:43:28.731 [REPO]  ... setting latest tracking package to: 9NMW3C7204PX
2024-09-03 23:43:28.731 [REPO]  ... using tracking package: 9NMW3C7204PX
2024-09-03 23:43:28.731 [REPO] Search request meets optimized search criteria.
2024-09-03 23:43:28.731 [REPO] Sending http GET request to: https://storeedgefd.dsx.mp.microsoft.com/v9.0/packageManifests/9NMW3C7204PX?Market=PK
|
2024-09-03 23:43:28.974 [REPO] Response status: 404

winget search 9NMW3C7204PX -s msstore
An unexpected error occurred while executing the command:
0x8a150044 : The rest API endpoint is not found.

mdanish-kh commented 2 months ago

I was curious that if package metadata couldn't be found from source, then why didn't WinGet fallback to assigning it an ID of the form (MSIX|ARP)\<PackageFullName> as it does with other packages that don't seem to be available in a source

I tried to gain a deeper understanding by looking at the code, and though I can't understand all of it, I think what's happening is that the CLI tried to look into my StoreEdgeFD\installed.db (tracking index?) first to see if it could find a matching ID for the package. It did as I had the relevant package msstore ID in installed.db, and then it tried to use that ID to query the remote msstore source to get more info. The info wasn't available, so winget showed it as Failed when searching source; results will not be included: msstore, although it was only this one package that couldn't be found.

https://github.com/microsoft/winget-cli/blob/19b3b5ee94ea4fbb04691e7e45935fcbfb1b17b1/src/AppInstallerRepositoryCore/CompositeSource.cpp#L1504-L1536

If the msstore response for a package is Detail not found, is this an opportunity for the CLI to disregard the ID in installed.db (and also delete the entry? I see there's some logic related to timestamp but we want to delete it regardless)? That way WinGet would correctly treat the installed package as if it wasn't available from any source. And ideally, if only one package fails correlation, then it may be an opportunity to show that package in CLI output OR say that "Some packages could not be correlated from source: msstore"

@denelon This may not be https://github.com/microsoft/winget-cli/labels/Area-External if this is an opportunity for CLI improvement

microsoft / winget-cli