UTF-8 string character can never start with 10xxxxxx

Brief description of your issue

Tried installing a package in Windows Sandbox, got an error that there was a failure when searching MSStore source

Steps to reproduce

Run winget install <id> in Windows Sandbox

Expected behavior

The search to succeed and package to be installed from winget source

Actual behavior

Failed when searching source: msstore
An unexpected error occurred while executing the command:
UTF-8 string character can never start with 10xxxxxx

WinGet-2024-03-29-12-36-10.899.log

As a note - this seems to only affect Windows Sandbox (at least in my environment) and only affects winget install and winget uninstall

PS C:\Users\WDAGUtilityAccount\Desktop\winget-pkgs> winget search Microsoft.VCLibs.Desktop.14
The `msstore` source requires that you view the following agreements before using.
Terms of Transaction: https://aka.ms/microsoft-store-terms-of-transaction
The source requires the current machine's 2-letter geographic region to be sent to the backend service to function properly (ex. "US").

Do you agree to all the source agreements terms?
[Y] Yes  [N] No: y
Failed when searching source; results will not be included: msstore
Name                                                  Id                          Version      Source
------------------------------------------------------------------------------------------------------
Microsoft Visual C++ 2015 UWP Desktop Runtime Package Microsoft.VCLibs.Desktop.14 14.0.33321.0 winget
PS C:\Users\WDAGUtilityAccount\Desktop\winget-pkgs> winget install Microsoft.VCLibs.Desktop.14
Failed when searching source: msstore
An unexpected error occurred while executing the command:
UTF-8 string character can never start with 10xxxxxx

The following packages were found among the working sources.
Please specify one of them using the --source option to proceed.
Name                                                  Id                          Source
----------------------------------------------------------------------------------------
Microsoft Visual C++ 2015 UWP Desktop Runtime Package Microsoft.VCLibs.Desktop.14 winget

Environment

PS C:\Users\WDAGUtilityAccount\Desktop\winget-pkgs> winget --info
Windows Package Manager v1.7.10861
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.26085.1
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.22.10861.0

Winget Directories
-------------------------------------------------------------------------------------------------------------------------------
Logs                               %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\DiagOutputDir
User Settings                      %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\settings.json
Portable Links Directory (User)    %LOCALAPPDATA%\Microsoft\WinGet\Links
Portable Links Directory (Machine) C:\Program Files\WinGet\Links
Portable Package Root (User)       %LOCALAPPDATA%\Microsoft\WinGet\Packages
Portable Package Root              C:\Program Files\WinGet\Packages
Portable Package Root (x86)        C:\Program Files (x86)\WinGet\Packages
Installer Downloads                %USERPROFILE%\Downloads

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale

Admin Setting                             State
--------------------------------------------------
LocalManifestFiles                        Enabled
BypassCertificatePinningForMicrosoftStore Disabled
InstallerHashOverride                     Disabled
LocalArchiveMalwareScanOverride           Enabled

After more investigation, this occurs only in the sandbox because the Region in the sandbox is set to World. This causes the PackageMatchField for Market to be -

{"PackageMatchField":"Market","RequestMatch":{"KeyWord":"ZZ","MatchType":"CaseInsensitive"}}

This in turn causes the msstore source to return the following 400 response -

{
    "code": "InvalidParameterValue",
    "data": [],
    "details": [],
    "innererror": {
        "code": "InvalidParameterValue",
        "data": [],
        "details": [],
        "message": "Parameter market was specified with invalid value: ZZ.",
        "source": "StoreEdgeFD",
        "target": "market"
    },
    "message": "Parameter market was specified with invalid value: ZZ.",
    "source": "StoreEdgeFD",
    "target": "market"
}

Manually setting the region inside the sandbox causes the error to go away. The same can be verified in PowerShell (or PostMan)

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Content-Type", "application/json")

$body = @"
{`"Filters`":[{`"PackageMatchField`":`"Market`",`"RequestMatch`":{`"KeyWord`":`"ZZ`",`"MatchType`":`"CaseInsensitive`"}}],`"Inclusions`":[{`"PackageMatchField`":`"PackageName`",`"RequestMatch`":{`"KeyWord`":`"Xodo`",`"MatchType`":`"Substring`"}}]}
"@

$response = Invoke-RestMethod 'https://storeedgefd.dsx.mp.microsoft.com/v9.0/manifestSearch' -Method 'POST' -Headers $headers -Body $body

$body = @"
{`"Filters`":[{`"PackageMatchField`":`"Market`",`"RequestMatch`":{`"KeyWord`":`"US`",`"MatchType`":`"CaseInsensitive`"}}],`"Inclusions`":[{`"PackageMatchField`":`"PackageName`",`"RequestMatch`":{`"KeyWord`":`"Xodo`",`"MatchType`":`"Substring`"}}]}
"@

$response = Invoke-RestMethod 'https://storeedgefd.dsx.mp.microsoft.com/v9.0/manifestSearch' -Method 'POST' -Headers $headers -Body $body
$response | ConvertTo-Json

@denelon - Seems like this is an issue with the msstore source then, and not necessarily the sandbox. Would you mind updating the tags?

microsoft / winget-cli