search

Azure / aksArc

# Welcome to the Azure Kubernetes Service on Azure Stack HCI repo This is where the AKS-HCI team will track features and issues with AKS-HCI. We will monitor this repo in order to engage with our community and discuss questions, customer scenarios, or feature requests. Checkout our projects tab to see the roadmap for AKS-HCI!
MIT License
109 stars 45 forks source link

[BUG] Nondescript error with 'github.com/vbauerster/getparty' / 'msazure.visualstudio.com/msazure/msk8s/downloadsdk.git' #360

Open markamber opened 7 months ago

markamber commented 7 months ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Try on Windows Admin Center to make a cluster, and it failed with Applying platform configurations failed. Error: Processing data from remote server dellr730-01.ad.splchicago.com failed with the following error message: The WSMan provider host process did not return a proper response. A provider in the host process may have behaved improperly. For more information, see the about_Remote_Troubleshooting Help topic. so I did a bit more digging and learned that I can do it in powershell. And I now know the error is because the process actually kills powershell, not just simply failing which explains why Windows Admin Center sees some sort of generic powershell error.
  2. Try to do it on powershell via https://learn.microsoft.com/en-us/azure/aks/hybrid/kubernetes-walkthrough-powershell
  3. After I run the command in step 3 there is an error and I did a lot of googling and I do not see anything about this error - There is actually an issue on this github which I believe is unrelated #347 - but that has a more specific error message and I tried the solution to no avail.
  4. See error

Expected behavior I expect something more to happen. I expect at least an error message

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

Collect log files I collected the logs, it is very large and I am unsure if there is any PII inside it. Below is the console logs

PS C:\Windows\system32> Set-AksHciConfig -imageDir c:\clusterstorage\volume1\Images -workingDir c:\ClusterStorage\Volume
1\ImageStore -cloudConfigLocation c:\clusterstorage\volume1\Config -vnet $vnet -cloudservicecidr "192.168.10.34/24"
True
DownloadSdk Tests Execution Started 12.11.2023 16:49:11

 Test-KVASettings
    Kva - Downloading image Linux_k8s_1-26-6_1.0.21.11030

Connection to https://msk8s.api.cdp.microsoft.com ... Succeeded
Connection to http://msk8s.b.tlu.dl.delivery.mp.microsoft.com ... Succeeded

Host is able to reach list of URLs requirements

Test Succeeded
Details: Host is able to reach list of URLs requirements
Recommendation:
Test execution time: 271.7319 milliseconds

DownloadSdk Tests Execution Ended 12.11.2023 16:49:11

=====================================================
       All DownloadSdk Validation tests are successful
=====================================================

Check the test report(downloadsdk_validation_report.html) in current directory

Kva Tests Execution Started 12.11.2023 16:49:11
===============================================================================

Test (1 of 1): "Validate KVA". Category: KVA

panic: totalWritten != stat.Size()

goroutine 147 [running]:
github.com/vbauerster/getparty.Session.concatenateParts({{0x1c000182700, 0xf3}, {0x1c000182700, 0xf3}, {0x1c00007a5b9, 0x67}, {0x0, 0x0}, {0x1c0001a2cd0, 0x5}, ...}, ...)
        /home/vsts/go/pkg/mod/github.com/vbauerster/getparty@v1.19.5-0.20231024204429-dbaf1ad02b99/session.go:146 +0x15a5
github.com/vbauerster/getparty.(*Cmd).Run(0x1c000601da0, {0x1c00015e200, 0x5, 0x8}, {0x0, 0x0}, {0x0, 0x0})
        /home/vsts/go/pkg/mod/github.com/vbauerster/getparty@v1.19.5-0.20231024204429-dbaf1ad02b99/getparty.go:318 +0x1d85
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/http.DownloadFile({0x7fff0a0edba8?, 0x1c00009f320}, {0x1c000182700, 0xf3}, {0x1c00007a4d0, 0x67}, 0xa, {0x0, 0x0}, 0x1c00000a100)
        /home/vsts/work/1/s/sdk/pkg/http/http.go:85 +0x72f
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/sfsclient.(*SFSClient).GetVerifiedFiles(0x1c0000a81b0, {0x7fff0a0edba8, 0x1c00009f320}, {{{0x1c00001c820, 0x19}, {0x7fff0a01c27a, 0x7}, {0x1c0005cc874, 0xc}}}, {0x1c000022100, ...}, ...)
        /home/vsts/work/1/s/sdk/pkg/sfsclient/sfsclient.go:187 +0x85b
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/download/provider/sfs.(*sfsProvider).getReleaseInternal(0x1c0000a81c8, {0x7fff0a0edba8, 0x1c00009f320}, {0x1c00001c820?, 0x0?}, {0x1c0005cc874?, 0x0?}, {0x1c000022100, 0x38}, 0xa, ...)
        /home/vsts/work/1/s/sdk/pkg/download/provider/sfs/release.go:49 +0x454
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/download/provider/sfs.(*sfsProvider).GetRelease(0x0?, {0x7fff0a0edba8?, 0x1c00009f320?}, {0x1c00001c820?, 0x7fff0a2d6b80?}, {0x1c0005cc874?, 0xa?}, {0x0?, 0x0?}, {0x0, ...}, ...)
        /home/vsts/work/1/s/sdk/pkg/download/provider/sfs/release.go:16 +0x96
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/download.(*downloadClient).getRelease(0x1c000308cc0, {0x7fff0a0edba8, 0x1c00009f320}, {{0x1c00001c820, 0x19}, {0x1c0005cc874, 0xc}, {0x0, 0x0}, {0x0, ...}, ...})
        /home/vsts/work/1/s/sdk/pkg/download/release.go:73 +0x2b0
msazure.visualstudio.com/msazure/msk8s/downloadsdk.git/sdk/pkg/download.(*downloadClient).GetRelease(0x1c000519eb0?, {0x7fff0a0edba8?, 0x1c00009f320?}, {{0x1c00001c820, 0x19}, {0x1c0005cc874, 0xc}, {0x0, 0x0}, {0x0, ...}, ...})
        /home/vsts/work/1/s/sdk/pkg/download/release.go:56 +0x13d
main.GetRelease.func2()
        /home/vsts/work/1/s/sdk/main.go:175 +0x97
created by main.GetRelease
        /home/vsts/work/1/s/sdk/main.go:174 +0x5ec

C:\Windows\system32>powershellSet-AksHciConfig -offlineDownload $true -mode full -stagingShare c:\akshciimages -imageDir c:\clusterstorage\volume1\Images -workingDir c:\ClusterStorage\Volume1\ImageStore -cloudConfigLocation c:\clusterstorage\volume1\Config -vnet $vnet -cloudservicecidr "192.168.10.34/24" -verbose
'powershellSet-AksHciConfig' is not recognized as an internal or external command,
operable program or batch file.

C:\Windows\system32>
markamber commented 7 months ago

I seem to have partially resolved this.

I noticed that the paths I was using, while specifically called out in the documentation, do not work. You have to put things in C:\clusterstorage\volume1\akshci\{images/imagestore/config}

Now that I did I can reproduce #363

But to me this issue is still a separate bug unless documentation was added to say that custom paths (or inseed, the path listed in the documentation) do not work now.

abhilashaagarwala commented 7 months ago

@SummerSmith @walterov FYI

markamber commented 7 months ago

This one has come back with a vengeance. Idk if I even figured out the problem. I actually just delved into getting openshift today (which is like 2-5x as expensive) so I won't be looking into this more but between this and the other bug I reproduced on separate hardware/setup I'm not entirely sure it's possible to stand up any aks hybrid at the moment. Perhaps you can share your reference deployment. I mean I might be willing to give it a second look but not likely.

Elektronenvolt commented 7 months ago

@markamber - this is a log from WAC? I'm just wondering about this: image

If setup over WAC fails for you connect to a host and use Powershell: https://learn.microsoft.com/en-us/azure/aks/hybrid/kubernetes-walkthrough-powershell That's still my preferred way to setup until https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-hybrid-preview-overview is fully released.

markamber commented 7 months ago

@markamber - this is a log from WAC? I'm just wondering about this:

image

If setup over WAC fails for you connect to a host and use Powershell: https://learn.microsoft.com/en-us/azure/aks/hybrid/kubernetes-walkthrough-powershell

That's still my preferred way to setup until https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-hybrid-preview-overview is fully released.

No as I mentioned in my post I did start with WAC but moved to the powershell per the document I linked. As you can see there I just randomly typed powershell (because the command literally crashed powershell, so I was trying to re-enter) and then probably clicked the right mouse button with a string of text that had a new line character and included it in my post for no particular reason.