Azure / azure-sdk-for-go

This repository is for active development of the Azure SDK for Go. For consumers of the SDK we recommend visiting our public developer docs at:
https://docs.microsoft.com/azure/developer/go/
MIT License
1.64k stars 844 forks source link

[azcosmos] queryPager.NextPage(context.Background()) is throwing no host in url error #21491

Closed Muhilraaj closed 1 year ago

Muhilraaj commented 1 year ago

Bug Report

Package: github.com/Azure/azure-sdk-for-go/sdk/data/azcosmos Version : v0.3.6 Go version : 1.20.4 Machine Config: Windows/AMD64

Issue: After updating to newer version from 0.3.5 to 0.3.6, queryPager.NextPage(context.Background()) is throwing no host in url error.

`

    container, _ := client.NewContainer("db", "container")
pk := azcosmos.NewPartitionKeyNumber(1)

queryPager := container.NewQueryItemsPager("SELECT c['On-Behalf'] FROM c", pk, nil)
var result []map[string]interface{}

for queryPager.More() {
    queryResponse, err := queryPager.NextPage(context.Background())
    if err != nil {
        var responseErr *azcore.ResponseError
        errors.As(err, &responseErr)
        panic(responseErr)
    }

    for _, item := range queryResponse.Items {
        var itemResponseBody map[string]interface{}
        err = json.Unmarshal(item, &itemResponseBody)
        result = append(result, itemResponseBody)
        if err != nil {
            panic(err)
        }
    }
}`
ealsur commented 1 year ago

@Muhilraaj Can you please attach the error details? 0.3.6 has no changes related to query execution (https://github.com/Azure/azure-sdk-for-go/releases/tag/sdk%2Fdata%2Fazcosmos%2Fv0.3.6)

Muhilraaj commented 1 year ago

here it is: panic: no Host in request URL

goroutine 1 [running]: main.GetUser() C:/Users/muhil/Documents/Go Scripts/azcosmos/main.go:41 +0x616 main.main() C:/Users/muhil/Documents/Go Scripts/azcosmos/main.go:57 +0x17

here is where it is panicking queryResponse, err := queryPager.NextPage(context.Background()) if err != nil { var responseErr *azcore.ResponseError errors.As(err, &responseErr) panic(err) }

ealsur commented 1 year ago

@Muhilraaj Does this repro consistently? The project has CI/CD tests that perform real requests as part of any PR and on releases, all these tests are passing.

Are you initializing the client with a valid Endpoint URL?

ealsur commented 1 year ago

Also, if you revert to 0.3.5, does the issue go away?

Muhilraaj commented 1 year ago

Actually no, I tried reverting as well, it still didn't work. But also I didn't change anything with the endpoint url as well.

Muhilraaj commented 1 year ago

One more thing, I also observed similar issue with azblob data retrival , here is the code that I use

func GetLabel() { cred, _ := azblob.NewSharedKeyCredential(os.Getenv("Storage_Account_Name"), os.Getenv("Storage_Account_Key")) options := azblob.PipelineOptions{} u, _ := url.Parse(os.Getenv("BLOB_URL")) pipeline := azblob.NewPipeline(cred, options) url := azblob.NewBlobURL(*u, pipeline) ctx := context.Background() b, _ := url.Download(ctx, 0, azblob.CountToEnd, azblob.BlobAccessConditions{}, false, azblob.ClientProvidedKeyOptions{}) var label = make(map[string]interface{}) _ = json.NewDecoder(b.Body(azblob.RetryReaderOptions{})).Decode(&label) fmt.Println(label) }

In this line _ = json.NewDecoder(b.Body(azblob.RetryReaderOptions{})).Decode(&label) I am getting a no host url error. In both cases the connection went well but data read is causing issues

ealsur commented 1 year ago

If reverting does not fix it, it seems this is unrelated to the 0.3.6 release.

In both cases the connection went well but data read is causing issues

How are you certain the connection went well? Are you certain the Endpoint being used on the azblob or azcosmos clients is a valid URL and that the machine can resolve the DNS?

Muhilraaj commented 1 year ago

I tried the same url and key with python, It's working fine there.

ealsur commented 1 year ago

But is it the same machine? Can you run nslookup or some commandline tool with that DNS and does it work? If using the older library version faces the same problem then something else must have changed, the library code did not. Either some configuration is setting the wrong values or the machine cannot resolve the DNS records.

The panic seems to be coming from:

https://github.com/Azure/azure-sdk-for-go/blob/d1d0a7f803abdc1d18f204beaaf799c45521e442/sdk/azcore/internal/exported/request.go#L77-L79

For azcosmos, the URL is exactly what you put when you call NewClient:

https://github.com/Azure/azure-sdk-for-go/blob/d1d0a7f803abdc1d18f204beaaf799c45521e442/sdk/data/azcosmos/cosmos_client.go#L38-L40

https://github.com/Azure/azure-sdk-for-go/blob/d1d0a7f803abdc1d18f204beaaf799c45521e442/sdk/data/azcosmos/cosmos_client.go#L376-L382

So it probably means the endpoint might be an empty string? Have you tried logging that value to see what it is?

Muhilraaj commented 1 year ago

Sorry, my bad, the issue is on my end. I use a middleware code to get endpoint url and it's causing issue.

We can close this issue.