zscaler / zscaler-sdk-go

Zscaler SDK for Golang (ZPA/ZIA/ZDX/ZCC APIs)
https://zscaler.com
MIT License
10 stars 1 forks source link

ZDX applications GetAllApps endpoint bug #257

Closed bosatsu closed 3 months ago

bosatsu commented 3 months ago

Confirmation

zscaler-sdk-go version

v2.6.0

Go environment

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/spage/Library/Caches/go-build'
GOENV='/Users/spage/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/spage/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/spage/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.22.3/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.22.3/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.3'
GCCGO='gccgo'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/Users/spage/Developer/sandbox/sandbox-go/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/3j/r7v71px131x4578dxnm_7y_w0000gn/T/go-build1124777475=/tmp/go-build -gno-record-gcc-switches -fno-common'

Expected output

Object returned from GetAllApps endpoint is an *applications.Apps struct, per the documentation. However I think this is likely incorrect and should be a slice of applications.Apps structs, since this endpoint should retrieve all apps defined in ZDK. I'm new to using Go though, so I may be reading something incorrectly. However there is another issue that is happening before we get to this part of the code and I will describe this further down.

Actual output

Object returned from GetAllApps endpoint is nil

Catching and printing the error returned from GetAllApps reads:

json: cannot unmarshal array into Go value of type applications.Apps

Code demonstrating the issue

package main

import (
    "log"
    "os"

    "github.com/zscaler/zscaler-sdk-go/v2/zdx"
    "github.com/zscaler/zscaler-sdk-go/v2/zdx/services/reports/applications"
)

func main() {

    // Init zscaler zdx config
    apiKey := os.Getenv("ZDX_API_KEY_ID")
    apiSecret := os.Getenv("ZDX_API_SECRET")
    cfg, err := zdx.NewConfig(apiKey, apiSecret, "userAgent")
    if err != nil {
        log.Printf("[ERROR] creating client failed: %v\n", err)
        return
    }
    cli := zdx.NewClient(cfg)

    appsSDKEndpoint(cli)

}

func appsSDKEndpoint(cli *zdx.Client) {
    // Test getting all apps from "GetAllApps" endpoint
    appClient := applications.New(cli)
    _, _, err := appClient.GetAllApps(applications.GetAppsFilters{})
    if err != nil {
        log.Printf("SDK Endpoint Apps: [ERROR] using GetAllApps endpoint %v\n", err)
    }

}

Steps to reproduce

Run the code above.

References

No response

bosatsu commented 3 months ago

I want to enrich the issue above, because it seems like ZDK Rest API server is giving inconsistent results for GET operations against the v1/apps endpoint when using the Zscaler SDK.

Using the following env vars:

export ZSCALER_SDK_LOG=true
export ZSCALER_SDK_VERBOSE=true

If I run the following code using the client NewRequestDo utility endpoint:

package main

import (
    "log"
    "net/http"
    "os"

    "github.com/zscaler/zscaler-sdk-go/v2/zdx"
    "github.com/zscaler/zscaler-sdk-go/v2/zdx/services/common"
)

func main() {

    // Init zscaler zdx config
    apiKey := os.Getenv("ZDX_API_KEY_ID")
    apiSecret := os.Getenv("ZDX_API_SECRET")
    cfg, err := zdx.NewConfig(apiKey, apiSecret, "userAgent")
    if err != nil {
        log.Printf("[ERROR] creating client failed: %v\n", err)
        return
    }
    cli := zdx.NewClient(cfg)

    // Test pulling all apps using utility function
    resp, err := cli.NewRequestDo(http.MethodGet, "v1/apps", common.GetFromToFilters{}, nil, nil)
    if err != nil {
        log.Printf("[ERROR] using NewRequestDo %v\n", err)
    }
    defer resp.Body.Close()

}

I get the following zscaler log output:

zdx-logger: 2024/06/10 15:29:43 logger.go:36: [TRACE] Getting access token for ZDX_API_KEY_ID=(OMITTED)
zdx-logger: 2024/06/10 15:29:43 logger.go:36: [DEBUG] POST https://api.zdxcloud.net/v1/oauth/token
zdx-logger: 2024/06/10 15:29:44 logger.go:36: [DEBUG] Request "GET https://api.zdxcloud.net/v1/apps" details:
---[ ZSCALER SDK REQUEST | ID:a2770df1-c90b-44d3-be21-90d033e15913 ]-------------------------------
GET /v1/apps HTTP/1.1
Host: api.zdxcloud.net
User-Agent: userAgent
Authorization: Bearer (OMITTED)
Content-Type: application/json
Accept-Encoding: gzip

---------------------------------------------------------
zdx-logger: 2024/06/10 15:29:44 logger.go:36: [DEBUG] GET https://api.zdxcloud.net/v1/apps
zdx-logger: 2024/06/10 15:29:45 logger.go:36: [DEBUG] Response "GET https://api.zdxcloud.net/v1/apps" details:
---[ ZSCALER SDK RESPONSE | ID:a2770df1-c90b-44d3-be21-90d033e15913 | Duration:1.531663917s ]--------------------------------
HTTP/2.0 200 OK
Ccid: zscalertwo.net.51057371
Content-Type: application/json
Date: Mon, 10 Jun 2024 22:29:44 GMT
Ratelimit-Limit: 5
Ratelimit-Remaining: 4
Ratelimit-Reset: 1
Vary: Accept-Encoding, User-Agent
X-Api-Version: 2.1.0-2023-12-22T00:27:16Z
X-Ratelimit-Limit-Day: 10000
X-Ratelimit-Limit-Hour: 1000
X-Ratelimit-Limit-Minute: 30
X-Ratelimit-Limit-Second: 5
X-Ratelimit-Remaining-Day: 9702
X-Ratelimit-Remaining-Hour: 991
X-Ratelimit-Remaining-Minute: 28
X-Ratelimit-Remaining-Second: 4

(JSON RESPONSE OMITTED HERE, BUT WE RECEIVE THE EXPECTED JSON OUTPUT OF ALL APPLICATIONS DEFINED IN ZDX)
-------------------------------------------------------

However if I run this code using the applications GetAllApps endpoint:

package main

import (
    "log"
    "os"

    "github.com/zscaler/zscaler-sdk-go/v2/zdx"
    "github.com/zscaler/zscaler-sdk-go/v2/zdx/services/reports/applications"
)

func main() {

    // Init zscaler zdx config
    apiKey := os.Getenv("ZDX_API_KEY_ID")
    apiSecret := os.Getenv("ZDX_API_SECRET")
    cfg, err := zdx.NewConfig(apiKey, apiSecret, "userAgent")
    if err != nil {
        log.Printf("[ERROR] creating client failed: %v\n", err)
        return
    }
    cli := zdx.NewClient(cfg)

    // Test getting all apps from "GetAllApps" endpoint
    appClient := applications.New(cli)
    _, _, err = appClient.GetAllApps(applications.GetAppsFilters{})
    if err != nil {
        log.Printf("SDK Endpoint Apps: [ERROR] using GetAllApps endpoint %v\n", err)
    }

}

It doesn't seem like the ZDX REST API is giving a response:

zdx-logger: 2024/06/10 15:43:32 logger.go:36: [TRACE] Getting access token for ZDX_API_KEY_ID=(OMITTED)
zdx-logger: 2024/06/10 15:43:32 logger.go:36: [DEBUG] POST https://api.zdxcloud.net/v1/oauth/token
zdx-logger: 2024/06/10 15:43:34 logger.go:36: [DEBUG] Request "GET https://api.zdxcloud.net/v1/apps" details:
---[ ZSCALER SDK REQUEST | ID:0894c565-126c-4973-ab99-09007b655a4f ]-------------------------------
GET /v1/apps HTTP/1.1
Host: api.zdxcloud.net
User-Agent: userAgent
Authorization: Bearer (OMITTED)
Content-Type: application/json
Accept-Encoding: gzip

---------------------------------------------------------

It probably seems like I forgot something in the output above, but that is literally the only debug information that is printed to the terminal.

At this point if feels like I'm using the SDK wrong, but if I look at the SDK log information, I can't get past the lack of response returned. The GET request looks to be the same for both. What am I missing?

willguibr commented 3 months ago

@bosatsu There were some API changes with ZDX lately, and didn't get reflected in the SDK struct. We'll be putting out a new release in the next few days. In the meantime, if you cloned the repo, and navigate the package here you can use the code you've provided and the call should succeed without problems.

---------------------------------------------------------
zdx-logger: 2024/06/11 16:28:26 logger.go:36: [DEBUG] GET https://api.zdxcloud.net/v1/apps
zdx-logger: 2024/06/11 16:28:27 logger.go:36: [DEBUG] Response "GET https://api.zdxcloud.net/v1/apps" details:
---[ ZSCALER SDK RESPONSE | ID:b047bf70-4ae7-46dc-8866-7bac65c36a39 | Duration:743.022417ms ]--------------------------------
HTTP/2.0 200 OK
Ccid: zscalerthree.net.24326813
Content-Type: application/json
Date: Tue, 11 Jun 2024 23:28:26 GMT
Ratelimit-Limit: 5
Ratelimit-Remaining: 4
Ratelimit-Reset: 1
Vary: Accept-Encoding, User-Agent
X-Api-Version: 2.1.0-2023-12-22T00:27:16Z
X-Ratelimit-Limit-Day: 10000
X-Ratelimit-Limit-Hour: 1000
X-Ratelimit-Limit-Minute: 30
X-Ratelimit-Limit-Second: 5
X-Ratelimit-Remaining-Day: 9834
X-Ratelimit-Remaining-Hour: 991
X-Ratelimit-Remaining-Minute: 29
X-Ratelimit-Remaining-Second: 4

[{"id":1,"name":"SharePoint Online","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":2,"name":"OneDrive for Business","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":3,"name":"Outlook Online","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":5,"name":"ServiceNow","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":6,"name":"Box","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":10,"name":"Microsoft Teams Web App","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":19,"name":"Microsoft Login","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":12949,"name":"Outlook UDP","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":13279,"name":"Zoom2","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1},{"id":13493,"name":"AWS","score":0.0,"most_impacted_region":{"id":"0.0.CA","country":"Canada"},"total_users":1}]
-------------------------------------------------------
bosatsu commented 3 months ago

@willguibr

Thanks so much for looking into this and the quick turn around. I'll keep an eye for the new release.

willguibr commented 3 months ago

@bosatsu Please check release v2.61.0 for ZDX package improvements.

bosatsu commented 3 months ago

@willguibr Thank you for the quick turn around on this! Everything is working so far on the device report endpoints. If I could ask another favor, it would be to update the READMEs and examples for the zdx client, I had to go back and read the PR to figure out that we have to pass in a service to the report endpoint functions instead of initialing the service based on the report endpoint. Hopefully that made sense. Anyways, thank you for getting this fix in such a timely manner.

willguibr commented 2 months ago

https://github.com/zscaler/zscaler-sdk-go/releases/tag/v2.61.1#:~:text=on%20StatusPreconditionFailed%20412.-,Documentation,-PR%20%23259%20%2D%20Added @bosatsu 👆🏽

bosatsu commented 2 months ago

@willguibr Thank you!