http panic serving - Githubissues

torblerone commented 4 years ago

Hello, I've tried your AdminIO setup in Kubernetes and getting some problems. Here is my setup:

MinIO:

Version
2020-10-03T02:19:42Z
Platform
Host: minio-0 | OS: linux | Arch: amd64
Runtime
Version: go1.14.9 | CPUs: 4

AdminIO-API: latest Config:

- name: ADMINIO_HOST_PORT
  value: 0.0.0.0:8080
- name: MINIO_HOST
  valueFrom:
    secretKeyRef:
      name: s3-secret
      key: host
- name: MINIO_HOST_PORT
  value: $(MINIO_HOST)
- name: MINIO_ACCESS
  valueFrom:
    secretKeyRef:
       name: s3-secret
       key: testu
- name: MINIO_SECRET
  valueFrom:
     secretKeyRef:
        name: s3-secret
        key: testp
- name: ADMINIO_PROBES_ENABLE
  value: "true"
- name: ADMINIO_METRIC_ENABLE
  value: "false"
- name: MINIO_SSL
  value: "false"

Whenever I call the UI (or curl the API on my MINIO_HOST) I'm getting errors in the API logs which always look like this:

[HTTP Server] http: panic serving xxx.xxx.xxx.xxx:xxxxx: runtime error: invalid memory address or nil pointer dereference
goroutine 58 [running]:
net/http.(*conn).serve.func1(0xc00021e640)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/minio/minio/pkg/madmin.(*AdminClient).DataUsageInfo(0x0, 0x10aa6c0, 0xc0000b8020, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/minio/minio@v0.0.0-20200825193859-9acdeab73d7e/pkg/madmin/info-commands.go:137 +0x78
github.com/rzrbld/adminio-api/handlers.glob..func32(0xc00011cea0)
    /go/adminio-api/src/handlers/server.go:16 +0x7a
github.com/kataras/iris/v12/context.(*Context).Next(0xc00011cea0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc000510000, 0xc00011cea0)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000505950, 0xc00011cea0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc000574380, 0xc000136800)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc0000bc660, 0x10a7080, 0xc000574380, 0xc000136800)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00014a000, 0x10a7080, 0xc000574380, 0xc000136800)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc00021e640, 0x10aa680, 0xc000166880)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c

Do you know any fixes for this problem? Thank you in advance!

rzrbld commented 4 years ago

@torblerone hi, i'll try to look at this problem at weekend.

torblerone commented 4 years ago

Thank you @rzrbld if you need more information on any of my setup, I'll be reachable at weekend

rzrbld commented 4 years ago

@torblerone sadly i can't reproduce this behaviour :confused: but i prepared helm charts can you please deploy it and check is problem still exist?

torblerone commented 4 years ago

I will test your solution in the morning. Thank you in advance!

torblerone commented 4 years ago

Hello, I've been trying to get your helm charts to work. I've been dry-running them to get the YAMLs and edited them in several ways to fit my k8s cluster. I'm getting the following error when metrics are enabled for adminio-api:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xe29c18]

goroutine 52 [running]:
github.com/minio/minio/pkg/madmin.(*AdminClient).DataUsageInfo(0x0, 0x11d94a0, 0xc000038040, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /go/pkg/mod/github.com/minio/minio@v0.0.0-20201010033600-c2f16ee84624/pkg/madmin/info-commands.go:137 +0x78
github.com/rzrbld/adminio-api/handlers.RecordMetrics.func1()
        /go/adminio-api/src/handlers/metrics.go:35 +0x6f
created by github.com/rzrbld/adminio-api/handlers.RecordMetrics
        /go/adminio-api/src/handlers/metrics.go:33 +0x35

Setting ADMINIO_METRICS_ENABLE: "false" clears this error. I'll continue reporting when I find more.

torblerone commented 4 years ago

Update: I'm getting the same errors again.

AdminIO-API (release-1.4) log:

[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 99 [running]:
net/http.(*conn).serve.func1(0xc00017e640)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/rzrbld/adminio-api/handlers.glob..func6(0xc0005c2240)
    /go/adminio-api/src/handlers/buckets.go:89 +0x4c
github.com/kataras/iris/v12/context.(*Context).Next(0xc0005c2240)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc0005c2240)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc0005c2240)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc00019c700, 0xc00056eb00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc00019c700, 0xc00056eb00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc00019c700, 0xc00056eb00)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc00017e640, 0x10aa680, 0xc000175980)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c
[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 83 [running]:
net/http.(*conn).serve.func1(0xc00028e460)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/minio/minio/pkg/madmin.(*AdminClient).ServerInfo(0x0, 0x10aa6c0, 0xc00003a028, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/minio/minio@v0.0.0-20200825193859-9acdeab73d7e/pkg/madmin/info-commands.go:291 +0x83
github.com/rzrbld/adminio-api/handlers.glob..func31(0xc0005ea120)
    /go/adminio-api/src/handlers/server.go:10 +0x94
github.com/kataras/iris/v12/context.(*Context).Next(0xc0005ea120)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc0005ea120)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc0005ea120)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc0005f09a0, 0xc0005e8900)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc0005f09a0, 0xc0005e8900)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc0005f09a0, 0xc0005e8900)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc00028e460, 0x10aa680, 0xc0001b9300)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c
[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 85 [running]:
net/http.(*conn).serve.func1(0xc00028e500)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/minio/minio/pkg/madmin.(*AdminClient).ServerInfo(0x0, 0x10aa6c0, 0xc00003a028, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/minio/minio@v0.0.0-20200825193859-9acdeab73d7e/pkg/madmin/info-commands.go:291 +0x83
github.com/rzrbld/adminio-api/handlers.glob..func31(0xc0005eb200)
    /go/adminio-api/src/handlers/server.go:10 +0x94
github.com/kataras/iris/v12/context.(*Context).Next(0xc0005eb200)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc0005eb200)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc0005eb200)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc0005f0a80, 0xc0005e8a00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc0005f0a80, 0xc0005e8a00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc0005f0a80, 0xc0005e8a00)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc00028e500, 0x10aa680, 0xc0001b9480)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c
[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 76 [running]:
net/http.(*conn).serve.func1(0xc0000e8820)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/rzrbld/adminio-api/handlers.glob..func6(0xc00024cea0)
    /go/adminio-api/src/handlers/buckets.go:89 +0x4c
github.com/kataras/iris/v12/context.(*Context).Next(0xc00024cea0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc00024cea0)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc00024cea0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc00025e9a0, 0xc000166e00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc00025e9a0, 0xc000166e00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc00025e9a0, 0xc000166e00)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc0000e8820, 0x10aa680, 0xc000112bc0)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c
[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 79 [running]:
net/http.(*conn).serve.func1(0xc0000e8960)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/rzrbld/adminio-api/handlers.glob..func6(0xc00024ddd0)
    /go/adminio-api/src/handlers/buckets.go:89 +0x4c
github.com/kataras/iris/v12/context.(*Context).Next(0xc00024ddd0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc00024ddd0)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc00024ddd0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc00025ea80, 0xc000166f00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc00025ea80, 0xc000166f00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc00025ea80, 0xc000166f00)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc0000e8960, 0x10aa680, 0xc000112cc0)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c
[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 78 [running]:
net/http.(*conn).serve.func1(0xc0000e88c0)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/minio/minio/pkg/madmin.(*AdminClient).ServerInfo(0x0, 0x10aa6c0, 0xc00003a028, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/minio/minio@v0.0.0-20200825193859-9acdeab73d7e/pkg/madmin/info-commands.go:291 +0x83
github.com/rzrbld/adminio-api/handlers.glob..func31(0xc0005c2ab0)
    /go/adminio-api/src/handlers/server.go:10 +0x94
github.com/kataras/iris/v12/context.(*Context).Next(0xc0005c2ab0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc0005c2ab0)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc0005c2ab0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc00019c7e0, 0xc00056ec00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc00019c7e0, 0xc00056ec00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc00019c7e0, 0xc00056ec00)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc0000e88c0, 0x10aa680, 0xc000175b80)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c
[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 102 [running]:
net/http.(*conn).serve.func1(0xc00017e6e0)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/minio/minio/pkg/madmin.(*AdminClient).DataUsageInfo(0x0, 0x10aa6c0, 0xc00003a028, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/minio/minio@v0.0.0-20200825193859-9acdeab73d7e/pkg/madmin/info-commands.go:137 +0x78
github.com/rzrbld/adminio-api/handlers.glob..func32(0xc0005c2c60)
    /go/adminio-api/src/handlers/server.go:16 +0x7a
github.com/kataras/iris/v12/context.(*Context).Next(0xc0005c2c60)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc0005c2c60)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc0005c2c60)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc00019c8c0, 0xc00056ed00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc00019c8c0, 0xc00056ed00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc00019c8c0, 0xc00056ed00)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc00017e6e0, 0x10aa680, 0xc000175d80)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c
[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 87 [running]:
net/http.(*conn).serve.func1(0xc00028e5a0)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/minio/minio/pkg/madmin.(*AdminClient).DataUsageInfo(0x0, 0x10aa6c0, 0xc00003a028, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/minio/minio@v0.0.0-20200825193859-9acdeab73d7e/pkg/madmin/info-commands.go:137 +0x78
github.com/rzrbld/adminio-api/handlers.glob..func32(0xc0005eb3b0)
    /go/adminio-api/src/handlers/server.go:16 +0x7a
github.com/kataras/iris/v12/context.(*Context).Next(0xc0005eb3b0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc0005eb3b0)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc0005eb3b0)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc0005f0b60, 0xc0005e8b00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc0005f0b60, 0xc0005e8b00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc0005f0b60, 0xc0005e8b00)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc00028e5a0, 0x10aa680, 0xc0001b9700)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c
[HTTP Server] http: panic serving XXXXXXXXXX: runtime error: invalid memory address or nil pointer dereference
goroutine 104 [running]:
net/http.(*conn).serve.func1(0xc00017e780)
    /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xe40f40, 0x16b71e0)
    /usr/local/go/src/runtime/panic.go:975 +0x3e9
github.com/minio/minio/pkg/madmin.(*AdminClient).DataUsageInfo(0x0, 0x10aa6c0, 0xc00003a028, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go/pkg/mod/github.com/minio/minio@v0.0.0-20200825193859-9acdeab73d7e/pkg/madmin/info-commands.go:137 +0x78
github.com/rzrbld/adminio-api/handlers.glob..func32(0xc0005c2e10)
    /go/adminio-api/src/handlers/server.go:16 +0x7a
github.com/kataras/iris/v12/context.(*Context).Next(0xc0005c2e10)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:510 +0x5c
github.com/iris-contrib/middleware/cors.(*Cors).Serve(0xc00028e000, 0xc0005c2e10)
    /go/pkg/mod/github.com/iris-contrib/middleware/cors@v0.0.0-20200810001613-32cf668f999f/cors.go:189 +0x165
github.com/kataras/iris/v12/context.(*Context).Do(...)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/context/context.go:392
github.com/kataras/iris/v12/core/router.(*routerHandler).HandleRequest(0xc000283a40, 0xc0005c2e10)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/handler.go:406 +0x21a
github.com/kataras/iris/v12/core/router.(*Router).BuildRouter.func3(0x10a7080, 0xc00019c9a0, 0xc00056ee00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:171 +0x82
github.com/kataras/iris/v12/core/router.(*Router).ServeHTTP(0xc000098300, 0x10a7080, 0xc00019c9a0, 0xc00056ee00)
    /go/pkg/mod/github.com/kataras/iris/v12@v12.1.9-0.20200809192844-da029d6f3722/core/router/router.go:237 +0x48
net/http.serverHandler.ServeHTTP(0xc00019c000, 0x10a7080, 0xc00019c9a0, 0xc00056ee00)
    /usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc00017e780, 0x10aa680, 0xc000175f80)
    /usr/local/go/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2969 +0x36c

While the UI is logging normally:

IP1 - - [12/Oct/2020:07:59:05 +0000] "GET / HTTP/1.1" 304 0 "https://xyz-adminio-ui.dev.fqdn/server" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP2 - - [12/Oct/2020:07:59:06 +0000] "GET /env.js HTTP/1.1" 200 326 "https://xyz-adminio-ui.dev.fqdn/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP1 - - [12/Oct/2020:07:59:06 +0000] "GET /styles-es2015.js HTTP/1.1" 304 0 "https://xyz-adminio-ui.dev.fqdn/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP2 - - [12/Oct/2020:07:59:06 +0000] "GET /scripts.js HTTP/1.1" 304 0 "https://xyz-adminio-ui.dev.fqdn/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP3 - - [12/Oct/2020:07:59:06 +0000] "GET /main-es2015.js HTTP/1.1" 304 0 "https://xyz-adminio-ui.dev.fqdn/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP4 - - [12/Oct/2020:07:59:06 +0000] "GET /runtime-es2015.js HTTP/1.1" 304 0 "https://xyz-adminio-ui.dev.fqdn/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP3 - - [12/Oct/2020:07:59:06 +0000] "GET /polyfills-es2015.js HTTP/1.1" 304 0 "https://xyz-adminio-ui.dev.fqdn/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP4 - - [12/Oct/2020:07:59:06 +0000] "GET /vendor-es2015.js HTTP/1.1" 304 0 "https://xyz-adminio-ui.dev.fqdn/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP1 - - [12/Oct/2020:07:59:08 +0000] "GET /main-es2015.js.map HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"
IP3 - - [12/Oct/2020:07:59:09 +0000] "GET /vendor-es2015.js.map HTTP/1.1" 200 5401714 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "XXX YYY"

In the Firefox network analysis, it looks as if a JS file or something is doing the API calls directly in the browser, which is why the UI service does not really log anything.

rzrbld commented 4 years ago

Looks like the problem with getting ServerInfo. Is your minio installation have some buckets and objects? You able to run 'mc admin --json info %my-alias%' ?

torblerone commented 4 years ago

Yes my MinIO cluster is up and running. There's two buckets with several thousand objects, users and policies.

rzrbld commented 4 years ago

@torblerone does this CLI command mc admin --json info %my-alias% works normally? Do you use "admin" credentials for adminio-api or some user with custom policy?

torblerone commented 4 years ago

Below you can find the output of mc admin --json info %alias%. Everything seems normal. For adminio-api I tried using the admin credentials and also created a new user with full rights but they both didn't work. Is is possible that there's a problem with my cluster working in distributed mode with erasure code?

{
    "status": "success",
    "info": {
        "mode": "online",
        "deploymentID": "anonymized",
        "buckets": {
            "count": 3
        },
        "objects": {
            "count": 610901
        },
        "usage": {
            "size": 73227714500
        },
        "services": {
            "vault": {
                "status": "disabled"
            },
            "ldap": {}
        },
        "backend": {
            "backendType": "Erasure",
            "onlineDisks": 4,
            "rrSCData": 2,
            "rrSCParity": 2,
            "standardSCData": 2,
            "standardSCParity": 2
        },
        "servers": [
            {
                "state": "ok",
                "endpoint": "minio-0.fqdn:9000",
                "uptime": 350093,
                "version": "2020-10-03T02:19:42Z",
                "commitID": "806625cbffcf7850720a45ff13aa25390ac757af",
                "network": {
                    "minio-0.fqdn:9000": "online",
                    "minio-1.fqdn:9000": "online",
                    "minio-2.fqdn:9000": "online",
                    "minio-3.fqdn:9000": "online"
                },
                "drives": [
                    {
                        "endpoint": "http://minio-0.fqdn:9000/storage/disc0",
                        "path": "/storage/disc0",
                        "state": "ok",
                        "uuid": "anonymized",
                        "totalspace": 751248084992,
                        "usedspace": 41817804800,
                        "availspace": 709430280192
                    }
                ]
            },
            {
                "state": "ok",
                "endpoint": "minio-2.fqdn:9000",
                "uptime": 350370,
                "version": "2020-10-03T02:19:42Z",
                "commitID": "806625cbffcf7850720a45ff13aa25390ac757af",
                "network": {
                    "minio-0.fqdn:9000": "online",
                    "minio-1.fqdn:9000": "online",
                    "minio-2.fqdn:9000": "online",
                    "minio-3.fqdn:9000": "online"
                },
                # minio1
                # minio2
                # minio3
            }
        ]
    }
}

Mickael-Roger commented 3 years ago

I had the same issue.

The problem was that I was using MINIO_HOST_PORT=http://127.0.0.1:9000 instead of MINIO_HOST_PORT=127.0.0.1:9000 (without the http://)

When using the correct env var, it worked

rzrbld / adminio-api

http panic serving #15