Minio doesn't work with NATS streaming server in cluster mode

[tpoxa]

As soon minio started sharing config along all nodes in cluster mode - no possibility to setup unique client ID for NATS streaming server which is required.

When I use NATS streaming server with streaming server enabled I have this error on 3 nodes of 4

API: GetBucketNotification(bucket=photo-originals)
Time: 17:25:56 UTC 08/25/2018
RequestID: 154E309909774222
RemoteHost: 172.18.0.1
UserAgent: Minio (linux; amd64) minio-go/v6.0.6 mc/2018-08-18T02:13:04Z
Error: ARN 'arn:minio:sqs::1:nats' not found
       1: cmd/logger/logger.go:294:logger.LogIf()
       2: cmd/notification.go:541:cmd.readNotificationConfig()
       3: cmd/bucket-notification-handlers.go:73:cmd.objectAPIHandlers.GetBucketNotificationHandler()
       4: cmd/api-router.go:86:cmd.(objectAPIHandlers).GetBucketNotificationHandler-fm()
       5: net/http/server.go:1947:http.HandlerFunc.ServeHTTP()

As possible solution:

Make client ID placeholder like %CLIENT_ID% which will be automatically populated by minio node ID or etc

---

Version: 2018-08-21T00:37:20Z Release-Tag: RELEASE.2018-08-21T00-37-20Z Commit-ID: 2d84b02bc429ad059a060001bccd712491e6e6a3

Linux sd-132731 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

[harshavardhana]

Can you provide an output of mc config get myminio | jq . to get the config JSON output?

[tpoxa]

{
  "version": "28",
  "credential": {
    "accessKey": "***",
    "secretKey": "***"
  },
  "region": "",
  "browser": "on",
  "worm": "off",
  "domain": "",
  "storageclass": {
    "standard": "",
    "rrs": ""
  },
  "cache": {
    "drives": [],
    "expiry": 90,
    "maxuse": 80,
    "exclude": []
  },
  "kms": {
    "vault": {
      "endpoint": "",
      "auth": {
        "type": "",
        "approle": {
          "id": "",
          "secret": ""
        }
      },
      "key-id": {
        "name": "",
        "version": 0
      }
    }
  },
  "notify": {
    "amqp": {
      "1": {
        "enable": false,
        "url": "",
        "exchange": "",
        "routingKey": "",
        "exchangeType": "",
        "deliveryMode": 0,
        "mandatory": false,
        "immediate": false,
        "durable": false,
        "internal": false,
        "noWait": false,
        "autoDeleted": false
      }
    },
    "elasticsearch": {
      "1": {
        "enable": false,
        "format": "",
        "url": "",
        "index": ""
      }
    },
    "kafka": {
      "1": {
        "enable": false,
        "brokers": null,
        "topic": ""
      }
    },
    "mqtt": {
      "1": {
        "enable": false,
        "broker": "",
        "topic": "",
        "qos": 0,
        "clientId": "",
        "username": "",
        "password": "",
        "reconnectInterval": 0,
        "keepAliveInterval": 0
      }
    },
    "mysql": {
      "1": {
        "enable": false,
        "format": "",
        "dsnString": "",
        "table": "",
        "host": "",
        "port": "",
        "user": "",
        "password": "",
        "database": ""
      }
    },
    "nats": {
      "1": {
        "enable": true,
        "address": "0.0.0.0:4222",
        "subject": "bucketevents",
        "username": "",
        "password": "",
        "token": "",
        "secure": false,
        "pingInterval": 0,
        "streaming": {
          "enable": true,
          "clusterID": "ml-cluster",
          "clientID": "minio1",
          "async": true,
          "maxPubAcksInflight": 9999999
        }
      }
    },
    "postgresql": {
      "1": {
        "enable": false,
        "format": "",
        "connectionString": "",
        "table": "",
        "host": "",
        "port": "",
        "user": "",
        "password": "",
        "database": ""
      }
    },
    "redis": {
      "1": {
        "enable": false,
        "format": "",
        "address": "",
        "password": "",
        "key": ""
      }
    },
    "webhook": {
      "1": {
        "enable": false,
        "endpoint": ""
      }
    }
  },
  "logger": {
    "console": {
      "enabled": true
    },
    "http": {
      "1": {
        "enabled": false,
        "endpoint": ""
      }
    }
  }
}

[Praveenrajmani]

@tpoxa It would be helpful if you provide us the mc admin info for all the nodes ?

[Praveenrajmani]

Meanwhile il try to reproduce it locally

[tpoxa]

Well, I was unable to keep environment as it is. I removed minio from all servers. To reproduce the problem you need to

• Setup minio 4 server cluster.
• Run the NATS streaming server
• Enable Nats Streaming notifications with any client ID on any single server. ( as I understand config will be populated to 3 other servers)
• Enable bucket notifications using nats like it described in minio manual
• Restart nodes one by one and check logs.

I hope it will help

On Thu, Aug 30, 2018 at 08:18 Praveen raj Mani notifications@github.com wrote:

Meanwhile il try to reproduce it locally

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/minio/minio/issues/6364#issuecomment-417192814, or mute the thread https://github.com/notifications/unsubscribe-auth/AAaVuqJpqOnErl6o_PRph_EktILof6IVks5uV3WYgaJpZM4WM4qV .

[Praveenrajmani]

Hey @tpoxa Can u check with the patch provided , By setting the client ID as empty ? BTW, this PR needs some decision . So this is just a temp fix.

This seems to work for me here when tested.

Let me know if you still find any difficulties. Thanks!

[tpoxa]

@Praveenrajmani Hello. Yes this fix works good for me. Thank you.

[tpoxa]

Hello! Is any chance to see this bugfix in master?

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.