NATS websocket Connection error

frogonhills commented 1 year ago

I tried to use NATS install by helm install with a values.yaml file to connect using WebSocket

nats:
image: nats:alpine

jetstream:
  enabled: true

  memStorage:
    enabled: true
    size: "2Gi"

  fileStorage:
    enabled: true
    size: "10Gi"
    storageDirectory: /data/
    # storageClassName: standard
    storageClassName: gp2

websocket:
  enabled: true
  port: 8443

cluster:
  enabled: true
  noAdvertise: true

replicas: 2

affinity:
  nodeAffinity:
     preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: node.kubernetes.io/zone
            operator: In
            values:
            - ap-south-1            
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app
              operator: In
              values:
                - nats
                - stan
        topologyKey: "kubernetes.io/hostname"

but I can't connect to my server link from react app where the demo " servers: ["wss://demo.nats.io:8443"] " works fine in this react example https://nats.io/blog/getting-started-nats-ws/

import {
  connect,
  NatsConnection,
} from "nats.ws";

import {
  useEffect,
  useState,
} from "react";

export default function Home() {
  const [nats, setNats] = useState<NatsConnection>();

useEffect(() => {
  (async () => {
    const nc = await connect({
      servers: ["wss://demo.nats.io:8443"],
    })
    setNats(nc)
    console.log("connected to NATS")
  })();

  return () => {
    nats?.drain();
    console.log("closed NATS connection")
  }
}, [])

return (
  <>
    {nats ? (
      <h1>Connected to {nats?.getServer()}</h1>
    ) : (
      <h1>Connecting to NATS...</h1>
    )}
  </>
)}

so what is the proper configuration in helm install to connect from react app using WebSocket?

aricart commented 1 year ago

one thing to check is your certificates - websocket server config has a tls directive - the way you are configuring the server, doesn't have the TLS section declared - so possibly this is getting rejected at that point. One thing to try is to supply the no_tls: true in your config, and attempt a ws:// connection.

@wallyqs may have more insight on the helm aspects.

jmndao commented 1 year ago

Hello @aricart, I have been having the same issue however, I'm running my NATS server in a cluster served behind an ec2. Whenever I try to connect I get the connection error. I have followed the guideline from the doc:

// nats.conf file

websocket {
  port: 8843  # Specify the WebSocket port (adjust to the desired port)
  no_tls: true  # Disable TLS (set to true for non-secure WebSocket)
  # Other WebSocket configuration options can be added here
}

I have used this configuration in a docker where I set it in the nats config:

FROM nats-streaming:latest

# Copy your custom nats.conf into the image
COPY nats.conf /etc/nats/nats.conf

# Expose client and management ports
EXPOSE 4222 8222 8843

# Run the NATS server with the configuration file
CMD ["--config", "/etc/nats/nats.conf"]

That image is the one I use in the pod to run NATS with the new config to make the websocket available

  spec:
      containers:
        - name: nats
          image: <...>/nats-websocket:<tag>
          args:
            - "-p"
            - "4222"
            - "-m"
            - "8222"
            - "-hbi"
            - "5s"
            - "-hbt"
            - "5s"
            - "-hbf"
            - "2"
            - "-SD"
            - "-cid"
            - "caytu"
            - "-c"
            - "/etc/nats/nats.conf"
          resources:
          ...

So, till now, I couldn't connect from my nextjs frontend app and seeing the same error (NATS websocket Connection error). 
Could you please help with this ?

aricart commented 1 year ago

@jmndao your server logs say the server is serving we socket?

What does the client connection options look like?

jmndao commented 1 year ago

Thank you for your quick reaction @aricart.

Yes, it actually is:

kubectl logs -f nats-depl-5fdf58c84b-kvrgp
[1] 2023/09/03 11:11:45.499335 [INF] STREAM: Starting nats-streaming-server[caytu] version 0.25.5
[1] 2023/09/03 11:11:45.499375 [INF] STREAM: ServerID: VlL66tnDTFrJqmZYwZWtu0
[1] 2023/09/03 11:11:45.499377 [INF] STREAM: Go version: go1.19.10
[1] 2023/09/03 11:11:45.499380 [INF] STREAM: Git commit: [abfd5a3]
[1] 2023/09/03 11:11:45.528908 [INF] Starting nats-server
[1] 2023/09/03 11:11:45.528927 [INF]   Version:  2.9.19
[1] 2023/09/03 11:11:45.528930 [INF]   Git:      [f4f3cce]
[1] 2023/09/03 11:11:45.528933 [INF]   Name:     NCZCCADHCKDSKXUVDW2DW5JKWOAEG4WGA7UVL2UUYJ3FFIB5AFZ7KPEQ
[1] 2023/09/03 11:11:45.528935 [INF]   ID:       NCZCCADHCKDSKXUVDW2DW5JKWOAEG4WGA7UVL2UUYJ3FFIB5AFZ7KPEQ
[1] 2023/09/03 11:11:45.531114 [INF] Using configuration file: /etc/nats/nats.conf
[1] 2023/09/03 11:11:45.541559 [INF] Starting http monitor on 0.0.0.0:8222
[1] 2023/09/03 11:11:45.544007 [INF] Listening for websocket clients on ws://0.0.0.0:8843
[1] 2023/09/03 11:11:45.544017 [WRN] Websocket not configured with TLS. DO NOT USE IN PRODUCTION!
[1] 2023/09/03 11:11:45.544217 [INF] Listening for client connections on 0.0.0.0:4222
[1] 2023/09/03 11:11:45.544318 [INF] Server is ready
[1] 2023/09/03 11:11:45.554719 [INF] STREAM: Recovering the state...
[1] 2023/09/03 11:11:45.554731 [INF] STREAM: No recovered state
[1] 2023/09/03 11:11:45.554954 [DBG] STREAM: Did not detect another server instance

aricart commented 1 year ago

And the client connect options?

jmndao commented 1 year ago

const connectionParams: ConnectionOptions = {
  servers: [`ws://${process.env.NEXT_PUBLIC_API_URL}:8843`],
  name: "web-depl",
};

process.env.NEXT_PUBLIC_API_URL: somedomain.com

jmndao commented 1 year ago

The rest goes like the doc in nats.ws

...
const EventListenerProvider = ({ children }: EventListenerProviderProps) => {
  const [nats, setNats] = useState<NatsConnection>();

  /**
   * Connect to NATS
   */
  useEffect(() => {
    (async () => {
      const connectToNats = async () => {
        // If already connected, return
        if (nats) return;

        await connect(connectionParams)
          .then((conn) => {
            setNats(conn);
            console.log("Connected to NATS");
          })
          .catch((err) => console.error("Error connecting to NATS", err));
      };

      connectToNats();
    })();

    return () => {
      nats?.drain();
      console.log("NATS connection closed");
    };
  }, []);
  ...

jmndao commented 1 year ago

Hi @aricart, thank you very much for the help. Fortunately, I did find a fix to the related issue which was not truly a NATS server problem.

Here is what was happening,

As I said at the very beginning I was running my cluster behind an ec2 with ingress (NGINX) then it was much more likely that the issue was about a connection problem. After a little dig in, I had to:

Enable the port behind the security group of my ec2

Enable WebSocket handling in the ingress configuration

nginx.org/websocket-services: "nats-srv"
...
 - path: /nats-ws
        pathType: Prefix
        backend:
          service:
            name: nats-srv
            port:
              number: 8843

So, in account of all of that, the issue was more related to a networking problem and finally got resolved.

Thanks once again

nats-io / nats.ws

NATS websocket Connection error #192