[NEXT-1119] Socket.IO Not working anymore from Next.js 13.2.5-canary.27 to latest Next.js 13.4.1

saicharan-gevme commented 1 year ago

Verify canary release

[X] I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.4.0: Mon Mar  6 21:00:41 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T8103
    Binaries:
      Node: 18.16.0
      npm: 9.5.1
      Yarn: N/A
      pnpm: N/A
    Relevant packages:
      next: 13.4.1-canary.2
      eslint-config-next: N/A
      react: 18.2.0
      react-dom: 18.2.0

warn  - Latest canary version not detected, detected: "13.4.1-canary.2", newest: "13.4.1".
        Please try the latest canary version (`npm install next@canary`) to confirm the issue still exists before creating a new issue.
        Read more - https://nextjs.org/docs/messages/opening-an-issue

Which area(s) of Next.js are affected? (leave empty if unsure)

App directory (appDir: true)

Link to the code that reproduces this issue

https://codesandbox.io/p/sandbox/nextjs-socketio-u89y82?file=%2Fpackage.json&selection=%5B%7B%22endColumn%22%3A1%2C%22endLineNumber%22%3A4%2C%22startColumn%22%3A1%2C%22startLineNumber%22%3A4%7D%5D

To Reproduce

Agenda of this project?

To setup the Socket.Io server and share the information from one client to another user socket.io events

The App is have the following things -

An api for initialising the Socket.Io server - inside /pages/api/socketio
A client page to connect to this server - inside /app/page

Whats the Issue?

There has been an unknown error in network connection from client side when its trying to create the Socket.Io connection.

Since when this issue is happening?

This has been happening from Next.js version 13.2.5-canary.27 version and its still happening even in latest canary release 13.4.1-canary.2 and also the latest release Next.js 13.4.1

Do we have an Example project?

Yes, I have created a testing project in codesandbox, that I have already shared. (Link)

Steps to Reproduce

Just see the console and network tabs for the client connection error. It will happen to all the versions on and above 13.2.5-canary.27
And to see if its fixed, you need to downgrade to 13.2.5-canary.26

Some References PR for / might this issue have created?

I some how gathered the PR for the version when this Socket.Io broken (i.e. 13.2.5-canary.27) - See here

Describe the Bug

There has been an unknown error in network connection from client side when its trying to create the Socket.Io connection.

Expected Behavior

There should be successful Socket.Io connection.
It should exactly work the same way as it was working 13.2.5-canary.26 or lower.
I was using 13.2.4 as stable version for our production application, as we had to use Socket.Io for websocket connection.

Which browser are you using? (if relevant)

I am using latest Chrome version, but this has to do more with Next.js Core server. It has nothing to do with any browsers.

How are you deploying your application? (if relevant)

Using Docker, but this is irrelevant too, as in localhost also, there is this issue.

_NEXT-1119

flipUp02 commented 1 year ago

Same problem @leerob

AdamGonda commented 1 year ago

Same here

leerob commented 1 year ago

Thank you for reporting! The best way to track this is by upvoting the issue versus commenting, unless you can provide additional information or reproductions. Greatly appreciate the assistance with making a minimal reproduction 🙏

leerob commented 1 year ago

Pointed these two discussions here:

And this issue:

https://github.com/vercel/next.js/issues/49249

Minisaints commented 1 year ago

It appears that after "13.2.5-canary.26", a trailing slash is appended to the end of the route.

Disabling "addTrailingSlash" in the Socket.IO Server constructor resolved this for me on the latest version (13.4.1).

"pages/api/socket.js"

const io = new Server(res.socket.server, {
   path: "/api/socket_io",
   addTrailingSlash: false
 });

saicharan-gevme commented 1 year ago

It appears that after "13.2.5-canary.26", a trailing slash is appended to the end of the route.

Disabling "addTrailingSlash" in the Socket.IO Server constructor resolved this for me on the latest version (13.4.1).
"pages/api/socket.js"

const io = new Server(res.socket.server, {
   path: "/api/socket_io",
   addTrailingSlash: false
 });

Thank you for this update, this has resolved the network issue we got earlier, but still the WS connection is not working like earlier, it always stays in pending mode

You can please check and confirm that its working when you switch back to old version of Next.js

Minisaints commented 1 year ago

It appears that after "13.2.5-canary.26", a trailing slash is appended to the end of the route. Disabling "addTrailingSlash" in the Socket.IO Server constructor resolved this for me on the latest version (13.4.1).
"pages/api/socket.js"

const io = new Server(res.socket.server, {
   path: "/api/socket_io",
   addTrailingSlash: false
 });
Thank you for this update, this has resolved the network issue we got earlier, but still the WS connection is not working like earlier, it always stays in pending mode

You can please check and confirm that its working when you switch back to old version of Next.js

No problem. Ah really, my WS connection finishes fine and shows as connected; it doesn't get stuck on pending.

All WS functionality with our chat application works as intended too (on 13.4.1).

flipUp02 commented 1 year ago

It appears that after "13.2.5-canary.26", a trailing slash is appended to the end of the route. Disabling "addTrailingSlash" in the Socket.IO Server constructor resolved this for me on the latest version (13.4.1).
"pages/api/socket.js"

const io = new Server(res.socket.server, {
   path: "/api/socket_io",
   addTrailingSlash: false
 });
Thank you for this update, this has resolved the network issue we got earlier, but still the WS connection is not working like earlier, it always stays in pending mode You can please check and confirm that its working when you switch back to old version of Next.js
No problem. Ah really, my WS connection finishes fine and shows as connected; it doesn't get stuck on pending.

All WS functionality with our chat application works as intended too (on 13.4.1).

can give us full code for test this ?

flipUp02 commented 1 year ago

It appears that after "13.2.5-canary.26", a trailing slash is appended to the end of the route. Disabling "addTrailingSlash" in the Socket.IO Server constructor resolved this for me on the latest version (13.4.1).
"pages/api/socket.js"

const io = new Server(res.socket.server, {
   path: "/api/socket_io",
   addTrailingSlash: false
 });
Thank you for this update, this has resolved the network issue we got earlier, but still the WS connection is not working like earlier, it always stays in pending mode

You can please check and confirm that its working when you switch back to old version of Next.js

your code is working now please follow this : in socket.js add: const io = new Server(res.socket.server, { path: '/api/socket_io', addTrailingSlash: false, });

in page.jsx : socket = io(undefined, { path: '/api/socket_io', });

and update all packages

flipUp02 commented 1 year ago

i think the problem in your old path path: "/api/my_awesome_socket",

newtmex commented 1 year ago

It appears that after "13.2.5-canary.26", a trailing slash is appended to the end of the route.

Disabling "addTrailingSlash" in the Socket.IO Server constructor resolved this for me on the latest version (13.4.1).
"pages/api/socket.js"

const io = new Server(res.socket.server, {
   path: "/api/socket_io",
   addTrailingSlash: false
 });

This worked for me

berrugo commented 1 year ago

It appears that after "13.2.5-canary.26", a trailing slash is appended to the end of the route.

Disabling "addTrailingSlash" in the Socket.IO Server constructor resolved this for me on the latest version (13.4.1).
"pages/api/socket.js"

const io = new Server(res.socket.server, {
   path: "/api/socket_io",
   addTrailingSlash: false
 });

Also worked for me on latest 13.4.1, thanks!

flipUp02 commented 1 year ago

Anyone can update code for app/api not pages/api ?

flipUp02 commented 1 year ago

It appears that after "13.2.5-canary.26", a trailing slash is appended to the end of the route.

Disabling "addTrailingSlash" in the Socket.IO Server constructor resolved this for me on the latest version (13.4.1).
"pages/api/socket.js"

const io = new Server(res.socket.server, {
   path: "/api/socket_io",
   addTrailingSlash: false
 });

Can make the code for app/api ?

ddadaal commented 1 year ago

The problem still remains and is about Next.js server ignores listeners to upgrade event of Next.js server from Next.js 13.3.

https://codesandbox.io/p/sandbox/busy-euclid-qbvdv8

This project creates an API route (pages/api/proxy.ts) that adds a listener to next.js server listening to upgrade event when accessed and prints a setup upgrade log. The listener just logs a upgrade log to indicate the upgrade event is handled. There is also a page (apps/page.tsx) that first calls the API route (to register the listener), and then tries to establish a WS connection to the server.

In Next.js 13.2.5, it works fine. Both setup upgrade and upgrade logs are printed.

In Next.js 13.4.2, only setup upgrade log is printed. It means the server.on("upgrade", {handler} is called, but the listener is ignored.

saicharan-gevme commented 1 year ago

The problem still remains and is about Next.js server ignores listeners to upgrade event of Next.js server from Next.js 13.3.

https://codesandbox.io/p/sandbox/busy-euclid-qbvdv8

This project creates an API route (pages/api/proxy.ts) that adds a listener to next.js server listening to upgrade event when accessed and prints a setup upgrade log. The listener just logs a upgrade log to indicate the upgrade event is handled. There is also a page (apps/page.tsx) that first calls the API route (to register the listener), and then tries to establish a WS connection to the server.

In Next.js 13.2.5, it works fine. Both setup upgrade and upgrade logs are printed.

In Next.js 13.4.2, only setup upgrade log is printed. It means the server.on("upgrade", {handler} is called, but the listener is ignored.

Thank you for explaining, I was also having same problem, when I explained here https://github.com/vercel/next.js/issues/49334#issuecomment-1539618193

But I was unable to explain with proper context, thank you for this example.

CKGrafico commented 1 year ago

The problem still remains and is about Next.js server ignores listeners to upgrade event of Next.js server from Next.js 13.3.

https://codesandbox.io/p/sandbox/busy-euclid-qbvdv8

This project creates an API route (pages/api/proxy.ts) that adds a listener to next.js server listening to upgrade event when accessed and prints a setup upgrade log. The listener just logs a upgrade log to indicate the upgrade event is handled. There is also a page (apps/page.tsx) that first calls the API route (to register the listener), and then tries to establish a WS connection to the server.

In Next.js 13.2.5, it works fine. Both setup upgrade and upgrade logs are printed.

In Next.js 13.4.2, only setup upgrade log is printed. It means the server.on("upgrade", {handler} is called, but the listener is ignored.

and do we have a workaround?

HarcheSamir commented 1 year ago

any solutions ? : (using App Router)

'use client' import { useEffect, useState } from 'react'; import io from 'socket.io-client';/////this line is producing the error

const socket = io('http://localhost:3000');

export default function Page(){ const [newDecision, setNewDecision] = useState(''); useEffect(() => { socket.on('newDecision', ({ decision }) => { setNewDecision(decision);} }); }, []);

return (

); };

errors :

Module not found: Can't resolve 'bufferutil' in ... Module not found: Can't resolve 'utf-8-validate' in

rafael-14 commented 1 year ago

any solution?

IHarisImran commented 1 year ago

Any solution for pure app dir work? Currently my working approach is api is from pages dir and website is from app dir.

vimlesh1975 commented 1 year ago

Module not found: Can't resolve 'bufferutil' in ... Module not found: Can't resolve 'utf-8-validate' in

npm install bufferutil utf-8-validate

ignwombat commented 1 year ago

For those still having issues, I found that removing the "transports" prop in the client worked for me (unless I did something else)

IHarisImran commented 1 year ago

GET http://localhost:3000/api/socket_io?EIO=4&transport=polling&t=OXIVwM_ 404 (Not Found)

For those who are having this 404 error, I found that removing not-found.jsx file fixes this.

markosoomets1 commented 1 year ago

Downgraded my NextJS version to 13.2.4 to resolve the issue temporarily.

kul-sudo commented 1 year ago

GET http://localhost:3000/api/socket_io?EIO=4&transport=polling&t=OXIVwM_ 404 (Not Found)

For those who are having this 404 error, I found that removing not-found.jsx file fixes this.

What is not-found.jsx?

kul-sudo commented 1 year ago

i think the problem in your old path path: "/api/my_awesome_socket",

I get no error when I use the right socket name.

neojpk commented 1 year ago

bump

SuttonJack commented 1 year ago

Module not found: Can't resolve 'bufferutil' in ... Module not found: Can't resolve 'utf-8-validate' in

npm install bufferutil utf-8-validate

For anyone encountering this, I added the following to my next.config.js:

/** @type {import('next').NextConfig} */

const nextConfig = {
  webpack: (config) => {

    config.externals.push({
      'utf-8-validate': 'commonjs utf-8-validate',
      bufferutil: 'commonjs bufferutil',
    })

    return config
  },
}

module.exports = nextConfig

which resolved it for me

EmitoSanti commented 1 year ago

Module not found: Can't resolve 'bufferutil' in ... Module not found: Can't resolve 'utf-8-validate' in npm install bufferutil utf-8-validate

For anyone encountering this, I added the following to my next.config.js:
/** @type {import('next').NextConfig} */

const nextConfig = {
  webpack: (config) => {

    config.externals.push({
      'utf-8-validate': 'commonjs utf-8-validate',
      bufferutil: 'commonjs bufferutil',
    })

    return config
  },
}

module.exports = nextConfig
which resolved it for me

which resolved it for me too !!! Thanks!!!!!! XD

WhitigolProd commented 1 year ago

13.4.4 here. Any resolution to this? REALLY need socket.io... I refuse to fall back to standard polling 🤦

ChristopherTrimboli commented 1 year ago

This is brutal and blocking me from using Vercel / Next.js as a fullstack backend for apps... I think we deserve a full Websockets docs section inside API Route Handlers v13.

Currently using v13.2.4 and old /pages just to get sockets to work.

S.O.S.

NorthernScythe commented 1 year ago

Our teams are currently thinking of migrating our existing applications (express/psql/sio stack) to Next 13 and benefiting from the new stable App Router API, but this Socket.IO issue is an absolute no-go for the moment. We really need a solution for this issue, and we also need to be able to bind the sio handler to the main server.

ddadaal commented 1 year ago

I have solved the issue by using custom server. It is possible to bind the http.Server object to req.socket and access the object from API route, where upgrade event can be bound to handle WS connections.

// server/index.ts
// ...
const app = next({ dev, hostname, port });
const handle = app.getRequestHandler();

async function main() {

  await app.prepare();

  // create our own http.Server object.
  const server = createServer(async (req, res) => {

    // bind the http.Server object before using next.js to handle the request
    (req.socket as any).server = server;
    await handle(req, res);
  });

  // ...
}

It requires an extra build step that builds every source file after next build, and the files from both build steps(.next and the output dir of this extra build) must be distributed to run the server.

ChristopherTrimboli commented 1 year ago

@ddadaal nice work, issue for me is would break Vercel native deploys to do it this way, but good investigation!

vimlesh1975 commented 1 year ago

Custom server with socket.io with App Router works. But then HMR doesn't work.

StringKe commented 1 year ago

@ddadaal Does this upgrade affect SSE (Server-Sent-Event) requests?

https://github.com/vercel/next.js/issues/50804

ashdephoenix123 commented 1 year ago

I am getting the same error while using socket io in next js 13.4. GET http://localhost:3000/socket.io?EIO=4&transport=polling&t=OZdX4xW 404 (Not Found) in my console. I really want to work it out for this project. Please drop any answer to work it out. Or if possible Please suggest any other library to use.

lucazsunion commented 1 year ago

Websocket not working but polling is work.

pages/api/socket.js
import { Server } from "socket.io";

export default function handler(req, res) { if (res.socket.server.io) { console.log("Server already started!"); res.end(); return; }

const io = new Server(res.socket.server, {
    path: "/api/socket_io",
    addTrailingSlash: false,
});
res.socket.server.io = io;

const onConnection = (socket) => {
    console.log("New connection", socket.id);
    socket.on("createdMessage", (msg) => {
        console.log("New message", msg);
        socket.emit("newIncomingMessage", msg);
    });
};

io.on("connection", onConnection);

console.log("Socket server started successfully!");
res.end();

}

>pages/index.jsx

import { useEffect } from "react"; import { io } from "socket.io-client";

let socket;

export default function Home() { const socketInitializer = async () => { // We call this just to make sure we turn on the websocket server await fetch("/api/socket");

    socket = io(undefined, {
        path: "/api/socket_io",
    });

    socket.on("connect", () => {
        console.log("Connected", socket.id);
    });

    socket.on("newIncomingMessage", (msg) => {
        console.log("New message in client", msg);
    });
};

useEffect(() => {
    socketInitializer();
}, []);

return (
    <div>
        <button
            onClick={() => {
                socket.emit("createdMessage", "Hello world!");
            }}
        >
            Send message
        </button>
    </div>
);

}


> Devtools
![image](https://github.com/vercel/next.js/assets/106571103/959ae36a-ed21-4c1c-a5df-cbce81671047)
And
![image](https://github.com/vercel/next.js/assets/106571103/f436e422-1545-4085-9a7f-6a8d31485cc0)

If i click `Send message` button.
![image](https://github.com/vercel/next.js/assets/106571103/40ca9cd3-a77d-48bd-b535-ddfecebc9285)
![image](https://github.com/vercel/next.js/assets/106571103/b2b8aca7-8933-47ff-9313-5ab348d21d68)

ashdephoenix123 commented 1 year ago

Websocket not working but polling is work.

pages/api/socket.js

import { Server } from "socket.io";

export default function handler(req, res) {
  if (res.socket.server.io) {
      console.log("Server already started!");
      res.end();
      return;
  }

  const io = new Server(res.socket.server, {
      path: "/api/socket_io",
      addTrailingSlash: false,
  });
  res.socket.server.io = io;

  const onConnection = (socket) => {
      console.log("New connection", socket.id);
      socket.on("createdMessage", (msg) => {
          console.log("New message", msg);
          socket.emit("newIncomingMessage", msg);
      });
  };

  io.on("connection", onConnection);

  console.log("Socket server started successfully!");
  res.end();
}

pages/index.jsx

import { useEffect } from "react";
import { io } from "socket.io-client";

let socket;

export default function Home() {
  const socketInitializer = async () => {
      // We call this just to make sure we turn on the websocket server
      await fetch("/api/socket");

      socket = io(undefined, {
          path: "/api/socket_io",
      });

      socket.on("connect", () => {
          console.log("Connected", socket.id);
      });

      socket.on("newIncomingMessage", (msg) => {
          console.log("New message in client", msg);
      });
  };

  useEffect(() => {
      socketInitializer();
  }, []);

  return (
      <div>
          <button
              onClick={() => {
                  socket.emit("createdMessage", "Hello world!");
              }}
          >
              Send message
          </button>
      </div>
  );
}

Devtools And

If i click Send message button.

I almost did the same, I have just one question n the code you provided, you used socket_io in path, what should be the code inside socket_io.js?

lucazsunion commented 1 year ago

Websocket not working but polling is work.

pages/api/socket.js

import { Server } from "socket.io";

export default function handler(req, res) {
    if (res.socket.server.io) {
        console.log("Server already started!");
        res.end();
        return;
    }

    const io = new Server(res.socket.server, {
        path: "/api/socket_io",
        addTrailingSlash: false,
    });
    res.socket.server.io = io;

    const onConnection = (socket) => {
        console.log("New connection", socket.id);
        socket.on("createdMessage", (msg) => {
            console.log("New message", msg);
            socket.emit("newIncomingMessage", msg);
        });
    };

    io.on("connection", onConnection);

    console.log("Socket server started successfully!");
    res.end();
}

pages/index.jsx

import { useEffect } from "react";
import { io } from "socket.io-client";

let socket;

export default function Home() {
    const socketInitializer = async () => {
        // We call this just to make sure we turn on the websocket server
        await fetch("/api/socket");

        socket = io(undefined, {
            path: "/api/socket_io",
        });

        socket.on("connect", () => {
            console.log("Connected", socket.id);
        });

        socket.on("newIncomingMessage", (msg) => {
            console.log("New message in client", msg);
        });
    };

    useEffect(() => {
        socketInitializer();
    }, []);

    return (
        <div>
            <button
                onClick={() => {
                    socket.emit("createdMessage", "Hello world!");
                }}
            >
                Send message
            </button>
        </div>
    );
}

Devtools And

If i click Send message button.

I almost did the same, I have just one question n the code you provided, you used socket_io in path, what should be the code inside socket_io.js?

Follow https://socket.io/docs/v4/server-options/#path

ashdephoenix123 commented 1 year ago

@huyc27 Hey thanks, for some reason it didnt work previously but i recreated the application and it is not showing 404 error now. Although it keeps on the status on pending and it make calls repeatedly, Do you have any idea what is going on here?

tiendnm commented 1 year ago

are there any solutions for app directory? 😭

lucazsunion commented 1 year ago

are there any solutions for app directory? 😭

Về version 13.2.4 là dùng được nhé ông. Tôi dùng được rồi nhưng không global socket được.

ashdephoenix123 commented 1 year ago

are there any solutions for app directory? 😭

You can do one thing, You can make your app in app directory but just for socket to work make an api under pages directory.

danya0365 commented 1 year ago

I can make it work by following code on following version

"socket.io": "^4.7.1",
"socket.io-client": "^4.7.1",
"next": "13.4.7",

on "use client" side app/chat/page.tsx

import { io as ClientIO } from 'socket.io-client';
...

  useEffect((): any => {
    const socket = new (ClientIO as any)(process.env.NEXT_PUBLIC_SITE_URL, {
      path: '/api/socket',
      addTrailingSlash: false
    });

on api side pages/api/socket.ts

import { Server as ServerIO } from 'socket.io';
...
export default async (req: NextApiRequest, res: NextApiResponseServerIO) => {
  if (!res.socket.server.io) {
    console.log('New Socket.io server...');
    // adapt Next's net Server to http Server
    const httpServer: NetServer = res.socket.server as any;
    const io = new ServerIO(httpServer, {
      path: '/api/socket',
      addTrailingSlash: false
    });
    // append SocketIO server to Next.js socket server response
    res.socket.server.io = io;
  }
  res.end();
};

full source code here https://github.com/danya0365/nextjs13-realtime-chat-with-socketio

tiendnm commented 1 year ago

are there any solutions for app directory? 😭

Về version 13.2.4 là dùng được nhé ông. Tôi dùng được rồi nhưng không global socket được.

Cảm ơn nhé, tôi áp dụng cách của ông trên 13.4.7 và thành công. Không biết sau này họ có hỗ trợ cho app directory không chứ tạo thêm một thư mục page chỉ xài cho socket thì hơi thừa 😁

lucazsunion commented 1 year ago

I can make it work by following code on following version

"socket.io": "^4.7.1",
"socket.io-client": "^4.7.1",
"next": "13.4.7",

on "use client" side app/chat/page.tsx

import { io as ClientIO } from 'socket.io-client';
...

  useEffect((): any => {
    const socket = new (ClientIO as any)(process.env.NEXT_PUBLIC_SITE_URL, {
      path: '/api/socket',
      addTrailingSlash: false
    });

on api side pages/api/socket.ts

import { Server as ServerIO } from 'socket.io';
...
export default async (req: NextApiRequest, res: NextApiResponseServerIO) => {
  if (!res.socket.server.io) {
    console.log('New Socket.io server...');
    // adapt Next's net Server to http Server
    const httpServer: NetServer = res.socket.server as any;
    const io = new ServerIO(httpServer, {
      path: '/api/socket',
      addTrailingSlash: false
    });
    // append SocketIO server to Next.js socket server response
    res.socket.server.io = io;
  }
  res.end();
};

full source code here https://github.com/danya0365/nextjs13-realtime-chat-with-socketio

websocket work?

vimlesh1975 commented 1 year ago

@danya0365 It works for app directory. MHR and socket all works.

saicharan-gevme commented 1 year ago

@danya0365 The method you've explained works but it uses Http/s Polling and Not WebSocket. The whole point to use socket.io is for using WebSocket messaging.

andrevenancio commented 1 year ago

I dont want to add more fuel to the fire, but downgrading to 13.2.4 seems to work localhost. as soon as I deploy to vercel I get those 404's

vercel / next.js