uNetworking / uWebSockets.js

μWebSockets for Node.js back-ends :metal:
Apache License 2.0
8k stars 571 forks source link

The number of messages from the dropped is not matched after publish. #1079

Open snz2 opened 3 months ago

snz2 commented 3 months ago

The number of messages from the dropped is not matched after publish. The WS.SEND is a dropped message of the same number of dropped messages as the counter 2 of the return results, and Publish seems to be missing.

Testing is a method of send 100000 sync(not async) messages from another processor and testing a message that the server cannot handle.

uWebsockets.js version=20.43.0

  1. publish proc1 send message * 100,000 -> proc2(uWebsockets.js) publish -> proc1 check receive Count ->return true=100,000, dropped callback count 13,000, echo receive 28,000, proc2 drain 1(buffer length=0)

  2. ws.send proc1 send message * 100,000 -> proc2(uWebsockets.js) ws.send -> proc1 check receive Count ->return result 2=72,000, dropped callback count 72,000(same), echo receive 28,000, proc2 drain 1(buffer length=0)

*There are detailed numbers, but omitted

In addition to the same message, the number of eco -response has been checked, so it can be seen that Publish was not successful. For reference, the Drain callback came once(1) out of 100,000 times in both publish and ws.send.

Publish is automatically managed by the backpressure value, but it seems that the MISSED must have a message. It is a test that sends the same message as WS.SEND. If Publish is a design that doesn't care much about missing messages, it is inevitable. Since Publish is aimed at multiple topic users, it is a problem because it is not structurally like GetBufferedamount, so I want to create something like Topicws = {} that manages the topic separately. Please let me know if I missed. thank you ps. I turned a language translator. Please understand

uNetworkingAB commented 3 months ago

Can you make this into a integration test? Can run it in CI.

uNetworkingAB commented 3 months ago

In other words; can you provide the code that you based your conclusion on?

snz2 commented 3 months ago

uWebsocket Server (proc2)

` var droppedCnt=0, droppedCnt2=0, doneCnt=0; var usp=uws.App().ws('/*', { idleTimeout:0, open:(ws)=>{ ws.subscribe('node'); }, message:async (ws, msg, isBinary)=>{ let data; try{ data=JSON.parse(Buffer.from(msg).toString()); let res=await db.get(data.query); // res is object {...} // ws.send res['uuid']=data.uuid; let sendRes=ws.send(JSON.stringify(res), true, true); if(sendRes==2)droppedCnt2++;

     // publish 
     // usp.publish('node', JSON.stringify(res), true, true);

     doneCnt++;
}catch(e){
    console.error('message', data, e);
}},
drain:(ws)=>{
    console.log('drain:', ws.getBufferedAmount().length)
},
dropped:(ws, message, isBinary)=>{
    droppedCnt++;
},
close:async (ws, code)=>{try{

}catch(e){

}}

})`

In Proc1, WS turns for loops and sends a 100,000 DB request message. recognize the request response in the WS message by putting UUID in the message.

Because I test like this In WS.SEND(Proc2 uWebsockets server side), DROPP Callback(droppedCnt) and ws.Send's Result: The result 2(droppedCnt2) is exactly matched, so handling is possible. Publish comes only 13,000 messages to DroppedCallBack(droppedCnt), and only 28,000 messages come to the WS side(proc 1).

doneCnt is well(doneCnt) -driven by 100,000 when doing send or publish. In other words, the DB query was no problem. no Crash, no error

Of course, I will check the buffer if I use WS.SEND in the production code, but the problem is that the publish is not controlled by the dropped.

snz2 commented 3 months ago

I don't think the code function is not good in the GitHub editor ... I can't modify it. Anyway, the server side code is conceptually everything.