deepstreamIO / deepstream.io

deepstream.io server
https://deepstreamio.github.io
MIT License
7.13k stars 382 forks source link

[Bug] SocketWrapper mismatch in "handlers/record/record-transitions" when storage enabled #1116

Closed wyl1232792 closed 2 years ago

wyl1232792 commented 2 years ago

I found this problem when requesting bulk record update via http api, I got a http timeout error on insert, and could always reproduce that. The response says that some records failed to handle, though actually records are successfully inserted since I can see in storage.

version: v6.1.0 client: python.deepstreamio_client (using batch mode) storage: rethinkdb cache: redis

https://github.com/deepstreamIO/deepstream.io/blob/117034a674f3fe76942f9c2d72960d9fddd6b117/src/handlers/record/record-transition.ts#L283-L294 Storage could be much slower then cache update, but it is designed to move on to next step at onCacheSetResponse, so basicly write ack is always choosing the wrong wrapper.

jaime-ez commented 2 years ago

Hi, thanks for trying the new version! I assume this error was not present on the previous one.

When you use the http api writeAck is set to true by default, and since you re writing to storage it will wait until storage responds. The previous server version was not handling all writeAck messages so these issues might arise.

Do you have http requestTimeout set in options ? If so whats the value you are using? Can you please share the error message you are receiving on the http response? How many records are you trying to batch update? Do you have an estimate of whats your storage response time?

jaime-ez commented 2 years ago

Hi, after some tests I couldn't reproduce the issue. Please reopen if it persists.

There is a non documented http endpoint parameter requestTimeout that defaults to 20000. You can tweak it to avoid this timeout responses:

connectionEndpoints:
  - type: http
    options:
      # number of milliseconds before returning an http timeout error. Defaults to 20000
     requestTimeout: 100000

Cheers