h0x91b / redis-fast-driver

78 stars 13 forks source link

question: how to properly handle `Redis rotocol error, unknown type 6` #42

Closed Dmitry-N-Medvedev closed 2 years ago

Dmitry-N-Medvedev commented 3 years ago

The context: a function writes to a record to a redis stream with manually specified ID (1620826620000-38):

XADD stream:symbol-price-volume 1620826620000-38 symbol yfi price 80020.99221433526 volume 3347506668.3488083 percentage 17.1950784 cap 2931568512.2607646

If the function tries to write a new record but with exactly the same id (1620826620000-38) the redis will throw the Redis rotocol error, unknown type 6.

The question: how to properly handle this protocol error?

h0x91b-wix commented 3 years ago

can you show me the code to reproduce it?

Dmitry-N-Medvedev commented 3 years ago

here you are. please consider only the streamCommand.

  async write(message = null, baseId = null, recordId = null) {
    const id = (baseId === null) ? '*' : `${baseId}-${recordId}`;
    const streamKVs = (Object.entries(message)).flat();
    const streamCommand = [
      'XADD',
      this.#config.streamName,
      id,
      ...streamKVs,
    ];
    const hashValue = {};
    for (let i = 0; i < streamKVs.length - 1; i += 2) {
      hashValue[streamKVs[i]] = streamKVs[i + 1];
    }
    const { symbol } = hashValue;

    delete hashValue.symbol;

    const hashCommand = [
      'HSET',
      this.#config.cache,
      symbol,
      JSON.stringify(hashValue),
    ];
    this.#debuglog({
      hashCommand,
    });

    await this.#redis.rawCallAsync(streamCommand);
    await this.#redis.rawCallAsync(hashCommand);
  }
h0x91b-wix commented 3 years ago

try to:

 const streamKVs =  Object.entries(message).flat().map(o => o.toString());
Dmitry-N-Medvedev commented 3 years ago

@h0x91b-wix, I did an experiment: I tried to write data with the same ID twice and I got the error. Your library correctly converts from integers to strings, no problem here. The problem is the attempt to write two records to a stream with manually defined IDs that are the same.

When I try to write two records with the same ID using the redis-cli I get the following error: ERR The ID specified in XADD is equal or smaller than the target stream top item.

When I try to write two records with the same ID using the redis-fast-driver I get the Redis rotocol error, unknown type 6.

h0x91b commented 2 years ago

Can't reproduce it

r.rawCall(['XADD', 'mystream1', '1', 'name', 'Sara', 'surname', 'OConnor'], function(err, resp){
    console.log('xadd', {err, resp})
})

r.rawCall(['XADD', 'mystream1', '1', 'name', 'Sara', 'surname', 'OConnor'], function(err, resp){
    console.log('xadd', {err, resp})
})

r.rawCall(['XRANGE', 'mystream1', '-', '+'], function(err, resp){
    console.log('XRANGE', {err}, resp)
})

Console output:

xadd { err: null, resp: '1-0' }
xadd {
  err: 'ERR The ID specified in XADD is equal or smaller than the target stream top item',
  resp: undefined
}
XRANGE { err: null } [ [ '1-0', [ 'name', 'Sara', 'surname', 'OConnor' ] ] ]
Dmitry-N-Medvedev commented 2 years ago

@h0x91b , let's close this. I cannot reproduce it any longer either :)