search

nautechsystems / nautilus_trader

A high-performance algorithmic trading platform and event-driven backtester
https://nautilustrader.io
GNU Lesser General Public License v3.0
2.02k stars 460 forks source link

Get radis error in trimming stream #1647

Closed HeidXC closed 4 months ago

HeidXC commented 4 months ago

Bug Report

I recently tested with msgbus backed by radis. Scenario is: I have a controller started a child strategy, controller will publish msg at some point, the published data looks like: [("sym_a", "sym_b", a_float_value)]. The strategy subscribed with the msg bus.

Expected Behavior

Child strategy receive the msg successfully.

Actual Behavior

2024-05-17T10:24:21.488935281Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.DataEngine': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.489311016Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.RiskEngine': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.489467391Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.ExecEngine': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.489600888Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.OrderEmulator': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.489743945Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.TESTER-001': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.489870013Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.CrossAssetsFundingController': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.489996291Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.BINANCE': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.490136908Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.BINANCE': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.490291762Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.BINANCE': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.490499550Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.BINANCE': An error was signalled by the server - ResponseError: syntax error 2024-05-17T10:24:21.490642952Z [ERROR] TESTER-001.nautilus_common::redis: Error trimming stream 'trader-TESTER-001:streams:events.system.DataEngine': An error was signalled by the server - ResponseError: syntax erro

Specifications

cjdsellers commented 4 months ago

Hey @VeraLyu

Which version of Redis are you running? There's a chance the trim command being used is not available for the Redis version you're using.

HeidXC commented 4 months ago

Hi Chris, My redis is shipped with ubuntu 22.04 LTS, below is the info

Server

redis_version:6.0.16 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:a3fdef44459b3ad6 redis_mode:standalone os:Linux 5.15.0-97-generic x86_64

HeidXC commented 4 months ago

yes, it seems my redis does not support TRIM command, pls let me know what version of Redis I need to install. Thanks. 127.0.0.1:6379> XTRIM trader-TESTER-001:streams:data.instrument.BINANCE.UNFIUSDT-PERP MINID 1715941466400-0 (error) ERR syntax error

HeidXC commented 4 months ago

It seems my redis does not support XTRIM MINID param, it just support MAXLEN param.

127.0.0.1:6379> XTRIM trader-TESTER-001:streams:data.instrument.BINANCE.UNFIUSDT-PERP MINID 1715941382421-0 (error) ERR syntax error 127.0.0.1:6379> HELP XTRIM XTRIM key MAXLEN [~] count summary: Trims the stream to (approximately if '~' is passed) a certain size since: 5.0.0 group: stream

127.0.0.1:6379> XTRIM trader-TESTER-001:streams:data.instrument.BINANCE.UNFIUSDT-PERP MAXLEN 1 (integer) 3

cjdsellers commented 4 months ago

Yes, so XTRIM with a MINID is available from Redis 6.2.0+ https://redis.io/docs/latest/commands/xtrim/

HeidXC commented 4 months ago

Updated to 7.2.4, it works, thanks

cjdsellers commented 4 months ago

I'll make sure to improve the docs to specify the minimum Redis version.

cjdsellers commented 4 months ago

Hey @VeraLyu

Just following up on this. The minimum supported Redis version (currently 6.2.0) is now better documented and we also check the version on connection and will log an error if below the minimum supported version https://github.com/nautechsystems/nautilus_trader/commit/326ebf19c62f736315be07f8bac02ed0ea0a5f26.