Open koradiyakaushal opened 1 year ago
I've found a solution and will share it in a few days, used the data frame to remove duplicates by (candle_start_time and symbol) and keep last, and used that as the final list.
@koradiyakaushal Does your solutions work? I'm curious for your solutions
@PeetCrypto I will share solution tomorrow, but few pointers on how it works, I use it for Postgres backend, instead of batch insert, I use batch upsert
it still has one issue as given in the above issue description solved using dataframe to remove duplicate open candle time and symbol subsets then used dataframe to batch upsert.
Is your feature request related to a problem? Please describe. Instead of the Insert statement for Postgres backend for kline stream, where is_closed param is not available in exchange(ex: kucoin) response, to use upsert statement to update the low, high, close, volume data in the database, to avoid duplicate candles
Describe the solution you'd like Upsert could be a solution for that solution, I've made a rough solution, like below using custom columns
in write_batch function use it like
await self.conn.execute(self.insert_statement + args_str + self.upsert_statement)
There are some issues with these upsert solutions like batch insert/batch upsert
what will be the standard way to solve this issue?