search

zvtvz / zvt

modular quant framework.
https://zvt.readthedocs.io/en/latest/
MIT License
3.09k stars 845 forks source link

can joinquant_record_runner1 skip previous fetched records ? #118

Closed ranglang closed 3 years ago

ranglang commented 3 years ago 

2020-10-30 18:02:26,072  INFO  MainThread  run to 2020/4145
2020-10-30 18:02:26,079  INFO  MainThread  entity_id:stock_sz_300543,evaluate_start_end_size_timestamps result:2020-10-30 00:00:00,None,1,None
2020-10-30 18:02:26,079  INFO  MainThread  sleeping 1 seconds
2020-10-30 18:07:40,559  INFO  MainThread  run to 1/4145
2020-10-30 18:07:40,593  INFO  MainThread  entity_id:stock_sz_000001,evaluate_start_end_size_timestamps result:2020-10-30 00:00:00,None,1,None
2020-10-30 18:07:41,090  INFO  MainThread  finish recording <class 'zvt.domain.quotes.stock.stock_1d_kdata.Stock1dKdata'> for entity_id:stock_sz_000001,latest_timestamp:2020-10-30 00:00:00
foolcage commented 3 years ago

now for the FixedCycleDataRecorder the size always plus 1 it's just for handling unfinished kdata and it would be treated as finished when only got duplicated record in not realtime mode.

from your trace,it seems some exception happen and start from the first entity again.

The problem is that we could not confirm whether the previois fetched entity finished yet, it need to run twice to check,so ...

just ignore the exeption point and continue to the next is another solution, I could provide this option.