binance / binance-public-data

Details on how to get Binance public data
1.54k stars 475 forks source link

Disordered dtat file for Futures AggTrade of 2023-10 #288

Open alaky2 opened 1 year ago

alaky2 commented 1 year ago

The last month data file: BTCUSDT-aggTrades-2023-10.zip is disordered. Here are some first lines: 1862715434,26951.0,0.003,4130731517,4130731517,1696118404732,false 1896999302,33944.3,0.001,4231745127,4231745127,1698418067575,false 1862715435,26950.9,0.005,4130731518,4130731518,1696118404758,true 1896999303,33944.4,0.006,4231745128,4231745132,1698418067575,false 1862715436,26951.0,0.001,4130731519,4130731519,1696118404760,false 1896999304,33944.5,0.01,4231745133,4231745133,1698418067575,false 1862715437,26950.9,0.023,4130731520,4130731520,1696118404769,true 1896999305,33944.6,0.001,4231745134,4231745134,1698418067575,false 1862715438,26951.0,0.001,4130731521,4130731521,1696118404771,false 1896999306,33944.7,0.002,4231745135,4231745136,1698418067575,false 1862715439,26950.9,0.855,4130731522,4130731545,1696118404796,true 1896999307,33944.9,0.088,4231745137,4231745140,1698418067575,false

as you can see the Trade ID jumps every other row.

CaymanTurtleBeach commented 11 months ago

The same issue occurs in November data. In both cases, if you look at time stamps, you'll see that a series of supposed aggTrade events (BTCUSDT) at the same time stamp (for OCT the time stamp is 1698418067575) are interspersed with what appear to be the 'correct' aggTrades, AND the Id of the presumably bogus aggTrades that all have the same time stamp increases sequentially by 1. In effect there are two intertwined series of monotonically increasing AggTrade Ids. But I am hard pressed to see how that could happen, and irked that it has not been fixed (given the addition of records with identical time stamps, it should be straightforward to find the bug). Do you have any further insights? Cheers Jack