[Bug]: stockdatastream with SIP feed is providing incomplete data

julianito20 commented 6 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

I subscribe to trades for a large number of symbols on the streaming API (StockDataStream) using the SIP feed. I don't have any visible failures but there are full minutes where no data is received even though trading volumes are high. Example PACB delivered 20+ trades during minute 9:30 EST on May 20th 2024 but no trades during minutes 9:31 and 9:32. I am not sure if this is a problem with the feed or with the client failing silently when the async function is called.

Expected Behavior

I should receive trades during every minute or a failure to process should be logged

SDK Version I encountered this issue in

alpaca-py version v0.21.1

Steps To Reproduce

1.Subscribe to 200+ trades
2. Save a record of each trade received aggregating by minute
3. Check that there are trades being recorded every minute. 

Example script:

data_stream = StockDataStream(apikey, token, feed=DataFeed.SIP)

async def bar_callback(bar):

    # Get the current time rounded down to the nearest minute
    minute_rounded = bar.timestamp.replace(second=0, microsecond=0)

    # Construct the Redis key using the symbol and the current minute
    key = f"{bar.symbol}:{minute_rounded.strftime('%Y-%m-%d %H:%M')}"

    # Use the timestamp as the score in the sorted set
    # timestamp_score = now.timestamp()

    # Add the timestamp and price to the sorted set, using the price as the score as timestamps must be unique
    r.zadd(key, {bar.timestamp.timestamp():bar.price})

keys = ['PACB','ZTO',..., '200th symbol']

logging.basicConfig(format='%(asctime)s  %(levelname)s %(message)s',
                    level=logging.INFO)

symbol = keys

data_stream.subscribe_trades(bar_callback, *symbol)

data_stream.run()

Filled out the Steps to Reproduce section?

[X] I have entered valid steps to reproduce my issue or have attached a minimally reproducible case in code that shows my issue happening; and understand that without this my issue will be flagged as invalid and closed after 30 days.

Anything else?

No response

gnvk commented 6 months ago

@julianito20 Could you please send me your account ID so that I can check the server logs?

chazzmoney commented 2 months ago

Has there been any follow up on this? Verification that it is or is not a bug?

alpacahq / alpaca-py