Intraday momentum example never changes values

[carthoris]

I ran the intraday momentum example and let it run for a bit. The momentum percentages sometimes do not change. Actually for most of them they do not change. How can a stock have the exact same 0.09% momentum each minute? Even if a stock does not trade and does not change value then the new momentum should be 0%. Of the 8 stocks I put in the list, maybe 1 changes every minute? Shouldn't they all change? I have had one have a momentum of 6.27% every minute for the past 6 mins. That does not seem to make sense.

[grzesir]

Hi Jay,

Can you please send a screenshot?

Also, what time of day did you run the bot?

[carthoris]

Yes. Absolutely can send a screen shot. Will be tomorrow though before I can get back to it. I ran it about 30 mins after market opening. I do have some ideas about why this is happening and will spend time tomorrow getting you a screen shot and maybe some ideas about the issue. Thanks for responding and will get back to you with that information as soon as I can.

[grzesir]

Great, thank you. Looking forward to seeing what you send over.

[carthoris]

Below is a 3 minute run of intraday momentum strategy of 6 stocks, You will notice that there are only 2 changes over that period of time. I am still not familiar with Python but I do have a thought. If these are low volume stocks and there has not been a trade of them in the last minute, will Alpaca return the same bar information as the previous time? If so that would explain the identical momentum numbers. You can see IMBI has -0.84% over the entire period. If so the momentum code will have to keep track of the time stamps and use them as a reference. Same timestamps would return a momentum of 0%

2021-02-12 08:52:01,182: INFO: Strategy IntradayMomentum: Executing the initialize lifecycle method 2021-02-12 08:52:01,646: INFO: Strategy IntradayMomentum: Executing the before_starting_trading lifecycle method 2021-02-12 08:52:01,703: INFO: Strategy IntradayMomentum: Executing the on_trading_iteration lifecycle method 2021-02-12 08:52:01,703: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 08:52:04,125: INFO: IMBI has a return value of -0.84% over the last 1 minutes(s). 2021-02-12 08:52:04,225: INFO: TRVN has a return value of -0.19% over the last 1 minutes(s). 2021-02-12 08:52:04,403: INFO: HMTV has a return value of 0.84% over the last 1 minutes(s). 2021-02-12 08:52:04,571: INFO: TAST has a return value of -0.16% over the last 1 minutes(s). 2021-02-12 08:52:04,685: INFO: APRN has a return value of 0.65% over the last 1 minutes(s). 2021-02-12 08:52:04,781: INFO: SOLO has a return value of 0.50% over the last 1 minutes(s). 2021-02-12 08:52:04,781: INFO: HMTV best symbol. 2021-02-12 08:52:04,781: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 08:52:04,893: INFO: Strategy IntradayMomentum: Sleeping for 60 seconds 2021-02-12 08:53:04,895: INFO: Strategy IntradayMomentum: Executing the on_trading_iteration lifecycle method 2021-02-12 08:53:04,896: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 08:53:05,047: INFO: IMBI has a return value of -0.84% over the last 1 minutes(s). 2021-02-12 08:53:05,122: INFO: TRVN has a return value of -0.19% over the last 1 minutes(s). 2021-02-12 08:53:05,314: INFO: HMTV has a return value of 0.84% over the last 1 minutes(s). 2021-02-12 08:53:05,412: INFO: TAST has a return value of -0.16% over the last 1 minutes(s). 2021-02-12 08:53:05,503: INFO: APRN has a return value of 0.60% over the last 1 minutes(s). 2021-02-12 08:53:05,592: INFO: SOLO has a return value of 0.50% over the last 1 minutes(s). 2021-02-12 08:53:05,592: INFO: HMTV best symbol. 2021-02-12 08:53:05,593: INFO: Keeping 876 shares of HMTV 2021-02-12 08:53:05,593: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 08:53:05,823: INFO: Strategy IntradayMomentum: Sleeping for 60 seconds 2021-02-12 08:54:05,825: INFO: Strategy IntradayMomentum: Executing the on_trading_iteration lifecycle method 2021-02-12 08:54:05,825: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 08:54:05,928: INFO: IMBI has a return value of -0.84% over the last 1 minutes(s). 2021-02-12 08:54:06,018: INFO: TRVN has a return value of -0.19% over the last 1 minutes(s). 2021-02-12 08:54:06,113: INFO: HMTV has a return value of 0.84% over the last 1 minutes(s). 2021-02-12 08:54:06,192: INFO: TAST has a return value of -0.16% over the last 1 minutes(s). 2021-02-12 08:54:06,270: INFO: APRN has a return value of 0.60% over the last 1 minutes(s). 2021-02-12 08:54:06,379: INFO: SOLO has a return value of -0.43% over the last 1 minutes(s). 2021-02-12 08:54:06,379: INFO: HMTV best symbol. 2021-02-12 08:54:06,379: INFO: Keeping 876 shares of HMTV 2021-02-12 08:54:06,379: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 08:54:06,568: INFO: Strategy IntradayMomentum: Sleeping for 60 seconds

[SlimBeji]

Hello Carthoris,

It seems the momentums displayed are daily momentums. This is why they did not change. We are investigating the matter.

What version do you get when you type the following command in your terminal:

pip show lumibot

[carthoris]

If they are the daily momentums then why would the few that change be changing at all?

I installed the code into PyCharm by having PyCharm pull directly off of github. So I do not have pip on the machine. I can run pip within PyCharm but it does not recognize that I downloaded the package. I am hoping the code on github is the latest.

Is there any other way to find the version that was pulled from github? I am looking.

I just had PyCharm do a git pull update and now it won't even run. Trying to figure that out now.

[carthoris]

Got it running again after the git update. Here is another 3 min run. It has only 1 change that I can see on SOLO.

2021-02-12 11:30:12,536: INFO: IMBI has a return value of 0.17% over the last 1 minutes(s). 2021-02-12 11:30:12,725: INFO: TRVN has a return value of 0.00% over the last 1 minutes(s). 2021-02-12 11:30:12,866: INFO: HMTV has a return value of 0.88% over the last 1 minutes(s). 2021-02-12 11:30:13,004: INFO: TAST has a return value of -0.31% over the last 1 minutes(s). 2021-02-12 11:30:13,114: INFO: APRN has a return value of -0.05% over the last 1 minutes(s). 2021-02-12 11:30:13,244: INFO: SOLO has a return value of 0.12% over the last 1 minutes(s). 2021-02-12 11:30:13,244: INFO: HMTV best symbol. 2021-02-12 11:30:13,245: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 11:30:13,387: INFO: Strategy IntradayMomentum: Sleeping for 60 seconds 2021-02-12 11:31:13,389: INFO: Strategy IntradayMomentum: Executing the on_trading_iteration lifecycle method 2021-02-12 11:31:13,390: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 11:31:13,511: INFO: IMBI has a return value of 0.17% over the last 1 minutes(s). 2021-02-12 11:31:13,704: INFO: TRVN has a return value of 0.00% over the last 1 minutes(s). 2021-02-12 11:31:13,834: INFO: HMTV has a return value of 0.88% over the last 1 minutes(s). 2021-02-12 11:31:13,951: INFO: TAST has a return value of -0.31% over the last 1 minutes(s). 2021-02-12 11:31:14,082: INFO: APRN has a return value of -0.05% over the last 1 minutes(s). 2021-02-12 11:31:15,004: INFO: SOLO has a return value of 0.00% over the last 1 minutes(s). 2021-02-12 11:31:15,005: INFO: HMTV best symbol. 2021-02-12 11:31:15,005: INFO: Keeping 875 shares of HMTV 2021-02-12 11:31:15,005: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 11:31:15,224: INFO: Strategy IntradayMomentum: Sleeping for 60 seconds 2021-02-12 11:32:15,225: INFO: Strategy IntradayMomentum: Executing the on_trading_iteration lifecycle method 2021-02-12 11:32:15,226: INFO: Strategy IntradayMomentum: Porfolio value of 10000 2021-02-12 11:32:15,390: INFO: IMBI has a return value of 0.17% over the last 1 minutes(s). 2021-02-12 11:32:15,612: INFO: TRVN has a return value of 0.00% over the last 1 minutes(s). 2021-02-12 11:32:15,764: INFO: HMTV has a return value of 0.88% over the last 1 minutes(s). 2021-02-12 11:32:15,862: INFO: TAST has a return value of -0.31% over the last 1 minutes(s). 2021-02-12 11:32:15,965: INFO: APRN has a return value of -0.05% over the last 1 minutes(s). 2021-02-12 11:32:16,104: INFO: SOLO has a return value of 0.00% over the last 1 minutes(s). 2021-02-12 11:32:16,105: INFO: HMTV best symbol.

[SlimBeji]

The momentum is calculated with the bar "close" value. The previous day "close" value cannot of course change but the current day "close" value can.

[SlimBeji]

The data is being pulled using alpaca API

You can play with the data returned to compare.

# Importing alpaca python api
import alpaca_trade_api as tradeapi
# Authentication
api = tradeapi.REST(YOUR_API_KEY, YOUR_API_SECRET)
# getting daily data, 2 for running and previous day
data = api.get_barset("IMBI", "1D", 2)
print(data)

You can replace 1D with 1Min for minutes data

[carthoris]

SO it is not based on a minute by minute momentum change which I was assuming that intraday momentum meant. It could be based on last trade and last trade time, but I guess I will write that one.

[SlimBeji]

We are investigating the matter because it should be based on a minute by minute data.

[SlimBeji]

I suppose you customized the strategy a little bit. Could you copy paste the strategy code please?

[SlimBeji]

Another explanation could be that you are using Yahoo as a data source. If this is the case, Yahoo gives only day by day data.

[carthoris]

I did not customize anything, only changed the stock list in the strategy. I am using all default as it comes from github. Not sure which data source comes by default.

[SlimBeji]

Okay I identified the issue. Thank you for your help. The strategy is pulling minutes data however there is an issue with how the momentum is calculated.

When you pull minutes data like so directly from alpaca

api = tradeapi.REST(YOUR_API_KEY, YOUR_API_SECRET)
# getting daily data, 2 for running and previous day
data = api.get_barset("IMBI", "1Min", 100)
# Getting the dataframe
df = data.df

If you look closely, sometimes even though it is minutes data, Alpaca skip some minutes.

Momentum (over a minute) is being calculated by checking the latest 2 rows like so:

data = api.get_barset("IMBI", "1Min", 2)
df = data.df
df["close"].pct_change()[-1]

The problem of this approach is that it supposes that alpaca always returns minute per minute data while in reality it can skips some minutes. In this case, the void should be filled with the same data as the latest available bar.

We will make an update so that this particular case is taken into account.

[carthoris]

Thanks for figuring out that so quickly. Is it possible that Alpaca could be returning the same minutes in subsequent calls? Or is that not possible.

[carthoris]

Another question regarding this specific issue. Given the explanation would the problem not manifest by wrong results instead of identical results?

[SlimBeji]

2021-02-12 11:31:13,511: INFO: IMBI has a return value of 0.17% over the last 1 minutes(s).

So at 11:31, lumibot requests from alpaca the latest two minute bars. Lumibot is falsely assuming that it would get the 11:30 and 11:29 bars from alpaca. In reality it will get for example 11:24 and 11:21 bars because these two are the latest available bars from alpaca. The momentum would be the bars closing value change recorded from 11:21 to 11:24 instead of being zero since no change has been observed since 11:24.

At 11:32, lumibot would again request the latest two minutes bars from alpaca. If there is still no new bars then again lumibot will get the 11:24 and 11:21 bars resulting in the same computations and same result.

So the results may be both wrong and identical to the latest computation.

[carthoris]

So if it gets the same two bars will the momentum be 0?

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: SlimBeji notifications@github.com Sent: Sunday, February 14, 2021 6:04:10 AM To: Lumiwealth/lumibot lumibot@noreply.github.com Cc: Jay Goldberg jgoldberg@pros.com; Author author@noreply.github.com Subject: [EXTERNAL] Re: [Lumiwealth/lumibot] Intraday momentum example never changes values (#26)

2021-02-12 11:31:13,511: INFO: IMBI has a return value of 0.17% over the last 1 minutes(s).

So at 11:31, lumibot requests from alpaca the latest two minute bars. Lumibot is falsely assuming that it would get the 11:30 and 11:29 bars from alpaca. In reality it will get for example 11:24 and 11:21 bars because these two are the latest available bars from alpaca. The momentum would be the bars closing value change recorded from 11:21 to 11:24 instead of being zero since no change has been observed since 11:24.

At 11:32, lumibot would again request the latest two minutes bars from alpaca. If there is still no new bars then again lumibot will get the 11:24 and 11:21 bars resulting in the same computations and same result.

So the results may be both wrong and identical to the latest computation.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/Lumiwealth/lumibot/issues/26*issuecomment-778768261__;Iw!!EQ82F1lD2o0!Jykjxa13n9JiwPXptKg6-cWChqVQ4t5EeMvMsOZWNB4ojsFJsqQCnkbRGPAfEws$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AA6SCEA3J2MCQWWGNQ62USLS6633VANCNFSM4XLIO5YQ__;!!EQ82F1lD2o0!Jykjxa13n9JiwPXptKg6-cWChqVQ4t5EeMvMsOZWNB4ojsFJsqQCnkbRzUONOwI$.

[SlimBeji]

No

From the above example, if 11:24 bar has a closing price of 123.0 and the 11:21 has a closing price of 121.0 for example, the momentum returned would always be (123 - 121)/121 until alpaca has a new bar.

We are working on this issue and you will be notified when fixed.

[carthoris]

If there are no new bars coming from Alpaca then the data will be flawed. Yes, it will return the same value based on the data given. But if the momentum algorithm sees the same 2 bars in subsequent calculations then it should return a 0. The bar API from alpaca does return a timestamp. It does not seem that the momentum algorithm uses or saves the timestamps of the previous calculation. If the momentum algorithm did in fact store and use the timestamps for each calculation then it would be able to determine that it had already used these bars and there has been no change since the last iteration. If there has been no change since the last iteration returning the same value would be incorrect. The momentum for the previous minute should not be the same even though it is using the same data.

From: SlimBeji notifications@github.com Sent: Sunday, February 14, 2021 10:45 AM To: Lumiwealth/lumibot lumibot@noreply.github.com Cc: Jay Goldberg jgoldberg@pros.com; Author author@noreply.github.com Subject: [EXTERNAL] Re: [Lumiwealth/lumibot] Intraday momentum example never changes values (#26)

No

From the above example, if 11:24 bar has a closing price of 123.0 and the 11:21 has a closing price of 121.0 for example, the momentum returned would always be (123 - 121)/121 until alpaca has a new bar.

We are working on this issue and you will be notified when fixed.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/Lumiwealth/lumibot/issues/26*issuecomment-778804072__;Iw!!EQ82F1lD2o0!OFTi1lHPgq9vSV9Fcbkk9VtnvAs0yxCrp5_dzqhs-cBbkcPqMvJAMcDbsfh_8Nw$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AA6SCEDHQTTA337EC4FLUJTS674Z5ANCNFSM4XLIO5YQ__;!!EQ82F1lD2o0!OFTi1lHPgq9vSV9Fcbkk9VtnvAs0yxCrp5_dzqhs-cBbkcPqMvJAMcDbYKxl894$.

[SlimBeji]

It would be easier to systematically scan the DataFrame returned by Alpaca and post-process it to fill the void. We are going to implement this.

[SlimBeji]

Release 0.0.6 and commit 8c5c6c4 should solve the issue