Live data support in DataPortal

[tibkiss]

A Zipline algo accesses historical and realtime data through DataPortal (via data.current property in handle_data()). This property calls DataPortal.get_spot_value() DataPortal.get_adjusted_value() to get the latest prices.

In live trading we'd like to get the latest data from the Broker, therefore a DataPortalLive class shall be created (based on DataPortal), which reaches out the BrokerClient to obtain the latest pricing data.

Depends on: https://github.com/zipline-live/zipline/issues/4

[mellertson]

I wrote a stock prediction system, which acts as a data provider to a Quantopian algo. The challenge I ran into is, I couldn't find a way to get my data (from an external system) into Quantopian, during live trading. Basically, my stock prediction system acts as a "signal" generator to Quantopian, telling the algo when and what to buy and sell.

It seems like the DataPortal class could be used to get data from any external system, including a system like mine, which generates "buy" and "sell" signals. Does this seem like a good fit for the DataPortal class? Or is there a better way to approach what I'm trying to do?

[bartosh]

Hi Mike

The challenge I ran into is, I couldn't find a way to get my data (from an external system) into Quantopian, during live trading. You can use fetch_csv for that. In live trading mode it's run every day before open: https://www.quantopian.com/posts/fetch-csv-during-live-trading

Regards, Ed

2017-06-11 4:36 GMT+03:00 Mike Ellertson notifications@github.com:

I wrote a stock prediction system, which acts as a data provider to a Quantopian algo. The challenge I ran into is, I couldn't find a way to get my data (from an external system) into Quantopian, during live trading. Basically, my stock prediction system acts as a "signal" generator to Quantopian, telling the algo when and what to buy and sell.

It seems like the DataPortal class could be used to get data from any external system, including a system like mine, which generates "buy" and "sell" signals. Does this seem like a good fit for the DataPortal class? Or is there a better way to approach what I'm trying to do?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/zipline-live/zipline/issues/8#issuecomment-307600319, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAxDEg6yGUH_J5OFAjzTCSdNN-4Updwks5sC0SVgaJpZM4Nb-1h .

-- BR, Ed

[mellertson]

Thanks Ed. I actually already implemented a few algorithms using fetch_csv(). It works well if the trading algorithm only needs one "buy/sell" signal per day. But, the system I've developed can send a "buy/sell" signal a few times a day, so I was looking for a solution to fetch buy/sell signals from an external source throughout the trading day.

I thought about making a web-services call in the handle_data() function. Of course that would work when running Zipline on my local system. But, do you know if Quantopian allows calls to external-web services? My guess is they might block those connections for security reasons.

Regards,

Mike

On Sun, Jun 11, 2017 at 6:51 AM, Ed Bartosh notifications@github.com wrote:

Hi Mike

The challenge I ran into is, I couldn't find a way to get my data (from an external system) into Quantopian, during live trading. You can use fetch_csv for that. In live trading mode it's run every day before open: https://www.quantopian.com/posts/fetch-csv-during-live- trading

Regards, Ed

2017-06-11 4:36 GMT+03:00 Mike Ellertson notifications@github.com:

I wrote a stock prediction system, which acts as a data provider to a Quantopian algo. The challenge I ran into is, I couldn't find a way to get my data (from an external system) into Quantopian, during live trading. Basically, my stock prediction system acts as a "signal" generator to Quantopian, telling the algo when and what to buy and sell.

It seems like the DataPortal class could be used to get data from any external system, including a system like mine, which generates "buy" and "sell" signals. Does this seem like a good fit for the DataPortal class? Or is there a better way to approach what I'm trying to do?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/zipline-live/zipline/issues/8#issuecomment-307600319 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAAxDEg6yGUH_ J5OFAjzTCSdNN-4Updwks5sC0SVgaJpZM4Nb-1h .

-- BR, Ed

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/zipline-live/zipline/issues/8#issuecomment-307627223, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHu1SzHW4FSLchlaB21Lo-12c2Bb_T7ks5sC-LGgaJpZM4Nb-1h .

--

Best regards,

Mike Ellertson Email: mdellertson@gmail.com

[bartosh]

Hi Mike,

But, do you know if Quantopian allows calls to external-web services? My guess is they might block those connections for security reasons.

As far as I know Quantopian doesn't allow any requests to external resources except of fetch_csv. All Python APIs and modules that can potentially request data from outside are blocked: https://www.quantopian.com/help#ide-module-import

Regards, Ed

2017-06-13 6:34 GMT+03:00 Mike Ellertson notifications@github.com:

Thanks Ed. I actually already implemented a few algorithms using fetch_csv(). It works well if the trading algorithm only needs one "buy/sell" signal per day. But, the system I've developed can send a "buy/sell" signal a few times a day, so I was looking for a solution to fetch buy/sell signals from an external source throughout the trading day.

I thought about making a web-services call in the handle_data() function. Of course that would work when running Zipline on my local system. But, do you know if Quantopian allows calls to external-web services? My guess is they might block those connections for security reasons.

Regards,

Mike

On Sun, Jun 11, 2017 at 6:51 AM, Ed Bartosh notifications@github.com wrote:

Hi Mike

The challenge I ran into is, I couldn't find a way to get my data (from an external system) into Quantopian, during live trading. You can use fetch_csv for that. In live trading mode it's run every day before open: https://www.quantopian.com/posts/fetch-csv-during-live- trading

Regards, Ed

2017-06-11 4:36 GMT+03:00 Mike Ellertson notifications@github.com:

I wrote a stock prediction system, which acts as a data provider to a Quantopian algo. The challenge I ran into is, I couldn't find a way to get my data (from an external system) into Quantopian, during live trading. Basically, my stock prediction system acts as a "signal" generator to Quantopian, telling the algo when and what to buy and sell.

It seems like the DataPortal class could be used to get data from any external system, including a system like mine, which generates "buy" and "sell" signals. Does this seem like a good fit for the DataPortal class? Or is there a better way to approach what I'm trying to do?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/zipline-live/zipline/issues/8# issuecomment-307600319 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAAxDEg6yGUH_ J5OFAjzTCSdNN-4Updwks5sC0SVgaJpZM4Nb-1h .

-- BR, Ed

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/zipline-live/zipline/issues/8#issuecomment-307627223 , or mute the thread https://github.com/notifications/unsubscribe- auth/ABHu1SzHW4FSLchlaB21Lo-12c2Bb_T7ks5sC-LGgaJpZM4Nb-1h .

--

Best regards,

Mike Ellertson Email: mdellertson@gmail.com

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/zipline-live/zipline/issues/8#issuecomment-307996969, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAxDM-VB1QRqFgAq2A5KExiSi6PG5C1ks5sDgM8gaJpZM4Nb-1h .

-- BR, Ed

[tibkiss]

Delivered with https://github.com/zipline-live/zipline/pull/28