Missing adj_close column still causing me problems

[EcoFin]

Farrell, I keep coming back to testing pinkfish because it so clean and efficient! But here is something I don't understand: It seems to me that, if I set use_adj = False, that should tell pf just to look at a 'close' column, ignoring whether 'adj_close' is there or not.

But when I do that, example 200 still blows up because fetch.select_tradeperiod at line 205 requires an adj_close column even when use_adj is set to False.

I don't see that it's a problem for portfolio.py. In fact, your example 200 just sets fields = ['close']. I would think that if I don't add 'adj_close' to that list, example 200 should run fine.

So two questions:

1. is there something I'm too dull to understand about what use_adj is for? and
2. do you see any easy way to tell select_tradeperiod that if use_adj=False there is no need to look for an adj_c;lose column?

I know there are other ways around it without altering the pf codebase. But I thought you might have an idea.

I have two two use cases:

1. index data doesn't have any adjustments

2. my price database puts adjusted data into 'close' and unadjusted data into ... yup, "unadjusted close'

   Best regards arthur

[fja05680]

Hi again Arthur,

Glad you stuck around, or keep trying out pinkfish. You may have noticed that I fixed most of the other issues that you raised before. Your input made me aware of some issues and resulted in improving pinkfish. Some of the best improvements I've made have come from user suggestions.

In regards to this particular issue, I agree. If you aren't using adj_close, it shouldn't do any operations with that column. I'll take a look at it this weekend and push up a fix.

Farrell

[EcoFin]

Farrell,

Great to hear from you! I have been playing with the newest version from pypi; I know there are a few new things on github.

I’m glad I was not completely confused about use_adj.

For some other experiments, I just wrote code to pre-process long-run index into yahoo format. That’s fine for a small number of data files.

But then I began to wonder how big a portfolio I could work with using pf. Pre-processing the entire US equity market is a bit cumbersome, although it could be done. I thought that perhaps example 200 just set fields = [‘close’] to conserve memory in cases where the entire OHLCV structure wasn’t needed.

I’d like to get to the point where pf was my primary research platform, so I’m trying little experiments in various directions that are of interest to me.

Best wishes

arthur

From: Farrell Aultman @.> Sent: Thursday, March 17, 2022 12:02 PM To: fja05680/pinkfish @.> Cc: EcoFin @.>; Author @.> Subject: Re: [fja05680/pinkfish] Missing adj_close column still causing me problems (Issue #53)

Hi again Arthur,

Glad you stuck around, or keep trying out pinkfish. You may have noticed that I fixed most of the other issues that you raised before. Your input made me aware of some issues and resulted in improving pinkfish. Some of the best improvements I've made have come from user suggestions.

In regards to this particular issue, I agree. If you aren't using adj_close, it shouldn't do any operations with that column. I'll take a look at it this weekend and push up a fix.

Farrell

— Reply to this email directly, view it on GitHub https://github.com/fja05680/pinkfish/issues/53#issuecomment-1071017154 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3HOEXCCIDIFJA6QE7LQKDVANJPFANCNFSM5Q5PQKTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread.Message ID: @.***>

[fja05680]

Author,

I have made a fix for this issue. You can see it in pinkfish/fetch.py. When I get time, I'd like to add support for Norgate, which I believe is what you are using? Thanks again for your input.

Farrell

[EcoFin]

Farrell,

Great! I will take a look today. You’re right, I use Norgate for price data. I have written a couple of bits of code to copy data files from the Norgate repository to a pf data folder and then do the bits of processing needed to get them into a format compatible with the current versions of pf

I attached a sample in case you wanted to see it.

Actually, there is a good deal more information in the Norgate db that never gets to the csv exports. Were you to look at the notes on the Norgate_data pypi site you’d see some very simple code for connecting to Backtrader. I do a bit more elaborate stuff in the code I run for my own equity and ETF rebalancing procedures.

I’m continuing my pf experiments!

Thanks again

arthur

From: Farrell Aultman @.> Sent: Friday, March 18, 2022 2:43 AM To: fja05680/pinkfish @.> Cc: EcoFin @.>; Author @.> Subject: Re: [fja05680/pinkfish] Missing adj_close column still causing me problems (Issue #53)

Author,

I have made a fix for this issue. You can see it in pinkfish/fetch.py. When I get time, I'd like to add support for Norgate, which I believe is what you are using? Thanks again for your input.

Farrell

— Reply to this email directly, view it on GitHub https://github.com/fja05680/pinkfish/issues/53#issuecomment-1072076620 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3HOEW24STULCQSQ64JSB3VAQQWJANCNFSM5Q5PQKTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread.Message ID: @.***>