sdementen / piecash

Pythonic interface to GnuCash SQL documents
Other
290 stars 73 forks source link

decimal.InvalidOperation when using yahoo_client to update_prices for my Commodities #202

Closed JCPvanHooren closed 5 months ago

JCPvanHooren commented 2 years ago

First off: thank you for creating piecash! It creates an opportunity to do something I've been wanting to do for a long time: update historical prices for my commodities in GnuCash. If/when I get this to work, it'll save me a lot of workarounds.

Unfortunately however, when using piecash.core.commodity.Commodity.update_prices(), I get the following error:

yahoo_client.py, line 104, in <listcomp>
    *[Decimal(f) for f in data[1:]]
decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]

I'm trying to retrieve prices for a fund with mnemonic 'ROLG.AS' over the period 2020-10-01 until today (2020-10-19), but it seems to happen with all funds and always with "yesterday's" price.

Unfortunately I'm not good enough with Python to decipher why / which value is making it carsh. Would you be able to identify and maybe fix?

JCPvanHooren commented 2 years ago

OK, I got one step further. Although I'm surprised no-one else would've run into this before, it seems it's tripping over getting a date from yahoo, but then all null values.

below, the values it received in csv_data

[['Date', 'Open', 'High', 'Low', 'Close', 'Adj Close', 'Volume'], ['2022-10-03', '233.289993', '233.289993', '233.289993', '233.289993', '233.289993', '0'], ['2022-10-04', '237.160004', '237.160004', '237.160004', '237.160004', '237.160004', '0'], ['2022-10-05', '238.720001', '238.720001', '238.720001', '238.720001', '238.720001', '0'], ['2022-10-06', '236.449997', '236.449997', '236.449997', '236.449997', '236.449997', '0'], ['2022-10-07', '231.839996', '231.839996', '231.839996', '231.839996', '231.839996', '0'], ['2022-10-10', '232.279999', '232.279999', '232.279999', '232.279999', '232.279999', '0'], ['2022-10-11', '231.639999', '231.639999', '231.639999', '231.639999', '231.639999', '0'], ['2022-10-12', '230.330002', '230.330002', '230.330002', '230.330002', '230.330002', '0'], ['2022-10-13', '231.919998', '231.919998', '231.919998', '231.919998', '231.919998', '0'], ['2022-10-14', '231.000000', '231.000000', '231.000000', '231.000000', '231.000000', '0'], ['2022-10-17', '234.240005', '234.240005', '234.240005', '234.240005', '234.240005', '0'], ['2022-10-18', 'null', 'null', 'null', 'null', 'null', 'null']]

JCPvanHooren commented 2 years ago

OK, so I could fix the error by using 0, where it returns 'null' in csv_data, hence confirming the root cause. I'm certain someone can find a better code solution than what I did to avoid updating 'null' prices and I also don't know if yahoo_client has further dependencies to consider.

Hope this helps to fix in a next version.

Wm-X commented 1 year ago

there are multiple issues with getting data from y! that are not piecash specific. you can follow the horror over at https://github.com/ranaroussi/yfinance and related. Many people are concerned about this, I think yfinance or someone else will fix these problems before piecash so I suggest people look for answers there.