cmallwitz / Financials-Extension

Extension for LibreOffice Calc to access stock market data
Other
138 stars 17 forks source link

General feedback from users requested (create new issue for specific questions) #10

Open cmallwitz opened 3 years ago

cmallwitz commented 3 years ago

Please leave feedback here about what platforms you use the extension on, what you use it for, what symbols/exchanges are important to you and what you think is missing.

SoftwareExplorerMe commented 1 year ago

Downloaded new 3.0.9 and installed (after removing 3.0.7) and Voila, my original sheet worked again with all Ticker symbols. However the COPY/PASTE FORMULA Then changing Ticker Symbol in formula still doesn't work. But THANKFULLY, the COPY/PASTE CELL DOES work for the existing Tickers. Just can't add in any new ones?!?!

gmishkin commented 1 year ago

Debian bullseye Linux amd64

Version: 7.0.4.2 Build ID: 00(Build:2) CPU threads: 4; OS: Linux 5.10; UI render: default; VCL: kf5 Locale: en-US (en_US.UTF-8); UI: en-US Debian package version: 1:7.0.4-4+deb11u4 Calc: threaded

Found on Stack Overflow

Using it for US mutual funds and ETFs. Yahoo latest

Any way we can support the project?

spacecatz03 commented 1 year ago

May just be me but I think the Yahoo cookie issue is back, getting the dreaded #N/A again today.

cmallwitz commented 1 year ago

Yahoo deployed a major change to their website - can you try the most recent release 3.2.0 ?

spacecatz03 commented 1 year ago

Oh, didn't realize the recent update. Yep, that works, thanks!

TheSerapher commented 1 year ago

Hey @cmallwitz,

I updated to 3.2.1 just now and most things work again but seeing this error for a few quotes:

trace.log

2023-01-05 11:18:06.335574 getRealtime *args=('0P000083LP.F', 21.0, 'YAHOO') r=''<' not supported between instances of 'float' and 'NoneType'' 0.090 ms

These are the entries in my table:

=GETREALTIME("0P000083LP.F";21;"YAHOO")
=GETREALTIME("UI3G.F";21;"YAHOO")
=GETREALTIME("U1IC.F";21;"YAHOO")
=GETREALTIME("U1IH.F";21;"YAHOO")

No cache files are written for these calls. Maybe something on Yahoo side changed with the format of the HTML that the scripts can't work with?

cmallwitz commented 1 year ago

Can you check the formulas? If I enter the formulas using ; they get converted to , (trace.log is independent of this)

Maybe Yahoo delivers you a different version of HTML than to me (your formulas work for me).

Could you attach (HOME directory) .financials-extension/yahoo-0P000083LP.F.html here ?

cmallwitz commented 1 year ago

Hmm - having just said that now I'm getting the error too - not that I know why :-(

Durhammer3111 commented 1 year ago

Great tool, sir! Just installed 3.2.1 on LO 7.0.4.2, running under MX Linux 21.2 . Extension works great, but had to debug its installation (needed LO package for Python scripting), and had to learn the hard way about the LO Calc "AutoCalculate" setting being default. With that setting on, modifying ANY cell, even just a text header -- churns and recalculates. Had to use your ticker test suggestion to find what Yahoo needed for some preferreds, but other than that, everything is working FINE. Thanks again!

TheSerapher commented 1 year ago

Can you check the formulas? If I enter the formulas using ; they get converted to , (trace.log is independent of this)

I think this is just a LibreOffice setting how to read formulas, shouldn't make any difference for the error message resulting.

Maybe Yahoo delivers you a different version of HTML than to me (your formulas work for me).

Could you attach (HOME directory) .financials-extension/yahoo-0P000083LP.F.html here ?

Wish I could, but the calls fail before the HTML is cached, probably because the evaluation of the current value isn't working? Rather thought that because it was looking for a float but got nothing.

If you have an easy way for me to dump the HTML even though the error happens let me know. Then I could attach it.

TheSerapher commented 1 year ago

As a sidenote, seeing that Yahoo has extracted some information from HTML and added it to Javascript instead I raise the question if their goal is to make parsing really hard or next to impossible. It may be that they are actively trying to stop this kind of requests which would mean we'll have to find other means of fetching this data.

After my research there are only APIs available for paid services which is really annoying considering this is public information. So probably not a great option either.

This is speculation of course but maybe one that comes true.

TheSerapher commented 1 year ago

Switched my YT calls to the proper symbols on Financial Times and they work just fine. Also it's much faster, so I might stick to that.

Obviously for others the issue may still exist but I have found a workaround for it :)

cmallwitz commented 1 year ago

I will keep looking into this - it should save the HTML before parsing anything - so something isn't quite right.

Re: paid services - you are absolutely right - not an option for most people. Additionally even if a paid service has a "free" tier, they have the annoying habit of e.g. only allowing a fixed number of lookups in any given minute (let's say 5). This means you can't lookup your favourite 10 ETFs at once, even if you only do it once a day.

alfonsi4 commented 1 year ago

I just installed it on Libre Office using Windows 10.... it works for what I need.

Thank You!

Honminhui commented 1 year ago

This is really great cmallwitz! One feature that might be nice is the ability to select the prior trading day in the case where the date selected is not a trading day in the historic data.

OXOOOOE9XD commented 1 year ago

I installed the extension today to test. And endup with several errors while enabling it.

This is the Libra Office verision I tested on. Version: 7.4.3.2 (x64) / LibreOffice Community Build ID: 1048a8393ae2eeec98dff31b5c133c5f1d08b890 CPU threads: 4; OS: Windows 10.0 Build 19044; UI render: Skia/Raster; VCL: win Locale: en-IN (en_IN); UI: en-US Calc: threaded

Following ZIP contain all the snapshot of error I capture. Finanace Errors.zip

I couldn't able to find log file in LibraOffice folder sorry for that.

roadskater commented 1 year ago

Thanks for a great extension! Perhaps this cell formula will be useful in the readme to help some whenever YAHOO results fail but the same info is available from FT, for example. It uses the data from YAHOO unless it is #N/A, in which case it uses the data from FT (which might of course be #N/A). [I guess it would be possible for the extension to implement a fallback feature as well.]

[Edited to reflect comments below from extension author.] =IFNA(GETREALTIME("QQQ",21,"YAHOO"),GETREALTIME("QQQ:NMQ:USD",21,"FT"))

https://wiki.documentfoundation.org/Documentation/Calc_Functions/IFNA

cmallwitz commented 1 year ago

You certainly can check if you get the data you expect with IFNA but I would be careful with using a ticker symbol from Yahoo on FT. In general the symbols are slightly different and you may not get the data you are expecting either. Having said that at least for US equity FT seems to accept the ticker symbol and redirects to the main symbol for FT (this happens for QQQ in your example). Note: that requires an additional network roundtrip and makes downloading symbol data slower...

Take a look at the URLs below and experiment a little with the URL paramaters:

https://finance.yahoo.com/quote/QQQ?p=QQQ https://markets.ft.com/data/etfs/tearsheet/summary?s=QQQ:NMQ:USD

SoftwareExplorerMe commented 1 year ago

Strange thing that whenever I install a new extension version, the new tickers that I previously added that didn't work suddenly start working. But I still cannot add new tickers without getting #NAME? as the result. I even tried plain old "IBM" and it won't work. I tried uninstalling the extension then reinstalling , but that didn't fix it. I have 2 oddball tickers I tried to get from FT instead of Yahoo "OP:NSQ" & "ISO:NSQ" but that wouldn't work either. So keep up the good work?? US WIN 10 PRO LibreOffice 7.4.3.2 Extension 3.3.0 64bit

roadskater commented 1 year ago

Thank you for the response. I've updated my post, hopefully correctly, to reflect your comment. On Saturday, January 21, 2023 at 08:24:13 AM EST, cmallwitz @.***> wrote:
You certainly can check if you get the data you expect with IFNA but I would be careful with using a ticker symbol from Yahoo on FT....[See full reply above.]

Futur3H4wk commented 1 year ago

Thanks this is great! I was looking for information on how I might go about scraping info from CoinMarketCap or CopinGecko, into a spread sheet I'm using to track my crypto holdings. Found you had already done the heavy lifting. The only recommendation I would make is in your ReadMe file, you might make a mention that Yahoo also tracks crypto. I've never used Yahoo before and was disappointed that while CoinBase will display the prices for all my coins, I could not import the prices on any they don't currently have available on their platform (which was about half of them).

cmallwitz commented 1 year ago

"OP:NSQ" & "ISO:NSQ"

I would expect "ISO:NSQ" to work but for OP do you mean "OP:NAQ" ?

SoftwareExplorerMe commented 1 year ago

"OP:NSQ" & "ISO:NSQ"

I would expect "ISO:NSQ" to work but for OP do you mean "OP:NAQ" ?

Yes, was a typo in my response. I did try it both ways though. Neither worked for either stock. Still cannot add a new stock on sheet even IBM as a test.

jdoss2020 commented 1 year ago

Thank you so much for developing this extension. It works great for simply retrieving stock and mutual fund/ETF quotes from Yahoo. It enabled me to ditch my Google sheets investment spreadsheet and implement the same functionality on my home linux system using libreoffice. Bye, bye Google. So far I've experienced no issues. I really appreciate your efforts in developing and maintaining this solution.

oscby commented 1 year ago

Great extension, I use it tonnes. Adding a function to grab the next earnings date would be useful, yahoo finance provides it (some time in advance).

TheSerapher commented 1 year ago

Hey @cmallwitz,

Looks like Yahoo changed something again since I am unable to receive quotes from them. Haven't looked into it but if it's on their end you should be able to reproduce it.

cmallwitz commented 1 year ago

Yes - looks like it - I will take a look.

jbourvic commented 1 year ago

I find that they usually shut down access during weekends. No set time frame but later on everything starts working fine again.Sent via the Samsung Galaxy Note20 Ultra 5G, an AT&T 5G smartphone -------- Original message --------From: Sebastian Grewe @.> Date: 7/3/23 03:46 (GMT-06:00) To: cmallwitz/Financials-Extension @.> Cc: jbourvic @.>, Comment @.> Subject: Re: [cmallwitz/Financials-Extension] General feedback from users requested (create new issue for specific questions) (#10) Hey @cmallwitz, Looks like Yahoo changed something again since I am unable to receive quotes from them. Haven't looked into it but if it's on their end you should be able to reproduce it.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

cmallwitz commented 1 year ago

I bumped up the cookies and it seems to be working again. Please try the latest version.

TheSerapher commented 1 year ago

Hey @cmallwitz,

Installed the released version but still seeing no results for my last two remaining YAHOO calls. All others work just fine. When I open Yahoo I am able to find quotes for these two.

2023-07-06 08:34:58.025302 getRealtime *args=('IWDA.L', 21.0, 'YAHOO') r='None' 561.250 ms
[...]
2023-07-06 08:35:45.973728 getRealtime *args=('NU-EUR', 21.0, 'YAHOO') r='None' 217.603 ms
estianic commented 1 year ago

Here also no results from Yahoo calls, even after updating to version 3.3.1 with Libreoffice 7.3.7.2

Spreadsheet shows #N/A as result. Logfile shows r='None' for all calls. Downloaded .html files can not be opened with browser, gets stuck at the point where you have to accept of refuse cookies (both choices have no effect). Downloaded .html files from a week ago can be opened.

cmallwitz commented 1 year ago

Can you try deleting (user home directory) .financials-extension / yahoo.cookiejar and then restart LibreCalc?

estianic commented 1 year ago

It works after deleting yahoo.cookiejar

Thanks!

cmallwitz commented 1 year ago

I checked both and I got values.

On Thu, 6 Jul 2023 at 06:47, Sebastian Grewe @.***> wrote:

Hey @cmallwitz https://github.com/cmallwitz,

Installed the released version but still seeing no results for my last two remaining YAHOO calls. All others work just fine. When I open Yahoo I am able to find quotes for these two.

2023-07-06 08:34:58.025302 getRealtime args=('IWDA.L', 21.0, 'YAHOO') r='None' 561.250 ms [...] 2023-07-06 08:35:45.973728 getRealtime args=('NU-EUR', 21.0, 'YAHOO') r='None' 217.603 ms

— Reply to this email directly, view it on GitHub https://github.com/cmallwitz/Financials-Extension/issues/10#issuecomment-1623085877, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA226T7V7X443L7JV3WVQXLXOZNQZANCNFSM4UZXTGMQ . You are receiving this because you were mentioned.Message ID: @.***>

TheSerapher commented 12 months ago

Can you try deleting (user home directory) .financials-extension / yahoo.cookiejar and then restart LibreCalc?

That worked 🙂 Thanks a lot!

DrakeBliss commented 11 months ago

Great Extension! I am using it to track Dividend Stock. I am also wanting to track some ETF Funds. Yahoo does not have the dividend data for them. Theses for example do not work for getting annual dividend and ex-dividend dates.

https://finance.yahoo.com/quote/SVOL https://finance.yahoo.com/quote/TSLY

I do see the data for annual dividend and ex-dividend date on this website

https://www.nasdaq.com/market-activity/etf/svol https://www.nasdaq.com/market-activity/etf/tsly

Would it be a hard to add NASDAQ as one of the lookup providers?

Thank you for making this extension!

Ulyzzeus commented 11 months ago

This is really great cmallwitz! One feature that might be nice is the ability to select the prior trading day in the case where the date selected is not a trading day in the historic data.

you can do this manually using =IF(WEEKDAY(H82)=7;GETHISTORIC("EUR=X";"90";H82-1;"YAHOO");IF(WEEKDAY(H82)=1;GETHISTORIC("EUR=X";"90";H82-2;"YAHOO");GETHISTORIC("EUR=X";"90";H82;"YAHOO"))) where H82 is a cell with the date, but a shorter way would be nice

Futur3H4wk commented 10 months ago

I've set up a spread sheet using =GETREALTIME("BTC-USD",21,"YAHOO") and =GETREALTIME(B12,21,"YAHOO") with B12 having a stock symbol. It has worked great for several months. The last few days I've been getting an error Yahoo.getRealtimeSummary(PRWCX, 21) - urlopen: argument must be an int, or have a fileno() method. I'm not finding any answers on the internet. Was there an update LibreOffice that has caused this? is there a workaround?

Futur3H4wk commented 10 months ago

I did a kernel update and rebooted my server and VBox. now everthing is working again.

Durhammer3111 commented 10 months ago

I have NO COMPLAINT about this extension, it is FABULOUS. However, it would be interesting to see if there is a way to get the Yahoo! "Fair Value" information on a given ticker? As an example, currently TSLA shows it is "Near Fair Value" with a return of +11%. Would love to be able to pull that type of thing in. Thanks!

Mago-actor commented 9 months ago

I loved it, it is very powerful, congratulations. Just missing a user manual to facilitate it with all its possibilities. For example, I have not yet discovered how to see Euro Dollar values of other dates, not just the current one. But it is an excellent extension. Thank you.

svg1234 commented 9 months ago

I joined github today just to give a heads up to anyone else who might encounter the same problem that I did in trying to get the extension to work. I'm using Arch and the AppImage version of LO v7.6.1.2. I did not compile the extension from scratch on my system because I don't have LO or the dev tools installed. I just downloaded the compiled latest release of the extension from here. I managed to track down the problem. Despite the LO error message, in this case, the problem was that something was expecting libcrypt.so.1 and currently Arch is using libcrypt.so.2.0.0.

While it's not a great workaround, I merely created a softlink from what was expected to what is there, and presto, the extension works.

# ln -s /usr/lib/libcrypt.so.2.0.0 /usr/lib/libcrypt.so.1

But using that lame method I would have to create a new link whenever a system update installs an updated version of libcrypt.

Where is the call to the old version of libcrypt coming from? Could this issue be addressed in future releases of the extension? Or is it the LO AppImage? Thank you.

UPDATE: turns out there's a libcrypt backward compatibility package in Arch core repository: libxcrypt-compat
It contains the older version of libcrypt. This is a much more elegant solution.

Durhammer3111 commented 5 months ago

Things have been going well until late this afternoon, when trying to get Calc to run calculations (Ctrl+Shift+F9). The cursor spun briefly, and the sheet was flooded with junk. Repeated attempts did the same. Wondered if Yahoo! was having issues, so waited and opened up an old sheet and got a bit less of the same thing (way simpler sheet setup). The main thing I see is this:

Yahoo.getRealtimeSummary(SMCI, 67) - urlopen: argument must be an int, or have a fileno() method.                  

What's the generally recommended debugging approach? Is it generally this?

Can you try deleting (user home directory) .financials-extension / yahoo.cookiejar and then restart LibreCalc?

UPDATE: Yep, that seems to help a lot!

cmallwitz commented 5 months ago

There is no real debugging approach other than what you have quoted and then check the files.

Can you try the latest extension version to see if it makes a difference?

22 Jan 2024 23:16:22 Durhammer3111 @.***>:

Things have been going well until late this afternoon, when trying to get Calc to run calculations (Ctrl+Shift+F9). The cursor spun briefly, and the sheet was flooded with junk. Repeated attempts did the same. Wondered if Yahoo! was having issues, so waited and opened up an old sheet and got a bit less of the same thing (way simpler sheet setup). The main thing I see is this:

*

*Yahoo.getRealtimeSummary(SMCI, 67) - urlopen: argument must be an int, or have a fileno() method.

What's the generally recommended debugging approach? Is it generally this?

Can you try deleting (user home directory) .financials-extension / yahoo.cookiejar and then restart LibreCalc?

— Reply to this email directly, view it on GitHub[https://github.com/cmallwitz/Financials-Extension/issues/10#issuecomment-1904921568], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AA226TZONTODK73ZXS3IVDTYP3QLHAVCNFSM4UZXTGM2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJQGQ4TEMJVGY4A]. You are receiving this because you were mentioned. [Tracking image][https://github.com/notifications/beacon/AA226T7OFA5YVQVMQI3XKRDYP3QLHA5CNFSM4UZXTGM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOOGFMXYA.gif]

H0lderness commented 5 months ago

Hello @ cmallwitz

I have just started using the Financial Extension with LibreOffice Calc and I find the spreadsheet very slow to load. BUT the extension works and downloads prices OK it just takes a long time'

I am using Windows 10 latest version, 16 gb Ram, Processor AMD Ryzen 5 2600 Six-Core Processor, 3400 Mhz, 6 Core(s), so an inadequately powered PC should not be an issue.

I am not overly technical but his is what I did:-

Created two separate spread sheets one for FT and the other for Yahoo

I used two of the functions as shown in your webpage ie

=GETREALTIME("EURUSD","LAST_PRICE","FT")

=GETREALTIME("IBM",21,"YAHOO")

I inputted on each sheet Name, Ticker symbol and Price as headers, then inputted various securities, Shares, Investment Trusts, Mutual Funds and one Bond and one Index 32 in all. I then put the formula in each of the cells under price, just changing the Ticker symbol to the ones I wanted Everything went fine first time round on each sheet all Securities Index and Bond prices were fetched correctly. It is just a) so long to load the spreadsheet b) so slow to fetch the prices. Since then it is repeatable but SLOW.

Do you have any suggestion as to why this is happening and what I can do about it?

Thank you

Terry

SoftwareExplorerMe commented 5 months ago

Was working fine for a long time (although always slow) but everything come up N/A from Yahoo today.

Futur3H4wk commented 5 months ago

Same situation here. I did recalc 1 cell and it populated (others did not) and the next time I did a hard recalc it went back to na. I expect the problem is on the yahoo side of things. had a similar situation a few moths ago and it corrected it self after a few days.

Futur3H4wk commented 5 months ago

also H0lderness it always take quite a while to update a bunch of cells. Mine takes 3 to 5 minuets with about 35 cells to update.

Durhammer3111 commented 5 months ago

@Futur3H4wk, how are your calls arranged, per ticker? I now have two versions of my spreadsheet, one where there is one ticker per row with numerous different attribute calls, and another sheet where there is one ticker per column, with the same stock attribute calls. Both have over 60 tickers. The row-oriented sheet takes 10-15 minutes to update, but the column-oriented sheet takes 2 to 3 minutes. Big difference. I also learned to minimize the attributes so they can all be satisfied by the stock's summary file alone. Looking at the trace.log file showed me these things.

⁣Get BlueMail for Android ​

On Jan 30, 2024, 6:29 PM, at 6:29 PM, Futur3H4wk @.***> wrote:

also H0lderness it always take quite a while to update a bunch of cells. Mine takes 3 to 5 minuets with about 35 cells to update.

-- Reply to this email directly or view it on GitHub: https://github.com/cmallwitz/Financials-Extension/issues/10#issuecomment-1918086892 You are receiving this because you commented.

Message ID: @.***>

Futur3H4wk commented 5 months ago

In columns across several sheets. There are other calculations that are updated at the same time, some of them using the results of the calls. It’s not a big deal waiting a few minutes, I usually just play a game while I wait, unless I have something more important to do then I just look at it when I come back. I track crypto, stocks, IRAs and precious metals, all through Yahoo Finance (no subscription required), and I only check it a few times a day at most. This little piece of code is Fabu.