looks like Yahoo changed again

[criddle3]

=GETREALTIME(B4,21,"YAHOO") returns #N/A for all requests. As do Previous Close, Day Change, Percent Change

[cmallwitz]

Can you try the latest release https://github.com/cmallwitz/Financials-Extension/releases/tag/3.4.0

[criddle3]

Tried that first. Updated yesterday before posting this. It worked fine up until the morning of 1/30/24

I do see html files pulled down for today in the ~/.financials-extension directory with data for the symbols

[spacecatz03]

Just FYI, I had the #N/A issues as well, just uninstalled old added new, now works like a charm again. Thanks.

[criddle3]

Got it in my case: Uninstall -> Install -> Restart - still failed to pull Uninstall -> Restart -> Install -> Restart - working again

Thanks all

[cmallwitz]

Good to hear - unfortunately, on that level it is all LibreCalc's doing, not mine...

[Futur3H4wk]

Unfortunately that didn’t work for me. My problem was likely using an older version of LibreOffice. I was using my Distro’s stock version 7.3, the latest stable version is 7.6. Took me a while to get all the old pieces removed but I went with the FlatPak version 24.2 (no guts, no glory). After the install added the updated extension and wonder of wonders it is working like a charm again.

[cmallwitz]

I'm still on 7.3 too (Ubuntu 20.04) and it is working so I was hoping everyone else would be fine too. Unfortunately a LC extension has basically no chance to deal with any issues in a local LC installation. It just gets called and if it isn't, it doesn't even know about it.

[techman2423]

Got it in my case: Uninstall -> Install -> Restart - still failed to pull Uninstall -> Restart -> Install -> Restart - working again

Thanks all

i had the same issue and ended up installing the same to get rid of issue last week. however it's back again today. I had to repeat this process....remove 3.4.0 (new) and restart. install new and restart. runs again...

[spacecatz03]

Just an FYI, I got another #N/A, initially this morning but then realized it was a VPN issue, if I pause the VPN connection it will update but otherwise, may not.

[techman2423]

Its still doing it. I'll run through the uninstall, restart, install, restart again.

[dtw2000]

@cmallwitz I'm not sure this is helpful. I have a few linux machines (mint/ubuntu) on 7.3 and a windows machine - was on 7.3 upgraded to 7.6 because of this "issue". I have a file that is synced through syncthing that I open on the various machines. I seem to get the N/A issue very consistently every time I open the file on a different machine; however, I consistently fix it by disabling/enabling the extension which requires a restart of libre. I followed the other suggestion of uninstall/reinstall and thought, cool that works, so I did it on all my machines. Then, I kept getting the error. Trial and error suggested that disable/enable/restart seems to fix the issue almost every time. Sometimes 2x of disable/enable/restart is the key. Does that help any?

[techman2423]

Yes in fact the disable/reenable with restarts worked. It was random how many I had to do to make it finally work.

Any idea why this is going on?

On Feb 9, 2024, at 10:11 PM, dtw2000 @.***> wrote:



@cmallwitzhttps://github.com/cmallwitz I'm not sure this is helpful. I have a few linux machines (mint/ubuntu) on 7.3 and a windows machine - was on 7.3 upgraded to 7.6 because of this "issue". I have a file that is synced through syncthing that I open on the various machines. I seem to get the N/A issue very consistently every time I open the file on a different machine; however, I consistently fix it by disabling/enabling the extension which requires a restart of libre. I followed the other suggestion of uninstall/reinstall and thought, cool that works, so I did it on all my machines. Then, I kept getting the error. Trial and error suggested that disable/enable/restart seems to fix the issue almost every time. Sometimes 2x of disable/enable/restart is the key. Does that help any?

— Reply to this email directly, view it on GitHubhttps://github.com/cmallwitz/Financials-Extension/issues/98#issuecomment-1936851318, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFW42CSMDZCVHACPBWGAP2LYS3XORAVCNFSM6AAAAABCSBV2VOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZWHA2TCMZRHA. You are receiving this because you commented.Message ID: @.***>

[zamar]

I just hit this issue. Uninstalling + Reinstalling twice resolved the problem.

[eford321]

This problem continues for me. Sometimes uninstall/reinstall with restarts works, but only for a short while. Disable/reenable does not work for me.

[zamar]

I'm seeing the same. Sometimes uninstall/reinstall solves the problem. Then it might work again for a little while, and then the problem reappears.

[Futur3H4wk]

for a while now, disable restart enable restart has been working for me. today did this twice several different error messages no joy. uninstall restart install restart still no joy. I don't check what updates and when I have a script that updates all my programs be they manually installed, aptitude , or flatpack. I just run it a few times a day. so if libre made an update i probably installed it without even knowing it. What ever the cause this issue persists.

[cmallwitz]

I have no idea why deinstalling/installing the extension would change anything. The extension is a python script with a exposed function that gets called for every formula you have in a spread sheet. the script is put in a ZIP file, the ZIP extension is renamed and when the LC extension manger install the extension, it is unzipped to a user specific directory. As far as I'm aware no modifications are done in the installation step to any of the files in the extension.

if you put =GETREALTIME("SUPPORT") in a cell, you can see some of the information available to the extension on your system including file=... pointing to main, unzipped extension script.

BTW what system are you on? could anything modify files in the installation directory of the extension?

[Futur3H4wk]

https://termbin.com/phr0

It's a VBox

Linux 5.15.0-97-generic - Mint 21.2 - Cinnamon 5.8.4 - 13th Gen Intel© Core™ i5-13600K × 1 - memory 19.1 GiB

Version: 24.2.1.2 (X86_64) / LibreOffice Community Build ID: db4def46b0453cc22e2d0305797cf981b68ef5ac CPU threads: 1; OS: Linux 5.15; UI render: default; VCL: gtk3 Locale: en-US (en_US.UTF-8); UI: en-US Flatpak Calc: threaded

I don't see a zip file I probably deleted it Financials-Extension.oxt Sun 05 Feb 2023 11:42:14 AM EST

As to files being modified, I'm not that good with programming I just download and follow the read-me file. So it's possible

[beckjake]

It's not necessary to uninstall + reinstall the extension to fix the issue, it's enough to close and re-open calc. It can take a few tries.

My =GETREALTIME("SUPPORT") output:

ctx=pyuno object (com.sun.star.uno.XComponentContext)0x59f6aed3f548{, supportedInterfaces={com.sun.star.uno.XWeak,com.sun.star.lang.XComponent,com.sun.star.lang.XTypeProvider,com.sun.star.uno.XComponentContext,com.sun.star.container.XNameContainer}}
id(self)=138356130421328
version=3.4.0
file=/home/<username>/.config/libreoffice/4/user/uno_packages/cache/uno_packages/lu184479zgrpi.tmp_/Financials-Extension.oxt/financials.py
cwd=/home/<username>/Documents
home=/home/<username>
uname=Linux <hostname> 6.7.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 03 Mar 2024 00:30:36 +0000 x86_64 
pid=12681
sys.executable=/usr/bin/python3
sys.version=3.11.7 (main, Jan 29 2024, 16:03:57) [GCC 13.2.1 20230801]
sys.path=['/home/<username>/.config/libreoffice/4/user/uno_packages/cache/uno_packages/lu184479zgrpi.tmp_/Financials-Extension.oxt', '/usr/lib/libreoffice/program', '/usr/lib/python311.zip', '/usr/lib/python3.11', '/usr/lib/python3.11/lib-dynload', '/usr/lib/python3.11/site-packages']
locale=('en_US', 'UTF-8')
defaultlocale=('en_US', 'UTF-8')
dateutil=2.8.2
pytz=2023.3
pyparsing=2.4.7
six=1.16.0

There's no obvious difference in the above output between the working and non-working states (PID and the memory address of the pyuno object is it). Last time it wasn't working I looked at the html files and they had the tags I'd expect.

[Futur3H4wk]

Uninstalling and reinstalling never seemed to make a dif for me. but including that I've disabled and reenabled at least 5 times. and run an update script. still no joy. wish I could be of more help but point and click is about the best I can do.

[eford321]

BTW what system are you on? could anything modify files in the installation directory of the extension?

M2 Macbook running MacOS 14.2.1. My GETREALTIME("SUPPORT") is the same as beckjake's above, except sys.version=3.8.18

[spacecatz03]

FWIW, this has been happening to me more frequently (like right now), I just close LibreOffice, wait a minute or 2 and then it's usually okay.

[Futur3H4wk]

Several days and multiple system reboots and still no joy. Ubuntu even pushed a kernel update today and still not working. I love this extension and to be honest I’m a little lost when it’s not working. Now I’m noticing a separate issue with my vbox and NoMachine resurfacing, The joys of Linux when you’re not a coder.

[estianic]

Works fine here on native Mint 21.3/Kernel 6.1.0/Libreoffice 7.3.7.2 (distro version)/extension 3.4.0

=GETREALTIME("SUPPORT"):

ctx=pyuno object (com.sun.star.uno.XComponentContext)0x55c769882c00{, supportedInterfaces={com.sun.star.uno.XComponentContext,com.sun.star.container.XNameContainer,com.sun.star.lang.XTypeProvider,com.sun.star.uno.XWeak,com.sun.star.lang.XComponent}} id(self)=140004769450784 version=3.4.0 file=/home/username/.config/libreoffice/4/user/uno_packages/cache/unopackages/lu647034xzn.tmp/Financials-Extension.oxt/financials.py cwd=/home/username home=/home/username uname=Linux hostname 6.1.0-1034-oem #34-Ubuntu SMP PREEMPT_DYNAMIC Mon Feb 5 18:29:21 UTC 2024 x86_64 x86_64 pid=65346 sys.executable=/usr/bin/python3 sys.version=3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] sys.path=['/home/username/.config/libreoffice/4/user/uno_packages/cache/unopackages/lu647034xzn.tmp/Financials-Extension.oxt', '/usr/lib/libreoffice/program', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.10/dist-packages', '/usr/lib/libreoffice/program'] locale=('en_US', 'UTF-8') defaultlocale=('en_US', 'UTF-8') dateutil=2.8.2 pytz=2023.3 pyparsing=2.4.7 six=1.16.0

[beckjake]

I ran the existing html file through the parser manually in ipython and then started doing the steps from here onwards.

I noticed there was no attribute named 'value' on d when I got to here, but there is an attribute named 'data-value' that looked like the exact value I wanted. I updated the code there to be (default(d.attrib, 'value') or default(d.attrib, 'data-value')) in place of the existing default(d.attrib, 'value'), and have successfully restarted libreoffice multiple times and always get the values.

I tested it out by adding a if default(d.attrib, 'data-value') and not default(d.attrib, 'value'): logger.error("data-value and no value"), and indeed, I see it in extension.log a bunch.

If I have time this weekend I can open a PR for this, but feel free to just do it yourself if you have the time. Perhaps Yahoo Finance is A/B testing a new response format.

[Durhammer3111]

@Futur3H4wk pointed me to this discussion after I posted on the general feedback entry. One of the main points there is that all of the "yahoo-xxxx.html" files are merely somewhere between 57 and 63 bytes total in size. Nothing I have done is making the extension work again. Latest versions of LO and the extension, MX Linux 21.3 kept in regular maintenance updates.

[beckjake]

@Durhammer3111

I just hit the same error you got, though for me it looked like the files hadn't downloaded at all - which I think makes more sense, given the error and where the stack trace is pointing. I added some logging around the problem area but then couldn't reproduce it. It's probably small solace, but rest assured, it's not just you :). That error manifests much differently than the data-value one I hit earlier, it's some sort of error I failed to capture as opposed to the data-value issue's N/A result. What it kind of looked like to me was that maybe I was getting blocked entirely. This might seem like kind of a silly question, but can you access yahoo finance from your browser when you're having this problem? I sort of wish I was able to hit this consistently, as it'd be easier to debug.

Also, so far as I can tell there's no need to go through uninstalling + reinstalling the extension. You can just close and re-open libreoffice for the same effect. Possibly even doing a simple ctrl+shift+f9 (refresh all cells, which ends up re-doing the http queries as long as it's been more than a minute or so) is sufficient.

[Durhammer3111]

Hi @beckjake -- yes, I keep a Yahoo! tab open in my browser all the time, use it regularly, and have no problem there. This issue has been ongoing for me since March 5. And as @Futur3H4wk said, I'm kinda dead in the water when it's not working. @cmallwitz has built us a wonderful, addictive magical black box. and now I've got the shakes in withdrawal....

[Durhammer3111]

Soooo, @cmallwitz or anyone else who knows how to code Python and such, is there a simple way to turn on any useful debug logging that might help here?

[eford321]

This might seem like kind of a silly question, but can you access yahoo finance from your browser when you're having this problem?

Browser works fine for me also.

[estianic]

If it looks like the .html files aren't downloaded or they are very small, I think that points to a cookie problem.

First step would be to delete yahoo.cookiejar in the extension directory (.financials-extension on Linux).

[Durhammer3111]

Hi @estianic -- your comment makes me think you're on an older version of the extension. @cmallwitz changed it at some point so that there no longer is a cookie file.

[Futur3H4wk]

I'm running 3.3.0 and after deleting .financials-extension and launcinig librecalc the file reapeared. also did not fix the proiblem

[Durhammer3111]

• For what it's worth, I've been playing around with some Linux commands (curl, pup, etc.) in thoughts of building a bash-based system. Interestingly enough, I'm seeing some weird results when trying to use curl to download some Yahoo! files that should be there. It seems to happen most with preferred stock tickers -- for example, I get 404 errors with QRTEP, CDR-PC, and OPINL, while CCLDP is fine. Go figure. So far, no problem with regular stock tickers. Something's weird on the Yahoo end. The game is afoot.

[estianic]

You're right, there is no cookie file anymore, I'm using extension version 3.4.0

Nevertheless, I still think it has something to do with the way cookies are handled. Someone suggested that Yahoo is doing A/B testing, because for some of us (like me) the extension still works fine. Cookies are handled differently in different countries because of legal differences.

These Yahoo .html files in the extension directory, that refuse to download and are very small. What happens when you open them in a browser ? Blurred text ?

[cmallwitz]

When you get errors and see a very small yahoo*.html saved to .financials-extension/, could you send/attach it here?

[Durhammer3111]

Well, I WOULD, but when I click on the paperclip tool to attach one of the files, they simply don't show up. Ah, that's probably because they're .html files. But here's what yahoo-DECK.html looks like:

<!-- 'https://finance.yahoo.com/quote/DECK?p=DECK' -->

[cmallwitz]

ah, so there is content whatsoever...

[Durhammer3111]

Right. I have no problem viewing the DECK quote page nor downloading it via curl:

curl -k -L -s --compressed https://finance.yahoo.com/quote/DECK > myyahoo-DECK.html

[Durhammer3111]

HOWEVER, as with certain tickers I previously mentioned, trying to download the statistics* file for DECK gets me a "404" error file.

*i. e., https://finance.yahoo.com/quote/DECK/key-statistics

[Futur3H4wk]

just updated to 3.4.0 still no luck. I made a tar file of the .finan* file and dropped the . off the front. not sure if this is helpful financials-extension.tar.gz

[Durhammer3111]

FWIW, I just made a "FT" copy of my spreadsheet, added all the various exchange suffixes (sheesh -- a lot, and it just looks ugly!), and changes all the function calls from "YAHOO" to "FT" and it works. Even has decent speed! There are some odd differences, such as the market cap is only shown in M in FT (Yahoo shows 'em in B if apropos), and the daily % change is 100 times what it should be. Thanks for giving us that option, @cmallwitz ! I'll keep fiddling with it to make it so the symbols are nicer, maybe by having a hidden cell with the exchange that can be concatenated onto the ticker (won't work for CDR-PC, tho -- sheesh!).

Edit: oh yeah, the annual yield is also 100 X what it oughta be.

[cmallwitz]

I have integrated the pull request from beckjake and created release version 3.5.0 - let us know if this improves things for you.

[Durhammer3111]

YAY -- it's now working again with Yahoo!! While I now have a backup scheme to allow flipping betwixt Yahoo! and FT, Yahoo! has far better better data point availability. Thanks, @cmallwitz !

[cmallwitz]

I still have no idea how Yahoo manages to generate different HTML depending on location. Maybe you guys having problems can share details about your location and/or internet provider. and if any of you has VPN available, is activating it to somewhere else actually fixing your problems?

When releasing I try to run a few tests while on VPN to UK , EU, US just to make sure there are no obvious changes to cookie handling depending on jurisdiction but so far I have not seen your problem locally...

[techman2423]

I had a heck of a time getting it to work yesterday. Finally did. I’m in Oklahoma with OEC as my provider. Fiber connection. Haven’t had any issues network wise with anything else. No vpn.

[beckjake]

For me, the new html seemed roughly correlated with the appearance of yahoo's new (slow and buggy) UI. If I navigate to a stock ticker page in yahoo using the new UI and "view source", I can find data-value. If I switch it back to the "classic yahoo finance" view, I can't find data-value in the source anymore. I've been getting A/B tested about it for a while now but the default seems to be mostly "new" lately.

[Durhammer3111]

Actually, I now wonder if one of my issues was of shooting myself in the foot when I did the "remove/reinstall" of the extension. When I upgraded to the 3.5.0 extension, I noticed in Calc's extension manager that the listed version was 3.2.1 !!! (I had the extension in two different places and looks like I picked up an old one by mistake.)

Maybe the version could be incorporated into the file name?

Oh yeah -- I'm in NC and use Google Fiber.

[cmallwitz]

Re: adding version to extension filename - Libre Calc is rather finicky about this. Users have done this while doing a "Save As" from browser and it turns out this breaks the installation process in Extension Manager. But maybe I need to generate the version somehow into the meta data inside the extension in order to get this to work...

[Durhammer3111]

You have a tough job, my friend! And we all appreciate the work you do!