"I/O error : buffer full" appears when reloading a specific feed

[DannyJJK]

Newsboat version (copy the output of newsboat -v or the first line of git show): Newsboat 2.29.0 System: Darwin 22.1.0 (x86_64) Compiler: g++ Apple LLVM 14.0.0 (clang-1400.0.29.102) ncurses: ncurses 5.7.20081102 (compiled with 5.7) libcurl: libcurl/7.84.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.47.0 (compiled with 7.79.1) SQLite: 3.39.4 (compiled with 3.37.0) libxml2: compiled with 2.9.4

Config file (copy from ~/.newsboat/config or ~/.config/newsboat/config):

browser "open %u"
color info white black
color listfocus black yellow
color listfocus_unread black yellow
color listnormal_unread yellow default
max-items 100
reload-threads 1
text-width 80

Note: I usually have reload-threads set to 10, but I can replicate it with it set to 1 as well

Steps to reproduce the issue:

1. Add https://youtube.com/feeds/videos.xml?channel_id=UCq7K7RspznFUw8DCufH74cg to Newsboat's urls file
2. Reload feed
3. Message I/O error : buffer full appears on the line of the highlighted feed/article, at the very left

Other info you think is relevant: This issue only started happening today, and may be time sensitive in case the RSS feed changes

[dennisschagt]

It looks like the I/O error : buffer full message is written by libxml2, when we call xmlReadMemory(): https://github.com/newsboat/newsboat/blob/89df1fdf679f14fbb75d9a251e82c83cd7bf86f8/rss/parser.cpp#L284-L288

When running with gdb, buffer.length() is 32771, which looks like something which is just above some "arbitrary" internal limit of 2^15 == 32768 bytes.

No idea if that is a limit we configure somewhere from newsboat, or if this is some default limit in libxml2 (and I'm out of time for today)

[dennisschagt]

It looks like the feed got updated (due to a new video upload), and now it is no longer reproducable.

I did save the original feed: feed.txt When I used a file:// url in newsboat, the problem was not reproducable (e.g. file:///home/dennis/projects/newsboat/densch-envs/issue-2252/feed.txt) However, by making sure parsing happened with the same libxml2 function, the problem can be reproduced again (calling xmlReadMemory instead of xmlReadFile, patch: patch.txt)

No idea yet why this is happening. I did find some strange behaviour (the problem goes away as soon as I remove/add a single character. It does not go away if I change a character while keeping the file length the same).

Backtrace from the place where the error is printed:

[DannyJJK]

I'm glad you are able to reproduce the issue, is it possible that the issue is in libxml2? I'm not a C/C++ developer but I can see that there are newer versions of libxml2, I'm not sure whether it's worth trying to compile newsboat with a newer version and see if the issue still occurs.

[dennisschagt]

is it possible that the issue is in libxml2?

It might be. I opened an issue on the libxml2 repository: https://gitlab.gnome.org/GNOME/libxml2/-/issues/438

[DannyJJK]

Thank you @dennisschagt, it looks like it was a libxml2 bug as they acknowledged and fixed it on the linked issue. I'm not sure whether to close this issue, unless there's any further testing you wish to do or if you want to wait for a new libxml2 version before closing this.

As a side note, I installed Newsboat from Homebrew which appears to use the libxml2 version supplied by macOS, so it may be worth contacting the Homebrew package maintainer to use the libxml2 version supplied by Homebrew to avoid macOS users running into this issue once a new version of libxml2 is released with the fix.

[dennisschagt]

I think it is fine to close the issue here. If you run into it again before the next libxml2 update, you can press Ctrl+L to force a full redraw of the Newsboat UI.

As a side note, I installed Newsboat from Homebrew which appears to use the libxml2 version supplied by macOS

I'm not familiar with MacOS/Homebrew and the processes they have. If the MacOS version of libxml2 is old enough, it might not even include the commit which causes the issue :) (if I understand correctly, the issue was introduced ~5 months ago)