Yast Network struggles with large hosts files

[gantsevdenis]

First of all: is this is correct place where to report bugs? Might look a little quiet for such a big software.

Problem: my hosts file contains around 90000 blocked domain names, and when I open Yest Network, it hangs at 'Reading configuration...", and keeps running with 100% CPU usage.

This totally prevents its usage. I had to create another hosts file with only 20 lines, and then I could open Yast Network

[kobliha]

@gantsevdenis We encourage people for reporting at out Bugzilla, but we understand that you need to create account there. On the other hand, there you are easily lead to select the product, component, and it's easier to track (at least for the team). Plus we usually ask for YaST logs to be attached there. See https://en.opensuse.org/openSUSE:Report_a_YaST_bug

We definitely had some performance problems at the past: https://github.com/yast/yast-network/pull/711 and we tried to mitigate the situation. On the other hand, YaST does not seem to be the right tool for editing /etc/hosts, but IIUC, you are saying that just starting YaST Network blocks you for ages?

@teclator @mchf Is this a known thing?

[gantsevdenis]

@kobliha Exactly, I am not using Yast Network to edit hosts, I just can't use Yast Network at all, if I happen to have a large hosts file. I actually never succeeded to launch Yast Network before, so I had to ask on forums.opensuse, and I pinned down the problem to my hosts

[mchf]

Reading /etc/hosts still suffers from using the old code (converted from ycp). It has already bean improved a bit, so yast is at least able to survive big hosts file without crashing.

Next step was rewriting it to use augeas. I have a patch proposal under test. So, I'll add huge /etc/hosts test too to see whether it behaves better than before.

[gantsevdenis]

Hm I obviously don't know the codebase behind Yast, but when I attached gdb to the running instance, I could see that it hangs (well, runs forever) within augeas already I can provide the full stacktrace if needed

[kobliha]

Thinking about it, if nothing helps, we could try to disable reading /etc/hosts if we find out that it's simply too large for being read in reasonable time. This would be a workaround, but it might at least make the rest of YaST users usable.

[jreidinger]

@mchf it already uses augeas and it does not help, as simple parsing so huge file takes reasonable amount of time. I already improved it in past to be reasonable fast with 10k entries, but already with 30k it is slow and getting worse.

[mchf]

@jreidinger: I did too much compression in my expression. What I wanted to say was that we use still the same approach when reading hosts even after rewriting to augeas in context of module startup and data processing. I'm experimenting with kind of lazy load to address similar issues - not to load it on module start, but at time when user needs it. It clearly will not help with speeding up loading but at least can do the module startable. Problem is of course in detection of "when user needs it" due to some connections.