Long first initiation after system startup

[BEST8OY]

Describe the bug Long first initiation after system startup

To Reproduce Steps to reproduce the behavior: 1- Updated from previous latest release to latest 2- Restarted the system (Arch Linux) and now it takes 2 Minutes just to start GD. (it was like 3 seconds with previous release)

Expected behavior GD initiation should be some seconds not minutes

OS and software versions Goldendict-ng 24.05.05 at 2024-05-05T21:55:41Z Qt 6.7.0 GCC 13.2.1 20240417 arch linux 6.8.9-arch1-1 x86_64-little_endian-lp64 Flags: MAKE_ZIM_SUPPORT USE_ICONV MAKE_CHINESE_CONVERSION_SUPPORT

Additional context It only happens in system startup. It means 1- GD first start takes 2 minutes 2- Fully closing it from background (or with ctrl+q) 3- Starting GD again and it takes like 4-5 seconds But if I restart the system and try to start GD it takes 2 minutes to come up. (and I'm sure there's no reindexing issue)

[shenlebantongying]

A trick to debug this is pausing the program during the long startup and see what's on the top of the program's stack.

Can you try to do this, presumably multiple times, due to this method's empirical nature? gdb goldendict then enter run and pause like this and paste the results https://stackoverflow.com/a/8703132/12494649

This cannot catch problems if it is not GD's problem btw 😅

This doesn't happen on my machine. I don't know what's possibly wrong.

[BEST8OY]

A trick to debug this is pausing the program during the long startup and see what's on the top of the program's stack.

Can you try to do this, presumably multiple times, due to this method's empirical nature? gdb goldendict then enter run and pause like this and paste the results https://stackoverflow.com/a/8703132/12494649

It seems gdb needs to download a lot of libs. I can't do it now so if I got the time, I'll surely do it and report back.

[shenlebantongying]

Putting unset DEBUGINFOD_URLS to ~/.gdbinit will disable Debuginfod to stop all the downloading.

[BEST8OY]

Putting unset DEBUGINFOD_URLS to ~/.gdbinit will disable Debuginfod to stop all the downloading.

Thanks for the tip. First start 2 minute or maybe more :)

(gdb) run
Starting program: /usr/bin/goldendict
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7fffd62006c0 (LWP 1804)]
[New Thread 0x7fffd58006c0 (LWP 1807)]
[New Thread 0x7fffd4c006c0 (LWP 1831)]
[New Thread 0x7fffcda006c0 (LWP 1832)]
[New Thread 0x7fffcd0006c0 (LWP 1833)]
[New Thread 0x7fffbfc006c0 (LWP 1834)]
[New Thread 0x7fffbf2006c0 (LWP 1838)]
[New Thread 0x7fffbe8006c0 (LWP 1841)]
Path override failed for key base::DIR_APP_DICTIONARIES and path '/usr/bin/qtwebengine_dictionaries'
[New Thread 0x7fffbd0006c0 (LWP 1852)]
[Detaching after fork from child process 1853]
[Detaching after fork from child process 1854]
[Detaching after fork from child process 1855]
Path override failed for key base::DIR_APP_DICTIONARIES and path '/usr/lib/qt6/qtwebengine_dictionaries'
Path override failed for key base::DIR_APP_DICTIONARIES and path '/usr/lib/qt6/qtwebengine_dictionaries'
[New Thread 0x7fffabe006c0 (LWP 1859)]
[New Thread 0x7fffab4006c0 (LWP 1860)]
[New Thread 0x7fffaaa006c0 (LWP 1861)]
[New Thread 0x7fffaa0006c0 (LWP 1862)]
[New Thread 0x7fffa96006c0 (LWP 1863)]
[New Thread 0x7fffa8c006c0 (LWP 1864)]
[New Thread 0x7fff9fe006c0 (LWP 1865)]
[New Thread 0x7fff9f4006c0 (LWP 1866)]
[New Thread 0x7fff9ea006c0 (LWP 1867)]
[New Thread 0x7fff9e0006c0 (LWP 1868)]
[New Thread 0x7fff9d6006c0 (LWP 1869)]
[New Thread 0x7fff9cc006c0 (LWP 1870)]
[New Thread 0x7fff93e006c0 (LWP 1871)]
[New Thread 0x7fff852006c0 (LWP 1872)]
[New Thread 0x7fff7be006c0 (LWP 1873)]
[New Thread 0x7fff7b4006c0 (LWP 1874)]
[New Thread 0x7fff7aa006c0 (LWP 1875)]
[New Thread 0x7fff7a0006c0 (LWP 1876)]
[New Thread 0x7fff796006c0 (LWP 1877)]
[New Thread 0x7fff78c006c0 (LWP 1878)]
[New Thread 0x7fff73e006c0 (LWP 1879)]
[New Thread 0x7fff71c006c0 (LWP 1880)]
handle path: "/mnt/Everything/Dicts/Aryanpour Dictionary"
[New Thread 0x7fff712006c0 (LWP 1881)]
[Thread 0x7fff712006c0 (LWP 1881) exited]
[New Thread 0x7fff712006c0 (LWP 1882)]
[New Thread 0x7fff934006c0 (LWP 1883)]
[Thread 0x7fff934006c0 (LWP 1883) exited]
[Thread 0x7fff712006c0 (LWP 1882) exited]
[New Thread 0x7fff712006c0 (LWP 1884)]
[New Thread 0x7fff934006c0 (LWP 1885)]
[Thread 0x7fff934006c0 (LWP 1885) exited]
[Thread 0x7fff712006c0 (LWP 1884) exited]
[New Thread 0x7fff712006c0 (LWP 1886)]
[New Thread 0x7fff934006c0 (LWP 1887)]
[New Thread 0x7fff87e006c0 (LWP 1888)]
[New Thread 0x7fff874006c0 (LWP 1889)]
[New Thread 0x7fff86a006c0 (LWP 1890)]
handle path: "/mnt/Everything/Dicts/Cambridge"
handle path: "/mnt/Everything/Dicts/MWDONT"
handle path: "/mnt/Everything/Dicts/Mosby's Dictionary of Medicine, Nursing & Health Professions,10th Edition"
handle path: "/mnt/Everything/Dicts/Stedman's Medical Dictionary"
handle path: "/mnt/Everything/Dicts/Dorland"
handle path: "/mnt/Everything/Dicts/LongMan"
handle path: "/home/best8oy/Everything/Dicts/Longman Pronunciation Dictionary"
[Thread 0x7fffbe8006c0 (LWP 1841) exited]
Load done

[Thread 0x7fff71c006c0 (LWP 1880) exited]
[New Thread 0x7fff71c006c0 (LWP 4206)]
[New Thread 0x7fffbe8006c0 (LWP 4207)]
starting create the fts with thread: 3
[New Thread 0x7fff860006c0 (LWP 4208)]
[New Thread 0x7fff734006c0 (LWP 4209)]
Reloading all the tabs...
[New Thread 0x7fff72a006c0 (LWP 4210)]
[FULLTEXT] checking fts for the dictionary: "Aryanpour Dictionary"
[FULLTEXT] checking fts for the dictionary: "Cambridge Dictionary English 2023.12"
[FULLTEXT] checking fts for the dictionary: "Merriam-Webster.com Dictionary"
[FULLTEXT] checking fts for the dictionary: "Mosby's Dictionary of Medicine, Nursing & Health Professions,10th Edition"
[FULLTEXT] checking fts for the dictionary: "Stedman's Medical Dictionary"
[FULLTEXT] checking fts for the dictionary: "Dorland's Illustrated Medical Dictionary - 32nd Edition"
[FULLTEXT] checking fts for the dictionary: "LDoCE online 2023.08"
waiting for all the fts creation to finish.
[FULLTEXT] checking fts for the dictionary: "Longman Pronunciation Dictionary"
finished/cancel all the fts creation
[Thread 0x7fffbe8006c0 (LWP 4207) exited]
[New Thread 0x7fffbe8006c0 (LWP 4211)]
[New Thread 0x7fff67e006c0 (LWP 4212)]
[New Thread 0x7fff674006c0 (LWP 4214)]
[New Thread 0x7fff66a006c0 (LWP 4215)]
[Thread 0x7fff66a006c0 (LWP 4215) exited]
[Thread 0x7fff674006c0 (LWP 4214) exited]
[New Thread 0x7fff674006c0 (LWP 4216)]
[New Thread 0x7fff66a006c0 (LWP 4217)]
Reloading all the tabs...
[New Thread 0x7fff660006c0 (LWP 4229)]
reset inspector
[New Thread 0x7fff656006c0 (LWP 4230)]
[New Thread 0x7fff64c006c0 (LWP 4231)]
[New Thread 0x7fff5be006c0 (LWP 4236)]
[New Thread 0x7fff5b4006c0 (LWP 4237)]
[New Thread 0x7fff5aa006c0 (LWP 4238)]
getResource: "gdlookup://localhost/?word=Welcome!&group=4294967295"
scheme: "gdlookup"
host: "localhost"
In-place finish.

====reading  4316 of (4316) bytes . Finished: 1
article view loaded url: "data:text/html;charset=UTF-8,%3C%21DOCTYPE html%3E%0A%3Chtml%3E%3Chead%3E%0A%3Cmeta charset%3D%22utf-8%22%3E%0A%3Cscript src%3D%22qrc%3A%2F%2F%2Fscripts%2Fjquery-3.6.0.slim.min.js%22%3E%3C%2Fscript%3E" true
article view loaded url: "gdlookup://localhost/?word=Welcome!&group=4294967295" true

Second start 3 seconds

(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/bin/goldendict
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7fffd62006c0 (LWP 6360)]
[New Thread 0x7fffd58006c0 (LWP 6361)]
[New Thread 0x7fffd4c006c0 (LWP 6377)]
[New Thread 0x7fffcda006c0 (LWP 6378)]
[New Thread 0x7fffcd0006c0 (LWP 6379)]
[New Thread 0x7fffbfc006c0 (LWP 6380)]
[New Thread 0x7fffbf2006c0 (LWP 6381)]
[New Thread 0x7fffbe8006c0 (LWP 6382)]
Path override failed for key base::DIR_APP_DICTIONARIES and path '/usr/bin/qtwebengine_dictionaries'
[New Thread 0x7fffbd0006c0 (LWP 6383)]
[Detaching after fork from child process 6384]
[Detaching after fork from child process 6385]
[Detaching after fork from child process 6386]
Path override failed for key base::DIR_APP_DICTIONARIES and path '/usr/lib/qt6/qtwebengine_dictionaries'
Path override failed for key base::DIR_APP_DICTIONARIES and path '/usr/lib/qt6/qtwebengine_dictionaries'
[New Thread 0x7fffabe006c0 (LWP 6389)]
[New Thread 0x7fffab4006c0 (LWP 6390)]
[New Thread 0x7fffaaa006c0 (LWP 6391)]
[New Thread 0x7fffaa0006c0 (LWP 6392)]
[New Thread 0x7fffa96006c0 (LWP 6393)]
[New Thread 0x7fffa8c006c0 (LWP 6394)]
[New Thread 0x7fff9fe006c0 (LWP 6395)]
[New Thread 0x7fff9f4006c0 (LWP 6396)]
[New Thread 0x7fff9ea006c0 (LWP 6397)]
[New Thread 0x7fff9e0006c0 (LWP 6398)]
[New Thread 0x7fff9d6006c0 (LWP 6399)]
[New Thread 0x7fff9cc006c0 (LWP 6400)]
[New Thread 0x7fff93e006c0 (LWP 6401)]
[New Thread 0x7fff934006c0 (LWP 6402)]
[New Thread 0x7fff912006c0 (LWP 6403)]
[New Thread 0x7fff85e006c0 (LWP 6404)]
[New Thread 0x7fff854006c0 (LWP 6405)]
[New Thread 0x7fff84a006c0 (LWP 6406)]
[New Thread 0x7fff792006c0 (LWP 6407)]
[New Thread 0x7fff73e006c0 (LWP 6408)]
[New Thread 0x7fff734006c0 (LWP 6409)]
[New Thread 0x7fff72a006c0 (LWP 6410)]
[New Thread 0x7fff712006c0 (LWP 6411)]
handle path: "/mnt/Everything/Dicts/Aryanpour Dictionary"
handle path: "/mnt/Everything/Dicts/Cambridge"
handle path: "/mnt/Everything/Dicts/MWDONT"
[New Thread 0x7fff674006c0 (LWP 6412)]
handle path: "/mnt/Everything/Dicts/Mosby's Dictionary of Medicine, Nursing & Health Professions,10th Edition"
[Thread 0x7fff674006c0 (LWP 6412) exited]
handle path: "/mnt/Everything/Dicts/Stedman's Medical Dictionary"
handle path: "/mnt/Everything/Dicts/Dorland"
handle path: "/mnt/Everything/Dicts/LongMan"
handle path: "/home/best8oy/Everything/Dicts/Longman Pronunciation Dictionary"
[New Thread 0x7fff674006c0 (LWP 6413)]
[New Thread 0x7fff92a006c0 (LWP 6414)]
[Thread 0x7fff92a006c0 (LWP 6414) exited]
[Thread 0x7fff674006c0 (LWP 6413) exited]
[New Thread 0x7fff674006c0 (LWP 6415)]
[New Thread 0x7fff92a006c0 (LWP 6416)]
[Thread 0x7fff92a006c0 (LWP 6416) exited]
[Thread 0x7fff674006c0 (LWP 6415) exited]
[New Thread 0x7fff674006c0 (LWP 6417)]
[New Thread 0x7fff92a006c0 (LWP 6418)]
[New Thread 0x7fff920006c0 (LWP 6419)]
[New Thread 0x7fff87e006c0 (LWP 6420)]
[New Thread 0x7fff874006c0 (LWP 6421)]
Load done

[Thread 0x7fff712006c0 (LWP 6411) exited]
[New Thread 0x7fff712006c0 (LWP 6441)]
Reloading all the tabs...
starting create the fts with thread: 3
[New Thread 0x7fff86a006c0 (LWP 6442)]
[New Thread 0x7fff79c006c0 (LWP 6443)]
[New Thread 0x7fff720006c0 (LWP 6444)]
[FULLTEXT] checking fts for the dictionary: "Aryanpour Dictionary"
[FULLTEXT] checking fts for the dictionary: "Cambridge Dictionary English 2023.12"
[FULLTEXT] checking fts for the dictionary: "Merriam-Webster.com Dictionary"
[FULLTEXT] checking fts for the dictionary: "Mosby's Dictionary of Medicine, Nursing & Health Professions,10th Edition"
[FULLTEXT] checking fts for the dictionary: "Stedman's Medical Dictionary"
[FULLTEXT] checking fts for the dictionary: "Dorland's Illustrated Medical Dictionary - 32nd Edition"
[FULLTEXT] checking fts for the dictionary: "LDoCE online 2023.08"
waiting for all the fts creation to finish.
[FULLTEXT] checking fts for the dictionary: "Longman Pronunciation Dictionary"
finished/cancel all the fts creation
[Thread 0x7fff712006c0 (LWP 6441) exited]
[New Thread 0x7fff712006c0 (LWP 6445)]
[New Thread 0x7fff67e006c0 (LWP 6446)]
[New Thread 0x7fff66a006c0 (LWP 6449)]
[New Thread 0x7fff660006c0 (LWP 6450)]
[Thread 0x7fff660006c0 (LWP 6450) exited]
[Thread 0x7fff66a006c0 (LWP 6449) exited]
[New Thread 0x7fff66a006c0 (LWP 6455)]
[New Thread 0x7fff660006c0 (LWP 6457)]
Reloading all the tabs...
[New Thread 0x7fff656006c0 (LWP 6478)]
reset inspector
[New Thread 0x7fff64c006c0 (LWP 6481)]
[New Thread 0x7fff5be006c0 (LWP 6483)]
[New Thread 0x7fff5b4006c0 (LWP 6484)]
[New Thread 0x7fff5aa006c0 (LWP 6485)]
[New Thread 0x7fff5a0006c0 (LWP 6486)]
getResource: "gdlookup://localhost/?word=Welcome!&group=4294967295"
scheme: "gdlookup"
host: "localhost"
In-place finish.

====reading  4316 of (4316) bytes . Finished: 1
article view loaded url: "data:text/html;charset=UTF-8,%3C%21DOCTYPE html%3E%0A%3Chtml%3E%3Chead%3E%0A%3Cmeta charset%3D%22utf-8%22%3E%0A%3Cscript src%3D%22qrc%3A%2F%2F%2Fscripts%2Fjquery-3.6.0.slim.min.js%22%3E%3C%2Fscript%3E" true
article view loaded url: "gdlookup://localhost/?word=Welcome!&group=4294967295" true

[shenlebantongying]

article view loaded url: "gdlookup://localhost/?word=Welcome!&group=4294967295" true

Strange, if you pause during the 2 mins startup, does this mean GD stuck at loading the "welcome" page for 2 mins :sweat_smile:

[BEST8OY]

Strange, if you pause during the 2 mins startup, does this mean GD stuck at loading the "welcome" page for 2 mins 😅

Yes, it takes 2 minutes to bring GD main screen up. If I interrupt it, I have this screen forever! By startup I mean this loading screen!

[BEST8OY]

I wanted to do a cleanbuild so I did a pacman -Rns goldendict-ng now I can't rebuild GD from aur because libeb also was removed and now I can't rebuild libeb (gives error) It seems to be related to gcc recent update (maybe)

puts_eucjp.c: In function ‘fputs_eucjp_to_locale’:
puts_eucjp.c:97:23: error: passing argument 2 of ‘iconv’ from incompatible pointer type [-Wincompatible-pointer-types]
   97 |         if (iconv(cd, &in_p, &in_left, &out_p, &out_left) != -1)
      |                       ^~~~~
      |                       |
      |                       const char **
In file included from puts_eucjp.c:40:
/usr/include/iconv.h:49:54: note: expected ‘char ** restrict’ but argument is of type ‘const char **’
   49 | extern size_t iconv (iconv_t __cd, char **__restrict __inbuf,
      |                                    ~~~~~~~~~~~~~~~~~~^~~~~~~
make[3]: *** [Makefile:418: puts_eucjp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
mv -f .deps/getumask.Tpo .deps/getumask.Po
mv -f .deps/makedir.Tpo .deps/makedir.Po
mv -f .deps/samefile.Tpo .deps/samefile.Po
mv -f .deps/yesno.Tpo .deps/yesno.Po
mv -f .deps/strlist.Tpo .deps/strlist.Po
mv -f .deps/ebutils.Tpo .deps/ebutils.Po
mv -f .deps/getopt.Tpo .deps/getopt.Po
make[3]: Leaving directory '/home/best8oy/.cache/yay/libeb/src/eb-4.4.3/libebutils'
make[2]: *** [Makefile:353: all] Error 2
make[2]: Leaving directory '/home/best8oy/.cache/yay/libeb/src/eb-4.4.3/libebutils'
make[1]: *** [Makefile:518: all-recursive] Error 1
make[1]: Leaving directory '/home/best8oy/.cache/yay/libeb/src/eb-4.4.3'
make: *** [Makefile:404: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...
 -> error making: libeb-exit status 4
 -> Failed to install the following packages. Manual intervention is required:
libeb - exit status 4

[BEST8OY]

New info that might help In latest version as you can see the 'Everything' disk is in use during those 2 minutes I mentioned. then I fully close GD and start it again and as you can see the 'Everything' disk is not in use anymore! (during its 4 seconds loading)

[shenlebantongying]

TBH, I don't think this is GD's problem.

I don't know why. There could be many reasons for slow IO.

There are many things to poke around :sweat_smile:

like check dmesg any errors associated with a disk, fsck the disk, S.M.A.R.T. tests and logs, any bad sectors? (is the disk gonna to die soon)?, SSD trim is working? HDD defragged? Some filesysmte specific things, like btrfs has many maintenance commands.

Not an expert, but those are things that I encountered before.

I fully close GD and start it again and as you can see the 'Everything' disk is not in use anymore! (during its 4 seconds loading)

I think this is because of file caching. If a program launches immediately after the first time, the related staff are likely still in caches. The second launch simply accesses them from the memory instead of the disk, thus zero IO involved. You can confirm this by clearing the cache and launching again.

https://unix.stackexchange.com/questions/87908/how-do-you-empty-the-buffers-and-cache-on-a-linux-system https://unix.stackexchange.com/questions/17936/setting-proc-sys-vm-drop-caches-to-clear-cache

[BEST8OY]

TBH, I don't think this is GD's problem.

Okay, I'm a 100% sure it's a GD issue. I just built previous latest release, rebooted, ran GD and it didn't even take 2 seconds! Goldendict-ng 24.01.22 at 2024-05-10T00:33:59 Qt 6.7.0 GCC 14.1.1 20240507 arch linux 6.8.9-arch1-2 x86_64-little_endian-lp64 Flags:USE_XAPIAN MAKE_ZIM_SUPPORT USE_ICONV MAKE_CHINESE_CONVERSION_SUPPORT

Should I build all alpha releases to see in which release this problem actually come to be!? Or could you make a guess and tell me which alpha release I should start building from?

[shenlebantongying]

Maybe you can try bisecting (starts from the middle, then try the middle of the bad half). Much efficient than one by one.

https://git-scm.com/docs/git-bisect

There aren't many changes in between recent 2 releases. I don't think one of them may cause the problem. Sorry.

[BEST8OY]

Maybe you can try bisecting (starts from the middle, then try the middle of the bad half). Much efficient than one by one.

https://git-scm.com/docs/git-bisect

It happens between these two releases https://github.com/xiaoyifang/goldendict-ng/compare/v24.02.16-alpha.6e007329...v24.02.16-alpha.4f905c75

No problem with v24.02.16-alpha.6e007329 It occurs in v24.02.16-alpha.4f905c75 Any guess? Sorry I didn't do bisect thing. I really don't know how to work with git in that manner. So, I have to learn it first but for now this is all I can do! If you can't make any guess at this point, I guess I have to do the bisect thing!

[shenlebantongying]

https://github.com/xiaoyifang/goldendict-ng/compare/v24.02.16-alpha.6e007329...v24.02.16-alpha.4f905c75

In this range, only this appears to be able to increase startup time. https://github.com/xiaoyifang/goldendict-ng/commit/e8f3a94541068f2464bc5eaaa38d95858c6b3148

Do you have a big sound dictionary?

[BEST8OY]

Do you have a big sound dictionary?

Yes, a huge forvo collection of EN voices!

Removing the SoundDir fixed the first loading problem!

[shenlebantongying]

Is there something special about this folder? What's your file system?

I created this 20GB, 1000k entries nested sound folder, and GD starts instantly :sweat_smile:

https://github.com/xiaoyifang/goldendict-ng/assets/20123683/0927d710-bb64-435f-8468-c468348f25d2

[BEST8OY]

Is there something special about this folder? What's your file system?

I believe the number of folders is the cause (?) It's a NTFS partition (HDD)

[shenlebantongying]

Most certainly Yes :sweat_smile:

Tried 100K folder sounddir on an NTFS partition on NVMe disk, the startup time goes up significantly.

However, if this folder on a Btrfs partition on SSD, the startup time is still a few seconds.

NTFS on Linux is probably slower.

Anyway, I will add a timeout to that part.

import os
for i in range(100000):
      os.makedirs(str(i))

[shenlebantongying]

Please try the PR. If it works, then the problem is indeed what we just discovered :sweat_smile:

[BEST8OY]

Please try the PR. If it works, then the problem is indeed what we just discovered 😅

With SoundDir takes like 6 seconds! (doesn't matter if it's the first startup or second startup it always takes 6 seconds (for my SoundDir)) Sweet spot with so many folders, I guess! 😅

Without SoundDir takes 1 sec or less!

[shenlebantongying]

You can move all files to one folder to reduce that time to near 0. Or move it to a none-NTFS partition.

That's all today I can do.

> cd soundDir
> mkdir newFolder
> fd . --extension mp3 --type file --exec mv {} ./newFolder/

[BEST8OY]

You can move all files to one folder to reduce that time to near 0. Or move it to a none-NTFS partition.

The problem is that there're multiple sound for a phrase or a word in Forvo, so they have to be structured by words or voices (Forvo users). Logically, structured by users seems to be more reasonable. As you said those two seems to be the only solutions for the delayed (not much now with latest PR😅) loading!