search

Sude- / lgogdownloader

LGOGDownloader is unofficial downloader to GOG.com for Linux users. It uses the same API as the official GOG Galaxy.
https://sites.google.com/site/gogdownloader/
Do What The F*ck You Want To Public License
721 stars 67 forks source link

hang during --status #44

Open dmhursh opened 9 years ago

dmhursh commented 9 years ago

I get a hang during the following.

./bin/Release/lgogdownloader --no-deb --directory ~/gog/ --platform 4 --language 1 --game wasteland --status

The output prior to the hang is (long numbers removed. Let me know if they matter)

user@host:lgogdownloader$ ./bin/Release/lgogdownloader --no-deb --directory ~/gog/ --platform 4 --language 1 --game wasteland --status WARNING: gnome-keyring:: couldn't connect to: /home/user/.cache/keyring-mNLVXi/pkcs11: No such file or directory Getting game info 1 / 1 OK wasteland_2_kickstarter gog_wasteland_2_1.7.0.11.tar.gz [code] [code] OK wasteland_2_kickstarter wasteland_2_manual.zip [code] [code] OK wasteland_2_kickstarter wasteland_2_reference_card.zip [code] [code] OK wasteland_2_kickstarter wasteland_2_map.zip 94517311 [code] [code] OK wasteland_2_kickstarter wl2_choir_songs_flac.zip [code] [code] OK wasteland_2_kickstarter wl2_choir_songs_mp3.zip [code] [code] ^C

At the time of the hang strace show only line repeating

read(7, 0x17ade80, 8192) = -1 EISDIR (Is a directory) read(7, 0x17ade80, 8192) = -1 EISDIR (Is a directory) read(7, 0x17ade80, 8192) = -1 EISDIR (Is a directory) . . .

/proc shows the the following open file handes

user@host:~$ ls -l /proc/4015/fd total 0 lrwx------ 1 user user 64 Dec 26 12:51 0 -> /dev/pts/1 lrwx------ 1 user user 64 Dec 26 12:51 1 -> /dev/pts/1 lrwx------ 1 user user 64 Dec 26 12:51 2 -> /dev/pts/1 lr-x------ 1 user user 64 Dec 26 12:51 3 -> /dev/urandom lrwx------ 1 user user 64 Dec 26 12:51 4 -> socket:[1906477] lrwx------ 1 user user 64 Dec 26 12:51 5 -> socket:[1906482] lrwx------ 1 user user 64 Dec 26 12:51 6 -> socket:[1906967] lr-x------ 1 user user 64 Dec 26 12:51 7 -> /home/user/gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_deluxe_edition_upgrade/extras

The contents of the wasteland_2_kickstarter directory are

user@host:~$ find gog/wasteland_2_kickstarter/ gog/wasteland_2_kickstarter/ gog/wasteland_2_kickstarter/dlc gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_classic_edition_upgrade gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_classic_edition_upgrade/extras gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_classic_edition_upgrade/extras/wasteland_2_soundtrack_mp3.zip gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_classic_edition_upgrade/extras/wasteland_2_wallpapers.zip gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_classic_edition_upgrade/extras/wl2_choir_songs_flac.zip gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_classic_edition_upgrade/extras/wl2_choir_songs_mp3.zip gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_classic_edition_upgrade/extras/wasteland_2_soundtrack_flac.zip gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_classic_edition_upgrade/extras/wasteland_2_artbook.zip gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_deluxe_edition_upgrade gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_deluxe_edition_upgrade/extras gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_deluxe_edition_upgrade/extras/wasteland_2_book_two.zip gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_deluxe_edition_upgrade/extras/wasteland_2_book_one.zip gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_deluxe_edition_upgrade/extras/wasteland_2_book_three.zip gog/wasteland_2_kickstarter/gog_wasteland_2_1.5.0.9.tar.gz gog/wasteland_2_kickstarter/extras gog/wasteland_2_kickstarter/extras/wasteland_2_reference_card.zip gog/wasteland_2_kickstarter/extras/wl2_choir_songs_flac.zip gog/wasteland_2_kickstarter/extras/wasteland_2_map.zip gog/wasteland_2_kickstarter/extras/wl2_choir_songs_mp3.zip gog/wasteland_2_kickstarter/extras/wasteland_2_manual.zip gog/wasteland_2_kickstarter/gog_wasteland_2_1.7.0.11.tar.gz

dmhursh commented 9 years ago

I did a debug build and got a stack trace

(gdb) bt

0 0x00007f4ceb5e57d0 in read () from /lib/x86_64-linux-gnu/libc.so.6

1 0x00007f4ceb58762f in ?? () from /lib/x86_64-linux-gnu/libc.so.6

2 0x00007f4ceb57d852 in fread () from /lib/x86_64-linux-gnu/libc.so.6

3 0x00007f4cec043cf2 in rhash_file_update () from /usr/lib/librhash.so.0

4 0x00007f4cec043da9 in rhash_file () from /usr/lib/librhash.so.0

5 0x000000000046948b in Util::getFileHash (filename=..., hash_id=4) at src/util.cpp:65

6 0x000000000044df50 in Downloader::getLocalFileHash (this=0x7fffc2ac3f60, filepath=..., gamename=...)

at src/downloader.cpp:2650

7 0x000000000044d5da in Downloader::checkStatus (this=0x7fffc2ac3f60) at src/downloader.cpp:2604

8 0x000000000041b2cc in main (argc=11, argv=0x7fffc2ac4c38) at main.cpp:521

Going to frame 5 I printed filename

(gdb) print filename $1 = (const std::string &) @0x7fffc2ac3360: {static npos = , _M_dataplus = {std::allocator = {<__gnu_cxx::new_allocator> = {}, }, _M_p = 0x15f4fe8 "/home/user/gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_deluxe_edition_upgrade/extras/"}}

Looks like it's trying to hash a directory. It will take some time before I will have time to dig into any code.

Sude- commented 9 years ago

7d5874d should prevent downloader from trying to hash a directory

dmhursh commented 9 years ago

That does prevent the hang, but not it just crashes. Maybe that was intentional for now.

user@host:lgogdownloader$ ./bin/Release/lgogdownloader --no-deb --directory ~/gog/ --platform 4 --language 1 --game wasteland --status WARNING: gnome-keyring:: couldn't connect to: /home/user/.cache/keyring-mNLVXi/pkcs11: No such file or directory Getting game info 1 / 1 OK wasteland_2_kickstarter gog_wasteland_2_1.7.0.11.tar.gz [code] [code] OK wasteland_2_kickstarter wasteland_2_manual.zip [code] [code] OK wasteland_2_kickstarter wasteland_2_reference_card.zip [code] [code] OK wasteland_2_kickstarter wasteland_2_map.zip [code] [code] OK wasteland_2_kickstarter wl2_choir_songs_flac.zip [code] [code] OK wasteland_2_kickstarter wl2_choir_songs_mp3.zip [code] [code] terminate called after throwing an instance of 'boost::filesystem3::filesystem_error' what(): boost::filesystem::file_size: Operation not permitted: "/home/user/gog/wasteland_2_kickstarter/dlc/wasteland_2_digital_deluxe_edition_upgrade/extras/" Aborted

dmhursh commented 9 years ago

I got a stack from the debug build. Let me know if there is more info I can grab. Also, I'll try to watch this closer. I thought github was going to email when there was an update.

(gdb) bt

0 0x00007ffff65e2165 in raise () from /lib/x86_64-linux-gnu/libc.so.6

1 0x00007ffff65e53e0 in abort () from /lib/x86_64-linux-gnu/libc.so.6

2 0x00007ffff6e3989d in __gnu_cxx::__verbose_terminate_handler() ()

from /usr/lib/x86_64-linux-gnu/libstdc++.so.6

3 0x00007ffff6e37996 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6

4 0x00007ffff6e379c3 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6

5 0x00007ffff6e37bee in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6

6 0x00007ffff7fca74b in boost::filesystem3::detail::file_size(boost::filesystem3::path const&, boost::system::error_code*) () from /usr/lib/libboost_filesystem.so.1.49.0

7 0x00000000004511ce in boost::filesystem3::file_size (p=...)

at /usr/include/boost/filesystem/v3/operations.hpp:447

8 0x000000000044d600 in Downloader::checkStatus (this=0x7fffffffd560) at src/downloader.cpp:2609

9 0x000000000041b2cc in main (argc=11, argv=0x7fffffffe238) at main.cpp:521

Sude- commented 9 years ago

Hopefully ae91c28 fixes the crash

dmhursh commented 9 years ago

That fixes the crash. We can call it "bug fixed". There was one interesting line in the output of --status

ND wasteland_2_kickstarter .

Let me know if you'd like to look into that more. Or does this look like some sort of user error?

Sude- commented 9 years ago

The underlying cause is that the downloader gets wrong filepath I'm guessing it is caused by the path assignment in Downloader::getExtras

I'd be interested in the output of

lgogdownloader --list-details --game wasteland_2

with this patch https://sites.google.com/site/gogdownloader/path.diff

dmhursh commented 9 years ago

Sorry. This is large. I chopped some really long codes out of the URLs, but they are still long. Let me know if you need them.

user@host:lgogdownloader$ ./bin/Release/lgogdownloader --directory ~/gog --list-details --game wasteland_2 WARNING: gnome-keyring:: couldn't connect to: /home/user/.cache/keyring-mNLVXi/pkcs11: No such file or directory URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Base/wasteland_2_manual.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_kickstarter/extras/Base/wasteland_2_manual.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Base/wasteland_2_reference_card.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_kickstarter/extras/Base/wasteland_2_reference_card.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Base/wasteland_2_map.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_kickstarter/extras/Base/wasteland_2_map.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/wl2_choir_songs_flac.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_kickstarter/extras/wl2_choir_songs_flac.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/wl2_choir_songs_mp3.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_kickstarter/extras/wl2_choir_songs_mp3.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_deluxe_edition_upgrade/extras/ URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Deluxe/wasteland_2_book_one.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_deluxe_edition_upgrade/extras/Deluxe/wasteland_2_book_one.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Deluxe/wasteland_2_book_two.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_deluxe_edition_upgrade/extras/Deluxe/wasteland_2_book_two.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Deluxe/wasteland_2_book_three.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_deluxe_edition_upgrade/extras/Deluxe/wasteland_2_book_three.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Classic/wasteland_2_soundtrack_mp3.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_classic_edition_upgrade/extras/Classic/wasteland_2_soundtrack_mp3.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Classic/wasteland_2_soundtrack_flac.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_classic_edition_upgrade/extras/Classic/wasteland_2_soundtrack_flac.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/Classic/wasteland_2_artbook.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_classic_edition_upgrade/extras/Classic/wasteland_2_artbook.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/wasteland_2_wallpapers.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_classic_edition_upgrade/extras/wasteland_2_wallpapers.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/wl2_choir_songs_flac.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_classic_edition_upgrade/extras/wl2_choir_songs_flac.zip URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/wasteland_2_digital_classic_edition/extras/wl2_choir_songs_mp3.zip?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_classic_edition_upgrade/extras/wl2_choir_songs_mp3.zip

gamename: wasteland_2_kickstarter title: Wasteland 2 icon: http://static.gog.com/upload/images/2014/09/474f607921ee82b65aa2b95f43c08efeef7a5dfc.jpg installers: id: en3installer4 name: Linux Installer, English path: /wasteland_2_digital_classic_edition/linux/gog_wasteland_2_1.7.0.11.tar.gz size: 10500 updated: True language: English

extras: id: 35383 name: wasteland_2_manual.zip path: /wasteland_2_kickstarter/extras/Base/wasteland_2_manual.zip size:

id: 35423 name: wasteland_2_reference_card.zip path: /wasteland_2_kickstarter/extras/Base/wasteland_2_reference_card.zip size:

id: 35433 name: wasteland_2_map.zip path: /wasteland_2_kickstarter/extras/Base/wasteland_2_map.zip size:

id: 37673 name: wl2_choir_songs_flac.zip path: /wasteland_2_kickstarter/extras/wl2_choir_songs_flac.zip size:

id: 37683 name: wl2_choir_songs_mp3.zip path: /wasteland_2_kickstarter/extras/wl2_choir_songs_mp3.zip size:

DLCs: gamename: wasteland_2_digital_deluxe_edition_upgrade id: 34743 name: path: /wasteland_2_digital_deluxe_edition_upgrade/extras/ size:

gamename: wasteland_2_digital_deluxe_edition_upgrade id: 34753 name: wasteland_2_book_one.zip path: /wasteland_2_digital_deluxe_edition_upgrade/extras/Deluxe/wasteland_2_book_one.zip size:

gamename: wasteland_2_digital_deluxe_edition_upgrade id: 34763 name: wasteland_2_book_two.zip path: /wasteland_2_digital_deluxe_edition_upgrade/extras/Deluxe/wasteland_2_book_two.zip size:

gamename: wasteland_2_digital_deluxe_edition_upgrade id: 34773 name: wasteland_2_book_three.zip path: /wasteland_2_digital_deluxe_edition_upgrade/extras/Deluxe/wasteland_2_book_three.zip size:

gamename: wasteland_2_digital_classic_edition_upgrade id: 35343 name: wasteland_2_soundtrack_mp3.zip path: /wasteland_2_digital_classic_edition_upgrade/extras/Classic/wasteland_2_soundtrack_mp3.zip size:

gamename: wasteland_2_digital_classic_edition_upgrade id: 35353 name: wasteland_2_soundtrack_flac.zip path: /wasteland_2_digital_classic_edition_upgrade/extras/Classic/wasteland_2_soundtrack_flac.zip size:

gamename: wasteland_2_digital_classic_edition_upgrade id: 35363 name: wasteland_2_artbook.zip path: /wasteland_2_digital_classic_edition_upgrade/extras/Classic/wasteland_2_artbook.zip size:

gamename: wasteland_2_digital_classic_edition_upgrade id: 35373 name: wasteland_2_wallpapers.zip path: /wasteland_2_digital_classic_edition_upgrade/extras/wasteland_2_wallpapers.zip size:

gamename: wasteland_2_digital_classic_edition_upgrade id: 37703 name: wl2_choir_songs_flac.zip path: /wasteland_2_digital_classic_edition_upgrade/extras/wl2_choir_songs_flac.zip size:

gamename: wasteland_2_digital_classic_edition_upgrade id: 37713 name: wl2_choir_songs_mp3.zip path: /wasteland_2_digital_classic_edition_upgrade/extras/wl2_choir_songs_mp3.zip size:

Sude- commented 9 years ago

Looks like there's a link without a filename

URL: http://wpc.11DF.edgecastcdn.net/0011DF/secure/?[code]&fileExtForIe=.exe PATH: /wasteland_2_digital_deluxe_edition_upgrade/extras/

Not really sure how to properly handle this but as a workaround in 525985d I made the downloader ignore extras that don't have a filename

dmhursh commented 9 years ago

It still appears to work. I also still see the following line in the output, but I don't really have a problem with it.

ND wasteland_2_kickstarter .

I suppose it could be an issue for someone trying to automate syncing, but it sounds like a 'garbage in/garbage out' problem. In any case, let me know if there is more you'd like me to try. Thanks.