search

Kalanyr / gogrepoc

Python-based tool for downloading all your GOG.com game and bonus collections to your local computer for full offline enjoyment.
255 stars 36 forks source link

xml parsing error #53

Closed Tokariew closed 2 years ago

Tokariew commented 2 years ago

Tried to update today my offline games… to meet with following errors

17:52:04 | xml parsing error occurred trying to get md5 data for patch_age_of_wonders_iii_1.802_(24203)_to_1.802_fix_(28713).exe
17:52:05 | xml parsing error occurred trying to get md5 data for patch_age_of_wonders_iii_1.801_b26272_(20409)_to_1.802_(24203).exe
17:52:06 | xml parsing error occurred trying to get md5 data for patch_age_of_wonders_iii_1.801_(19572)_to_1.801_b26272_(20409).exe
17:52:08 | (  6 / 263) fetching game details for akalabeth_world_of_doom...
17:52:10 | xml parsing error occurred trying to get md5 data for setup_akalabeth_2.0.0.1.exe
17:52:11 | xml parsing error occurred trying to get md5 data for akalabeth__world_of_doom_enUS_1_0_30902.pkg
17:52:12 | xml parsing error occurred trying to get md5 data for gog_akalabeth_world_of_doom_2.0.0.3.sh
17:52:13 | (  7 / 263) fetching game details for alders_blood_prologue...
17:52:28 | xml parsing error occurred trying to get md5 data for setup_alders_blood_prologue_1.0.20a_(64bit)_(37554).exe
17:52:28 | (  8 / 263) fetching game details for aliens_versus_predator_classic_2000...
17:52:57 | xml parsing error occurred trying to get md5 data for setup_avp_classic_20160330_(2149).exe
17:52:58 | (  9 / 263) fetching game details for amnesia_a_machine_for_pigs...
17:53:01 | xml parsing error occurred trying to get md5 data for setup_amnesia_-_a_machine_for_pigs_1.0_(20583).exe

Did gog change something on their end or is it temporary?

z4zen commented 2 years ago

gogrepoc worked for me 5 days ago. However, today I get the same "xml parsing error occurred trying to get md5 data" error.

cyotek commented 2 years ago

I'm not hugely familiar with Python but having experienced the same issue I threw a few warn statements in and started testing URLs.

The problem seems to be in function fetch_file_info and the statement tmp_md5_url = response.url.replace('?', '.xml?').

This is an example of a URL I intercepted (with the token blatted)

https://gog-cdn-lumen.secure2.footprint.net/token=nva=1644948183~dirs=6~token=notokenthankyousecure/offline/notokenthankyou/2849/setup_cloudpunk_6809232_(64bit)_(53407).exe

There's no query string here, so the MD5 function actually just downloads the full executable and then tries to parse it as XML which clearly isn't going to fly.

A really dirty hack given I don't know any string handling functions in Python was to comment out that line and replace it with this one

                #tmp_md5_url = response.url.replace('?', '.xml?')
                tmp_md5_url = response.url + '.xml'

This assumes that there is no query and basically just slaps the .XML onto the end. As I write this response the downloader is whirring away quite successfully.

Obviously this needs fixing properly to do a check to see if a query string is present or not before deciding how to process the URL, hence no PR - even though this is working for me I don't even know if this is the way you're supposed to concat strings in Python 🤷 .

Still... hope it helps.

z4zen commented 2 years ago

It is possible to add the file extensions to SKIP_MD5_FILE_EXT in line 200 as a workaround. That skips MD5 checking completely and the "update" and "download" commands will work again.

idkicarus commented 2 years ago

@z4zen Yeah, that works. I gave it a try since I was getting the XML parsing error while trying to update Cyberpunk 2077, and now I can update the manifest and download the game without any issues.

racofer commented 2 years ago

Did gog change something on their end or is it temporary?

Interesting, as I'm having the same issues with lgogdownloader as of today being unable to parse any XML file.

IdamkinI commented 2 years ago

Thank you @cyotek for pointing out the problematic place!

I've created pull requests with something that looks to me like a little bit more robust fix 🙂

Tokariew commented 2 years ago

https://github.com/Sude-/lgogdownloader/issues/214 <-- lgogdownloader fixed it… gog is probably responding with compressed xml data?

IdamkinI commented 2 years ago

@Tokariew thanks for a link

As far as I know the requests library used in this script should handle gzip and deflate encoding of responses (here is related doc https://2.python-requests.org/en/master/user/quickstart/#response-content). So I don't think there is need to implement anything similar and decoding itself should already work out of the box.

I have encountered problems with xml parsing for some games after PR (one of them is superhot_mind_control_delete) but the data return by response is just empty.

It looks to me like we are hitting either major API change or some problems with gog servers being under heavy load. There are more details on gog forum: https://www.gog.com/forum/general/gogrepopy_python_script_for_regularly_backing_up_your_purchased_gog_collection_for_full_offline_e/post2887

GenericJohnDoeNO0 commented 2 years ago

Sooo, does anybody now when we can get a fix or how to fix by ourselfes for this mess?

Kalanyr commented 2 years ago

Had some severe weather here last week, so I'm running behind schedule on several projects. This is still on my to do list but I'm guessing it'll be late this week-ish before I get to it.

On Mon, 7 Mar 2022, 5:53 pm GenericJohnDoeNO0, @.***> wrote:

Sooo, does anybody now when we can get a fix or how to fix by ourselfes for this mess?

— Reply to this email directly, view it on GitHub https://github.com/Kalanyr/gogrepoc/issues/53#issuecomment-1060285689, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKZ33ZDJU4XJIPCAVFNM7DU6WYYZANCNFSM5OPD662A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

GenericJohnDoeNO0 commented 2 years ago

No stress, take your time! ;)

Kalanyr commented 2 years ago

Reopening this, until it's verified the issue is fully fixed, thanks to the Pull Request. I'm doing an update now, so we'll see if any errors pop up.

Kalanyr commented 2 years ago

Added some additional fixes for this issue (the GOG servers are timing out when reading some of the md5 xml files)

If you see an error of the form "Unexpected connection error fetching md5 data for ITEM. This error may be temporary. Please retry in 24 hours." Please wait 24 hours and retry it, and if the error persists let me know what game and which specific item is having the problem, so I can report it to GOG.

There is one other error I'm aware of concerning broken encoding when downloading files but I need to do some more investigation before I fix it / request reports.

I have another project I have to get back to now, but I'll try and get back to this later this week.

GenericJohnDoeNO0 commented 2 years ago

Dying light still throwing errors, will wait now for 24 hours but this problem was still over the last days persistent.

https://drive.google.com/file/d/1xZBg4eSgN0Z-AVjM6-Ldm8QCe4OBDUIJ/view?usp=sharing

pasbec commented 2 years ago

Hi,

Thank you very much for this nice tool!

As you have asked - I'm still getting the xml parsing error for several games:

elex_ii

16:30:59 | ( 2 / 15) fetching game details for elex_ii...
16:31:06 | xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-1.bin
16:31:06 | xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-3.bin
16:31:07 | xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-5.bin
16:31:08 | xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-7.bin
16:31:08 | xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-8.bin
16:31:08 | xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-9.bin

the_witcher_3_wild_hunt_game

16:34:35 | (13 / 15) fetching game details for the_witcher_3_wild_hunt_game...
16:34:57 | xml parsing error occurred trying to get md5 data for patch_the_witcher_3_wild_hunt_1.31_(A)_to_1.32.exe
16:35:15 | xml parsing error occurred trying to get md5 data for patch_the_witcher_3_wild_hunt_1.31_(A)_to_1.32.exe
16:35:38 | xml parsing error occurred trying to get md5 data for setup_the_witcher3_exp1_2.0.0.52-1.bin

worms_wmd_game

16:36:06 | (15 / 15) fetching game details for worms_wmd_game...
16:36:25 | xml parsing error occurred trying to get md5 data for setup_worms_w.m.d_1.0.0.273_(a)_(49738)-1.bin
16:36:26 | xml parsing error occurred trying to get md5 data for setup_worms_w.m.d._all_stars_pack_1.0.0.273_(a)_(49738).exe

cyberpunk_2077_game

23:37:16 | (1 / 1) fetching game details for cyberpunk_2077_game...
23:37:56 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen-cp77.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s...
23:38:31 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen-cp77.secure2.footprint.net', port=443): Read timed out. (2 retries left) -- will retry in 5s...
23:38:36 | xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_Hotfix_1.5_(53778)_to_Build_3864499Change_4719897_(53890).exe
23:38:52 | xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.5_(53680)_to_Hotfix_1.5_(53778)-1.bin
23:38:53 | xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.5_(53680)_to_Hotfix_1.5_(53778)-2.bin
23:39:33 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen-cp77.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s...
23:40:18 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen-cp77.secure2.footprint.net', port=443): Read timed out. (2 retries left) -- will retry in 5s...
23:41:03 | request failed: HTTPSConnectionPool(host='gog-cdn-lumen-cp77.secure2.footprint.net', port=443): Read timed out. (1 retries left) -- will retry in 5s...
23:41:49 | unexpected connection error fetching md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-1.bin This error may be temporary. Please retry in 24 hours.

I'll now also wait for 24 hours...

Kalanyr commented 2 years ago

Okay, pushed an update that should fully restore parity with the situation before the GOG updates (ie it should work as well as the website), let me know if any downloads are still broken in exciting ways.

If there aren't any of those, then I'll close this up and open a couple of additional topics for reporting stuff that's still broken on GOG's side to get them to fix, one for stuff that works through the website (like most of the md5 xml errors that the last 2 commits fixed) and one for stuff that's just completely broken.

I've updated the logging/display a bit to more clearly indicate if an exception is handled / unhandled (a line of text before the exception dumping indicating if the exception was handled or not) and a line indicating when the dump is finished. Hopefully this should make it clearer which errors aren't a severe problem An exception without the wrapper is probably something extremely bad. I'll probably reduce the number of things that actually log detailed exceptions in the future, once we're clearer on the status of stuff.

The list of games that I've identified that have issues with their md5 data are

a_plague_tale_innocence anomaly_korea baldurs_gate_iii bloodstained_ritual_of_the_night children_of_morta champions_of_krynn craft_the_world cyberpunk_2077_game dragon_star_varnir expeditions_rome interstellar_space_genesis iron_harvest_base kingdom_come_deliverance oriental_empires phantom_doctrine phoenix_point_year_one_edition questron_ii rimworld simon_the_sorcerer star_trek_25th_anniversary star_trek_starfleet_command_iii stellaris stronghold_warlords the_riftbreaker the_universim starpoint_gemini_3 tropico_6 warhammer_40000_gladius_relics_of_war

And something of a special case imperator_rome

Which is completely missing actual files again, and those files include the installers.

graveyard_keeper is also missing a file but it's only a patch file from a very old version to a less old version that's still too old to continue the patch chain actually get up to date, so I'm not going to lose sleep over it.

GenericJohnDoeNO0 commented 2 years ago

Unfortunatelly still many xml errors, i add a full logfile for a complete manifest generation from today.

https://drive.google.com/file/d/132OzezuoocS8SQcF8QwgKLcX5nARhYLP/view?usp=sharing

Manifest was created with command "python gogrepoc.py update -os windows -lang de en -updateonly"

Kalanyr commented 2 years ago

From a quick look all those errors seem to be handled* (they are md5 xml data errors that the last couple of commits addressed), so while it's good to know about them so I can report the issue to GOG they shouldn't actually be interfering with you adding the actual file to the manifest or downloading things anymore (it'll just be missing the md5 data).

*Except for the 403 Forbidden ones, but I think those may be files that aren't actually available for download (it looks like those are pre-order items or special bonus files that never got uploaded from the link form), you'd have to find the item with the corresponding link on the website in the Dying Light game and check that they are downloadable.

On Thu, Mar 10, 2022 at 10:49 PM GenericJohnDoeNO0 @.***> wrote:

Unfortunatelly still many xml errors, i add a full logfile for a complete manifest generation from today.

https://drive.google.com/file/d/132OzezuoocS8SQcF8QwgKLcX5nARhYLP/view?usp=sharing

— Reply to this email directly, view it on GitHub https://github.com/Kalanyr/gogrepoc/issues/53#issuecomment-1064019860, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKZ33YTMH7CYPXMWGLCK3TU7HVU3ANCNFSM5OPD662A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you modified the open/close state.Message ID: @.***>

GenericJohnDoeNO0 commented 2 years ago

ok, will have a look but now the script has to shove 300 gb to my external hdd. ;)

GenericJohnDoeNO0 commented 2 years ago

After a quick Check: I assume - for dying light - this are dlc which are not downloadable because there are no offlineinstallers for them (the skin packs i think). They are not on the downloadpage of my library and after an offlineinstallation gog galaxy "updates" the game with the missing dlcs.

After a quick check in gog galaxy: Here are all dlcs listed with their offline installers....

It seems that the homepage library is fucked up for the browser side....

Or it is something with the "Enhanced Edition":

[Crash Test Skin Pack]()
[Dying Light: The Following DLC]()
[Dying Light: Bozak Horde DLC]()
[Cuisine & Cargo DLC]() [Ultimate Survivor Bundle DLC]()

are listed with "0 Bytes" and i assume this is the content included in the Enhanced Edition Installer.

pasbec commented 2 years ago

Here is what I found from a full update on my library:

cyberpunk_2077_game
 - xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_Hotfix_1.5_(53778)_to_Build_3864499Change_4719897_(53890).exe
 - xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.5_(53680)_to_Hotfix_1.5_(53778)-1.bin
 - xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.5_(53680)_to_Hotfix_1.5_(53778)-2.bin
 - unexpected connection error fetching md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-1.bin
 - xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-4.bin
 - xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-5.bin
 - unexpected connection error fetching md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-11.bin

elex_ii
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048).exe
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-1.bin
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-2.bin
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-3.bin
 - unexpected connection error fetching md5 data for setup_elex_ii_1.00b_(54048)-4.bin
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-5.bin
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-6.bin
   -> request failed: ('Connection aborted.', BadStatusLine('п\x93=b¡rc ·\x00öbp\\Êl\x1eìÕÆ\x9b.7?F>>\x967"X=£\x0f\r<¬*ò\x1cõÙ\x02ô%¼fvx\x1a¦¤aô+Ê\x7fL}|,\x0b"\x90KåC\x07Ï$ëwp\tÏK\x1a\x10×9½&k6¥\x8czë«1?\x06?~,K\x07\x8f±!Çtèà±ÝW¥9\x8a\x89e©6yU¼ë\x1cÐZÙÑsýNÂ\x11\x8feADR&\x1c:xrÀ\x07\n'))
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-7.bin
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-8.bin
 - xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-9.bin

grim_dawn
 - unexpected connection error fetching md5 data for setup_grim_dawn_-_ashes_of_malmouth_1.1.9.5_(53749).exe

kingdom_come_deliverance
 - xml parsing error occurred trying to get md5 data for setup_kingdom_come_deliverance_-_hd_sound_pack_1.9.6-404-504u_(43503).exe
 - unexpected connection error fetching md5 data for setup_kingdom_come_deliverance__band_of_bastards_1.9.6-404-504u_(43503).exe
 - xml parsing error occurred trying to get md5 data for setup_kingdom_come_deliverance_-_hd_voice_pack_-_english_1.9.6-404-504u_(43503)-1.bin

metro_exodus
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433).exe
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-1.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-5.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-6.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-7.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-8.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-9.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-10.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-11.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-12.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-13.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-14.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-15.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-16.bin
 - xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-17.bin
 - failed to fetch https://www.gog.com/downloads/metro_exodus/85424
   -> request failed: 403 Client Error: Forbidden for url: https://content-system.gog.com/products/1407287452/users/48029035928056325/namespaces/website/download?path=%2Fsecurex&token=eyJhbGciOiJIUzI1NiIsImV4cCI6IjE2NDcwMTMyOTcifQ.eyJjb250ZW50Ijoie1widXNlcl9pZFwiOjQ4MDI5MDM1OTI4MDU2MzI1LFwicHJvZHVjdF9pZFwiOjE0MDcyODc0NTIsXCJwYXRoXCI6XCJcXFwvc2VjdXJleFwifSIsImV4cCI6IjE2NDcwMTMyOTcifQ.er8Ve_3N4DKXDLUAVK-c3Sq3PgtwC09cGRmEIm97FgI&access_token=0DccoptSnwI-MD2WyFmrpM8hFjqAr8I8863u6_KytLRQyyQ7l6WyIZPFHraaR6fm2A1FStWuK-3K_fHGic_FkOwY7HaULAPYu5K-boyXJ9BhKfhHGXqnr5fb9Mx6eI5E78nQABhlGXnuSzOmpkr1xQ.  will not retry.

metro_last_light_redux
 - unexpected connection error fetching md5 data for setup_metro_last_light_redux_2.0.0.2-2.bin

star_wars_knights_of_the_old_republic
 - unexpected connection error fetching md5 data for setup_star_wars_-_knights_of_the_old_republic_1.03_(29871)-1.bin
 - xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_1.03_(german)_(29871).exe
 - xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_1.03_(german)_(29871)-1.bin

star_wars_knights_of_the_old_republic_ii_the_sith_lords
 - xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_ii_1.0b_(29869).exe
 - xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_ii_1.0b_(29869)-1.bin
 - xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_ii_1.0b_(german)_(29869)-1.bin

the_witcher_3_wild_hunt_game
 - xml parsing error occurred trying to get md5 data for patch_the_witcher_3_wild_hunt_1.31_(A)_to_1.32.exe
 - xml parsing error occurred trying to get md5 data for setup_the_witcher3_exp1_2.0.0.52.exe
 - xml parsing error occurred trying to get md5 data for setup_the_witcher3_exp1_2.0.0.52-1.bin

the_witcher_3_wild_hunt_game_of_the_year_edition_game
 - no md5 data found for patch_witcher3_de_goty_2.0.0.51.exe
   -> request failed: 404 Client Error: Not Found for url: https://gog-cdn-lumen.secure2.footprint.net/token=nva=1646928686~dirs=4~token=0c690a551c73fb1366e67/secure/witcher_3/goty/lang/patch_witcher3_de_goty_2.0.0.51.exe.xml.  will not retry.
 - no md5 data found for patch_witcher3_de_goty_2.0.0.51-1.bin
   -> request failed: 404 Client Error: Not Found for url: https://gog-cdn-lumen.secure2.footprint.net/token=nva=1646928689~dirs=4~token=07142534667750ef7a7b0/secure/witcher_3/goty/lang/patch_witcher3_de_goty_2.0.0.51-1.bin.xml.  will not retry.
 - no md5 data found for patch_witcher3_de_goty_2.0.0.51-2.bin
   -> request failed: 404 Client Error: Not Found for url: https://gog-cdn-lumen.secure2.footprint.net/token=nva=1646928690~dirs=4~token=0bd5c0dbabe10e75fba34/secure/witcher_3/goty/lang/patch_witcher3_de_goty_2.0.0.51-2.bin.xml.  will not retry.
 - no md5 data found for patch_witcher3_de_goty_2.0.0.51-3.bin
   -> request failed: 404 Client Error: Not Found for url: https://gog-cdn-lumen.secure2.footprint.net/token=nva=1646928691~dirs=4~token=069a8164706782d282845/secure/witcher_3/goty/lang/patch_witcher3_de_goty_2.0.0.51-3.bin.xml.  will not retry.

victor_vran
 - xml parsing error occurred trying to get md5 data for setup_victor_vran_2.07(20190131)_(27183).exe
 - xml parsing error occurred trying to get md5 data for setup_victor_vran_2.07(20190131)_(27183)-1.bin
 - xml parsing error occurred trying to get md5 data for setup_victor_vran_2.07(20190131)_(27183)-2.bin

walking_dead_a_new_frontier_the
 - xml parsing error occurred trying to get md5 data for setup_the_walking_dead_a_new_frontier_patch_1_(64bit)_(20923)-2.bin

worms_wmd_game
 - unexpected connection error fetching md5 data for setup_worms_w.m.d_1.0.0.273_(a)_(49738)-1.bin
 - xml parsing error occurred trying to get md5 data for setup_worms_w.m.d._all_stars_pack_1.0.0.273_(a)_(49738).exe

xcom2
 - xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen_374751_(64bit)_(35526)-5.bin
 - xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen_374751_(64bit)_(35526)-8.bin
 - xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen_374751_(64bit)_(35526)-9.bin
 - unexpected connection error fetching md5 data for setup_xcom2_war_of_the_chosen_374751_(64bit)_(35526)-10.bin
 - xml parsing error occurred trying to get md5 data for setup_xcom2_resistance_warrior_pack_374751_(64bit)_(35526).exe

Those unexpected connection errors cause a huge traceback which makes reading the log extremely difficult at the moement.

How do we know that everyhing is handled correctly?

Kalanyr commented 2 years ago

The surest way to make sure something was handled correctly is to

1) check the !info.txt in the corresponding folder and verify that there's an entry for the file / link and that it's got a proper filename specified (this indicates the update command worked) 2) let a download command run (either generally or limited to the specific game id) and check that it either gives a pass for the existing file or correctly downloads the new file. (Which verifies that the local file is good)

If you're concerned primarily with the logs though: Anything that's A short description of a problem, followed by "The handled exception was:" , then an exception trackball followed by "End exception report" should be fine. The traceback is just extra info that might be helpful to me to understand a problem. I'll probably pair that back once I'm sure that all the errors are identified and handled properly. I'm logging more thoroughly at the moment just to verify that all the md5 data XML issues are fixed and that I'm not accidentally caching unintended errors.

On Fri, 11 Mar 2022, 2:28 am pasbec, @.***> wrote:

Here is what I found from a full update on my library:

cyberpunk_2077_game

  • xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_Hotfix1.5(53778)_to_Build_3864499Change4719897(53890).exe

  • xml parsing error occurred trying to get md5 data for patch_cyberpunk_20771.5(53680)_to_Hotfix1.5(53778)-1.bin

  • xml parsing error occurred trying to get md5 data for patch_cyberpunk_20771.5(53680)_to_Hotfix1.5(53778)-2.bin

  • unexpected connection error fetching md5 data for patch_cyberpunk_20771.31(50000)_to_Build_3836782Change_4719897dllfix(53680)-1.bin

  • xml parsing error occurred trying to get md5 data for patch_cyberpunk_20771.31(50000)_to_Build_3836782Change_4719897dllfix(53680)-4.bin

  • xml parsing error occurred trying to get md5 data for patch_cyberpunk_20771.31(50000)_to_Build_3836782Change_4719897dllfix(53680)-5.bin

  • unexpected connection error fetching md5 data for patch_cyberpunk_20771.31(50000)_to_Build_3836782Change_4719897dllfix(53680)-11.bin

elex_ii

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048).exe

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048)-1.bin

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048)-2.bin

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048)-3.bin

  • unexpected connection error fetching md5 data for setup_elex_ii1.00b(54048)-4.bin

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048)-5.bin

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048)-6.bin

    -> request failed: ('Connection aborted.', BadStatusLine('п\x93=b¡rc ·\x00öbp\Êl\x1eìÕÆ\x9b.7?F>>\x967"X=£\x0f\r<¬*ò\x1cõÙ\x02ô%¼fvx\x1a¦¤aô+Ê\x7fL}|,\x0b"\x90KåC\x07Ï$ëwp\tÏK\x1a\x10×9½&k6¥\x8czë«1?\x06?~,K\x07\x8f±!Çtèà±ÝW¥9\x8a\x89e©6yU¼ë\x1cÐZÙÑsýNÂ\x11\x8feADR&\x1c:xrÀ\x07\n'))

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048)-7.bin

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048)-8.bin

  • xml parsing error occurred trying to get md5 data for setup_elex_ii1.00b(54048)-9.bin

grim_dawn

  • unexpected connection error fetching md5 data for setup_grimdawn-_ashes_of_malmouth1.1.9.5(53749).exe

kingdom_come_deliverance

  • xml parsing error occurred trying to get md5 data for setup_kingdom_comedeliverance-_hd_sound_pack1.9.6-404-504u(43503).exe

  • unexpected connection error fetching md5 data for setup_kingdom_come_deliverance__band_of_bastards1.9.6-404-504u(43503).exe

  • xml parsing error occurred trying to get md5 data for setup_kingdom_comedeliverance-_hd_voicepack-_english1.9.6-404-504u(43503)-1.bin

metro_exodus

metro_last_light_redux

  • unexpected connection error fetching md5 data for setup_metro_last_light_redux_2.0.0.2-2.bin

star_wars_knights_of_the_old_republic

  • unexpected connection error fetching md5 data for setup_starwars-_knights_of_the_old_republic1.03(29871)-1.bin

  • xml parsing error occurred trying to get md5 data for setup_starwars-_knights_of_the_old_republic1.03(german)_(29871).exe

  • xml parsing error occurred trying to get md5 data for setup_starwars-_knights_of_the_old_republic1.03(german)_(29871)-1.bin

star_wars_knights_of_the_old_republic_ii_the_sith_lords

  • xml parsing error occurred trying to get md5 data for setup_starwars-_knights_of_the_old_republic_ii1.0b(29869).exe

  • xml parsing error occurred trying to get md5 data for setup_starwars-_knights_of_the_old_republic_ii1.0b(29869)-1.bin

  • xml parsing error occurred trying to get md5 data for setup_starwars-_knights_of_the_old_republic_ii1.0b(german)_(29869)-1.bin

the_witcher_3_wild_hunt_game

  • xml parsing error occurred trying to get md5 data for patch_the_witcher_3_wild_hunt1.31(A)_to_1.32.exe

  • xml parsing error occurred trying to get md5 data for setup_the_witcher3_exp1_2.0.0.52.exe

  • xml parsing error occurred trying to get md5 data for setup_the_witcher3_exp1_2.0.0.52-1.bin

the_witcher_3_wild_hunt_game_of_the_year_edition_game

victor_vran

  • xml parsing error occurred trying to get md5 data for setup_victor_vran2.07(20190131)(27183).exe

  • xml parsing error occurred trying to get md5 data for setup_victor_vran2.07(20190131)(27183)-1.bin

  • xml parsing error occurred trying to get md5 data for setup_victor_vran2.07(20190131)(27183)-2.bin

walking_dead_a_new_frontier_the

  • xml parsing error occurred trying to get md5 data for setup_the_walking_dead_a_new_frontier_patch1(64bit)_(20923)-2.bin

worms_wmd_game

  • unexpected connection error fetching md5 data for setup_worms_w.m.d1.0.0.273(a)_(49738)-1.bin

  • xml parsing error occurred trying to get md5 data for setup_worms_w.m.d._all_stars_pack1.0.0.273(a)_(49738).exe

xcom2

  • xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen374751(64bit)_(35526)-5.bin

  • xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen374751(64bit)_(35526)-8.bin

  • xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen374751(64bit)_(35526)-9.bin

  • unexpected connection error fetching md5 data for setup_xcom2_war_of_the_chosen374751(64bit)_(35526)-10.bin

  • xml parsing error occurred trying to get md5 data for setup_xcom2_resistance_warrior_pack374751(64bit)_(35526).exe

Those unexpected connection errors cause a huge traceback which makes reading the log extremely difficult at the moement.

How do we know that everyhing is handled correctly?

— Reply to this email directly, view it on GitHub https://github.com/Kalanyr/gogrepoc/issues/53#issuecomment-1064255680, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKZ334LX5ENEHVM5BXXCPDU7IPL3ANCNFSM5OPD662A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you modified the open/close state.Message ID: @.***>

pasbec commented 2 years ago

The surest way to make sure something was handled correctly is to 1) check the !info.txt in the corresponding folder and verify that there's an entry for the file / link and that it's got a proper filename specified (this indicates the update command worked) 2) let a download command run (either generally or limited to the specific game id) and check that it either gives a pass for the existing file or correctly downloads the new file. (Which verifies that the local file is good) If you're concerned primarily with the logs though: Anything that's A short description of a problem, followed by "The handled exception was:" , then an exception trackball followed by "End exception report" should be fine. The traceback is just extra info that might be helpful to me to understand a problem. I'll probably pair that back once I'm sure that all the errors are identified and handled properly. I'm logging more thoroughly at the moment just to verify that all the md5 data XML issues are fixed and that I'm not accidentally caching unintended errors.

Thank you very much for the clarification. I very much appreciate your tool and your active support!

GenericJohnDoeNO0 commented 2 years ago

Thumbs up for your work Kalanyr!

Kalanyr commented 2 years ago

Okay, this seems like it's resolved so closing this thread. I've opened two additional threads for reporting the GOG side errors

One for problems that happen on GOGrepoc but the item can still be downloaded from the website https://github.com/Kalanyr/gogrepoc/issues/60

And one where the download is also unavailable on the GOG website

https://github.com/Kalanyr/gogrepoc/issues/61