QBeat: Always fails to fetch mods

[Jibodeah]

Everytime I run ./QBeat --list (or any other command where it would fetch the mod list) I get ERROR: Failed to fetch mod list from beatmods.

It's not my internet connection as visiting the url it fetches via my browser does work.

Version: Alpha 9 (but I've had the problem since I first tried QBeat at Alpha 7) OS: Fedora 31

My armchair guess is there's some dependency I'm missing? The only dependencies I see mentioned for the built release is Qt5 (which I definitely have) and minizip (which I also have).

[geefr]

That's odd..

I just tested in a clean fedora 31 VM, other than needing to install minizip-compat things are working as expected for me.

Do you get the error 'QSocketNotifier: Can only be used with threads started with QThread'? Which version of Qt do you have installed?

I'd agree this can only be a dependency/deployment issue really, I'm just giving the url to Qt and letting it do the hard bits.

[Jibodeah]

I have the minizip-compat 1.2.11-20.fc31 x86_64 package installed, so unless I need the development or 32 bit versions that should be fine.

I don't get the QSockNotifier error. The only other thing in the log is QFileSystemWatcher::removePaths: list is empty (twice).

My version of Qt is 5.12.5-1.fc31, but there are loads of Qt packages it's possible I don't have the one it needs or something. What's the installed qt packages on your VM?

I tried a distro-sync on my Qt packages just in case I had something strange locally compared to a clean install but no luck.

[geefr]

Pretty sure if you didn't have the required packages it just wouldn't run. I didn't need to install qt, so fedora must have it in the base install. I've only build qbeat as 64-bit so no need for the compat libs.

Anything above qt 5.11 will work, never had any cross-distro issues in that regard.

Here's a build with more logging for that area, should log a more precise error and all of the data sent back from beatmods.

QBeat-beatmods-logging.tar.gz

[Jibodeah]

Alright that gives me ERROR: Failed to fetch mod list from beatmods: 6: URL: https://beatmods.com/api/v1/mod?status=approved&gameVersion=1.5.0. Error 6 seems to be SslHandshakeFailedError according to the docs.

Snooping the network with wireshark I see a "Change Cipher Spec" as part of the negotiation, but it doesn't look like it errors the connection due to that...

Could it be a bug in Qt? I tried fetching the same URL using Python Requests and it went without a hitch.

[geefr]

Not too surprising..

I can fetch the url through wget/curl, so I'd agree it's Qt specific. Maybe there's something on your system that prevents the connection working correctly.

This build tells Qt to accept any protocol, in theory it should work or there's something else to tweak in that area: QBeat-any-ssl-protocol.tar.gz

It could be a certificate trust issue, but I'd expect if Qt fails then curl would also. Not entirely sure how Qt does its network requests internally.

Otherwise I can add an option to disable ssl verification, but as qbeat downloads and extracts whatever beatmods lists that certainly can't be the default

[Jibodeah]

Same error, sadly...

Works fine with curl for me too.

[geefr]

Have another build: QBeat-ssl-logging-env-vars.tar.gz

set these environment variables (to anything, '1' works) it'll change some options in the connection:

• QBEAT_DISABLE_SSLVERIFY : Disable peer verification completely
• QBEAT_ALLOW_ANY_SSL : Allow any ssl protocol (Not sure what the default it tbh)

I can force the connection to fail by setting the ssl protocol to obscure things, but otherwise can't see any way it's failing.

I've added some more logging that in theory will print any ssl errors when they happen, couldn't get it to log any here.

I guess the wireshark logs would be the best place to look, but I wouldn't know where to begin on how SSL works and such.

[Jibodeah]

Disabling the SSL with the environment variable does, of course, work. So I can try modding Beat Saber now at least. (which knowing my luck with modding Beat Saber will prove to be a whole other can of worms)

Anyway, here's the SSL error.

ERROR: SSL Errors during request: 
     : The issuer certificate of a locally looked up certificate could not be found
ERROR: Failed to fetch mod list from beatmods: 6: URL: https://beatmods.com/api/v1/mod?status=approved&gameVersion=1.5.0

I found this Qt Bug which may be related... but the workings of SSL are largely beyond me too so I'm not sure.

[geefr]

Well at least the error logging works. My best guess is that your system doesn't trust cloudflare for some reason, and that curl has overridden that? If it's the qt bug I can just use libcurl directly but that's more dependencies and awkward to use.

As you've got a workaround for now I'll let you enjoy the modding process and come back to this one later. Most likely I'll just add the ssl verification flag properly and see if anyone else gets this kind of thing.

[Jibodeah]

Installing doesn't work... Looks like it's still trying to use SSL even though I set the environment? (it works on the --list command though)

[jibodeah@apathia QBeat-ssl-logging-env-vars]$ env QBEAT_DISABLE_SSLVERIFY=1 ./QBeat --install "BSIPA"
WARNING: SSL peer verification has been disabled
ERROR: SSL Errors during request: 
     : The issuer certificate of a locally looked up certificate could not be found
ERROR: Failed to download mod file from beatmods: https://beatmods.com/uploads/5dc475ab8ed3a56f6197baf8/universal/BSIPA-3.13.2.zip
ERROR: Failed to download mod: BSIPA
Installing mod: BSIPA
ERROR: Failed to install mod: BSIPA

(also found a report bugzilla which seems similar and does also affect me)

[geefr]

Whoops, I missed the 2nd function..try this one: QBeat-more-ssl-hacks.tar.gz

[Jibodeah]

...now it thinks the returned list of mods is empty...? This definitely worked before!

[jibodeah@apathia QBeat-more-ssl-hacks]$ env QBEAT_DISABLE_SSLVERIFY=1 ./QBeat --list
WARNING: SSL peer verification has been disabled
QFileSystemWatcher::removePaths: list is empty
QFileSystemWatcher::removePaths: list is empty
[jibodeah@apathia QBeat-more-ssl-hacks]$ env QBEAT_DISABLE_SSLVERIFY=1 ./QBeat --install "BSIPA"
WARNING: SSL peer verification has been disabled
Installing mod: BSIPA
ERROR: Unable to find mod named: BSIPA
QFileSystemWatcher::removePaths: list is empty
QFileSystemWatcher::removePaths: list is empty

[geefr]

I need another coffee..One more build, and my compiler warnings are now stricter: QBeat-please-work-this-time.tar.gz

The removePaths message is still unknown to me, I'm not using QFileSystemWatcher directly at least.

[Jibodeah]

Alright, that works and installs mods and patches the game and everything.

...the patched game then... crashes on boot... But that's another issue! (I'll see if I can find the relevant logs...)

[geefr]

The one last point is critical if you hadn't seen it yet:

With any luck you should have a modded copy of Beat Saber now, only one this left to do

Restart Steam
Right click Beat Saber -> Properties
Check 'Force the use of a specific Steam Play compatibility tool
Select the tool called 'Proton BeatSaber'
Go have fun

And if you didn't run the game once before installing mods that's another one that'll do it. The special version of proton is needed to work around a bug when loading winhttp.dll, which bsipa uses to hook into the game.

[Jibodeah]

Yeah I had done. Looks like the game nearly immediately segfaults though.

Still! This is progress! Previously via other modding tools (such as installing mods on Windows and copying over the files) I had been unable to get it to show any sign of being modded at all on Linux (game would boot but there would be no mods). And this time I can see the logs and it looks like the mod (I'm only trying with BSIPA right now) is there. The game just crashes. (there's no obvious cause in the log either...)

I guess I'll try a complete wipe and redo from start, but that's for another day.

[geefr]

(Closing due to inactivity)

[Jibodeah]

For what it's worth I did a fresh reinstall when I upgraded to Fedora 32 which resolved the issue for me personally.

Meanwhile the upstream bug was found to be a duplicate of another bug where the cause is known.