[Android11] [Homebrew Store] Connection Error 200!

[ghost]

What should happen

My phone is connected to a stable wifi but always connection error I can't download homebrew games on ppsspp.

Who would this benefit

Everyone?

Platform (if relevant)

Android

Games this would be useful in

None

Other emulators or software with a similar feature

IDK

Checklist

• [X] Check the latest git build in case it's already implemented.
• [X] Search for other requests of the same feature.

[Panderner]

Homebrew store works fine for me. i tested my PC and my Android phone it's working.

[unknownbrackets]

It might be that your Internet provider is applying a proxy on your traffic, and changing it slightly in a way that breaks the homebrew store.

If you load it off data, do you also get an error? It may be hard to troubleshoot this without seeing the request, but an adb logcat might also help (if it can include PPSSPP messages.)

-[Unknown]

[ghost]

Same, :(

[anr2me]

Is the homebrew store a web server? if it's may be ask them to open it on a browser to see whether it's accessible or not

[ghost]

The playstore version is working fine 😏

[unknownbrackets]

This is the URL it tries to load: http://store.ppsspp.org/index.json

Very strange if it's only affecting latest git... some permissions issue?

-[Unknown]

[Panderner]

Still same in latest build for my phone:

[ghost]

ppsspp latest git build for android logcat.txt

ppsspp playstore version homebrew is working. logcat.txt

[ghost]

Even using mobile data same issue. PPSSPP v1.11.3-1176

[ghost]

WiFi Info where my phone is connected

[iota97]

Could reproduce this on Samsung A31 (or A41, I really can't remember) running Android 11:

1019 Bad (scoped storage #14619 merge) 1003 Good

[Panderner]

Hey @Gamemulatorer what Android version are you using? Me I'm using Android 10.

[ghost]

Panderner my phone is redmi note 9 a11 MIUI 12.5.1 latest update.

[Panderner]

MIUI 12.5.1 = Android 11. We need to test more devices running Android 11 that are affected for this issue.

[hrydgard]

huh, really surprising. Works just fine for me on Android 12 beta, even. Gotta be some ISP meddling - but on the other hand, if 1003 is good and 1019 bad... Hm.

[hrydgard]

@Gamemulatorer here's a trick for getting less noise in logs:

adb logcat -s DEBUG PPSSPPNativeActivity PPSSPP NativeGLView NativeRenderer NativeSurfaceView PowerSaveModeReceiver InputDeviceState

Filters out of all the other stuff that's not interesting.

[ghost]

MIUI 12.5.1 = Android 11. We need to test more devices running Android 11 that are affected for this issue.

@Panderner you should try also update your Poco M3 to a11 miui 12.5.1 and see if you can reproduce this issue?

[hrydgard]

Could you also get another log with the command line I posted above? I'm having trouble finding anything relevant in the long log.

[Panderner]

MIUI 12.5.1 = Android 11. We need to test more devices running Android 11 that are affected for this issue.

@Panderner you should try also update your Poco M3 to a11 miui 12.5.1 and see if you can reproduce this issue?

MIUI 12.0.8.0 is the latest version for my POCO M3. Sorry @Gamemulatorer no MIUI 12.5 for my phone yet.

[ghost]

Try this Panderner go to about phone > click miui 12 5 times > update settings > receive update early. Goodluck 🎉

[ghost]

@Gamemulatorer here's a trick for getting less noise in logs:

adb logcat -s DEBUG PPSSPPNativeActivity PPSSPP NativeGLView NativeRenderer NativeSurfaceView PowerSaveModeReceiver InputDeviceState

Filters out of all the other stuff that's not interesting.

I don't have a PC to try that :( But I try a different app that catch logs hope it can help :) logcat_08-22-2021_19-49-25.txt

[iota97]

Seem to be JSON related, getting a fancy Connection error: 200 in the latest build :)

[hrydgard]

So if something is mangling the JSON, that's quite surprising :(

We really should be using HTTPS, it's just such of a mess to support cross platform without bringing in huge libraries like OpenSSL. Could of course do HTTPS through Java on Android...

~But hang on, I don't even see in the logic how it's possible that you get the error with 200 .. must be missing something.~ No I do see..

[Panderner]

Try this Panderner go to about phone > click miui 12 5 times > update settings > receive update early. Goodluck 🎉

I tried for these methods but no MIUI 12.5 Beta received for my phone. Still latest stable version.

[hrydgard]

@iota97 maybe you can log out the JSON that you're receiving?

[iota97]

Today I have a PC unable to compile for android I'm afraid, I'll look into that tomorrow.

I will test #14754 after the buildbot will finish (or may I can get an apk from CI now somehow?)

[ghost]

https://github.com/hrydgard/ppsspp/commit/821a6a60aaa838318798e1619409f3a6a1d254ab

[iota97]

Is our Buffer code safe?

Here dest is a std::string * and data_ a std::vector<char> https://github.com/hrydgard/ppsspp/blob/821a6a60aaa838318798e1619409f3a6a1d254ab/Common/Buffer.cpp#L56

And then it get passed to this (with dest now being the char * from &(*dest)[0] https://github.com/hrydgard/ppsspp/blob/821a6a60aaa838318798e1619409f3a6a1d254ab/Common/Buffer.cpp#L61

Can we memcpy thing around like that? I think this is kinda an UB... Not sure if it's causing this tho'.

[unknownbrackets]

The std::string was already resized before that, and the buffer is writable. It returns a char & specifically, which is writable. This is also for sure not the only place we do this. It'd be better to use dest->data() instead of &(*dest)[0] though, from a clarity perspective... I think that used to not be available in some compilers we targeted.

-[Unknown]

[iota97]

Thx for the explanation unknown (seem std::string::data return a const char * tho').

Anyway I got a debug build running and hit this (are assert disabled on non debug build?): https://github.com/hrydgard/ppsspp/blob/541524b119371fd2eea918cfc3885d0ff896b170/ext/gason/gason.h#L32

Seem malloc for the buffer returned an address that is too big for gason... Basically it store pointers and tag in the same uint64_t.

This changes seem to fix it (but I'm too tired to get into the logic ATM so it's more for testing than an actual fix yet):

git diff
diff --git a/ext/gason/gason.h b/ext/gason/gason.h
index 1f1cd976a..8cde2d339 100644
--- a/ext/gason/gason.h
+++ b/ext/gason/gason.h
@@ -21,16 +21,17 @@ struct JsonNode;
 #define JSON_VALUE_TAG_MASK 0xF
 #define JSON_VALUE_TAG_SHIFT 47

-union JsonValue {
+struct JsonValue {
     uint64_t ival;
     double fval;
+    void *pval;

     JsonValue(double x)
-        : fval(x) {
+        : ival(JSON_VALUE_NAN_MASK), fval(x) {
     }
     JsonValue(JsonTag tag = JSON_NULL, void *payload = nullptr) {
-        assert((uint64_t)payload <= JSON_VALUE_PAYLOAD_MASK);
-        ival = JSON_VALUE_NAN_MASK | ((uint64_t)tag << JSON_VALUE_TAG_SHIFT) | (uintptr_t)payload;
+        ival = JSON_VALUE_NAN_MASK | ((uint64_t)tag << JSON_VALUE_TAG_SHIFT);
+       pval = payload;
     }
     bool isDouble() const {
         return (int64_t)ival <= (int64_t)JSON_VALUE_NAN_MASK;
@@ -38,9 +39,9 @@ union JsonValue {
     JsonTag getTag() const {
         return isDouble() ? JSON_NUMBER : JsonTag((ival >> JSON_VALUE_TAG_SHIFT) & JSON_VALUE_TAG_MASK);
     }
-    uint64_t getPayload() const {
+    void* getPayload() const {
         assert(!isDouble());
-        return ival & JSON_VALUE_PAYLOAD_MASK;
+        return pval;
     }
     double toNumber() const {
         assert(getTag() == JSON_NUMBER);

@Gamemulatorer could you please test this as well?

[anr2me]

200 is a normal response code for a successful HTTP GET isn't?

May be it contains HTML/JS instead of JSON because something in the middle tampering the content? (ie. anti-virus or ISP). For example, in my country ISP often randomly inject Ads or telemetry javascript on HTTP response, they sometimes redirect HTTP request to an Ads page where we need to click a link to go the the actual page we're trying to open, which is why i had issue when PPSSPP trying to check it's version.json in the past (before the parsing got fixed) because it tried to parse HTML/JS as JSON.

[unknownbrackets]

Ah, I guess it's this: https://github.com/vivkin/gason/issues/36

See here: https://github.com/vivkin/gason#nan-boxing

I think either the separate pointer or separate value for the tag can work. I was a bit concerned about this when I saw gason, but it seemed like the best available parser...

-[Unknown]

[iota97]

The fix on that issues seem fine, I would go for that (not really sure how license work in this situation tho' eheh).

[jyaif]

The proposed fix is under the same license as the library, so feel free to use it to patch things up.

[ghost]

Confirm fixed. Thanks everyone!