Sync slows down at certain times every day these days.

minyoung-jun commented 1 year ago

Steps to Reproduce

UTC 13:00 ~ 15:00 Occurs
CodePush.sync slow down (Synchronization takes more than a minute)
At that time, many users suffer.

Expected Behavior

Fast sync (less than 3 seconds)

Environment

react-native-code-push version: 7.0.4
react-native version: 0.64.4
iOS/Android/Windows version: iOS/Android
Does this reproduce on a debug build or release build? Release
Does this reproduce on a simulator, or only on a physical device? Physical Device

alexdr00 commented 1 year ago

+1 I can confirm. This has happened to me as well.

HwangYoonSeong commented 1 year ago

me too

wkdgudcjf commented 1 year ago

Me too. The app was running very slowly because the checkForUpdate method was unresponsive. Can't we timeout the checkForUpdate method?

nikolastill commented 1 year ago

We are seeing this currently too 😢 Sometimes it hangs indefinitely.

zeusstl commented 1 year ago

Today, it is extremely bad. Literally, the app is not loading at all for many people and it is just stuck on the update screen.

This is an ongoing issue with many tickets. Many people rely on this service and are willing to pay to have better quality. But if the promise from microsoft is that it is supported, maintained and free, then please provide a decent service.

evmorov commented 1 year ago

Yes. Can confirm. Right now it's getting better, but the past hours it barely worked

zereight commented 1 year ago

Me too, same problem happening around the same time in south korea region.

2023.03.14 ~ 2023.03.16(current) every UTC 13:00 ~ 15:00, KST 22:00 ~ 00:00

Sometimes, checkUpdate has 500 error. If checkUpdate is normal, bundle download speed is very slow. 화가난다

zeusstl commented 1 year ago

I would recommend to everyone to please leave a comment here

And also...

To please log into appcenter codepush and let them know that this is an ongoing issue that has gotten worse and worse.

There is a "?" on the top right of the codepush dashboard, where you can send a support request.

kyle-ssg commented 1 year ago

Hmm this is really worrying to see. Starting to wonder forking CodePush to upload/download to a different CDN might be an idea.

SoohyungJung commented 1 year ago

I'm also facing the same issue 🥲

zereight commented 1 year ago

short-term solution timeout like this

https://github.com/microsoft/react-native-code-push/issues/1942#issuecomment-977608845

goguda commented 1 year ago

Just started getting a full on 500 Internal Server Error here...

Edit: 500 status seems to be resolved but getting intermittent SSL errors here now.

It's not the first time something like this has happened with this service. As much as I appreciate Microsoft promoting it as a free service, we should at least make sure it's reliable. Otherwise would be nice to give the user the option to self-host their bundles or offer a paid tier.

alexdr00 commented 1 year ago

It seems it's working smoothly now. However, considering the instability CodePush servers might suffer from time to time, we created a workaround so we don't leave the users waiting for too long in case this happens again. The tradeoff is that the user will not see the update applied right away, but at least we won't leave it waiting in the loading screen.

The workaround is basically this: Check for update, install update immediately. If it takes too long (more than 4 seconds), go beyond loading screen and install update silently in the background.

This workaround implements the timeout solution mentioned in the comment above and it also combines it with a silent update.

I created a gist for this workaround if you are interested.

I put all the update logic inside a hook called "useCodePushSetup". The most important part to see here is the function "checkUpdates". It first executes CodePush.checkForUpdate with a timeout of 4 seconds; If it takes longer than 4 seconds then it throws the "timeout" error. Inside the catch, it updates the state and marks the app as ready (allowing the user to go beyond the loading screen) and it also executes CodePush.sync with installMode set to ON_NEXT_RESUME which starts a silent update. Thanks to the silent update the user won't notice any delay in the CodePush servers.

ngocle2497 commented 1 year ago

same +1

gourav-singhal commented 1 year ago

+1

minyoung-jun commented 1 year ago

Fortunately, it doesn't seem to happen in recent days. Has anyone experienced this in the last 2 days?

alexdr00 commented 1 year ago

@minyoung-jun it's been working well for me too.

luzhanyi commented 1 year ago

+1

minyoung-jun commented 1 year ago

Currently, this issue appears to be resolved. The cause was unknown. I'll close the issue for now, and open it when it happens again.

minyoung-jun commented 1 year ago

@luzhanyi Has this problem happened to you again?

zeusstl commented 1 year ago

The issue is back. Stuck in check updating to the point where the app doesn't even load

Please see my comment from earlier to let them know that the issue is back.

I would recommend to everyone to please leave a comment here

And also...

To please log into appcenter codepush and let them know that this is an ongoing issue that has gotten worse and worse.

There is a "?" on the top right of the codepush dashboard, where you can send a support request.

antseburova commented 1 year ago

I can confirm that the issue is back

goguda commented 1 year ago

Yup, happening again...

minyoung-jun commented 1 year ago

The issue reopened😢

zeusstl commented 1 year ago

This issue should not be closed.

Extreme latency on a service that is the first thing a user sees is a nightmare.

The fact that in the US users this morning had to wait upwards of 2 minutes for codepush to finish loading and outside the US is was dramatically worse - not even loading in some cases - is a gigantic problem.

Somehow, this dramatic event is not even noted on the status page: https://status.appcenter.ms/

minyoung-jun commented 1 year ago

Maybe this announcement is addressing this issue? https://status.appcenter.ms/incidents/4vz96dxnj6jb

zeusstl commented 1 year ago

Good point. Good to know that they are actively monitoring it. I'm guessing that the fix yesterday did not work, but it's good to know there is activity towards a resolution.

zeusstl commented 1 year ago

Codepush check updates is completely dead right now. Nothing loads!

Literally no user can load the app. It is fully stuck on check for updates.

goguda commented 1 year ago

500 errors, again............

Update: looks like they have acknowledged an issue with App Center that could be related? https://status.appcenter.ms/

goguda commented 1 year ago

Intermittent 500 errors again preventing full load. Got to do something on our end for now, this isn't fair to our userbase...

goguda commented 1 year ago

So, I've been observing the times of this behaviour, and it does seem to be the worst around the times OP states (13:00 ~ 15:00 UTC). In my observations I'd almost say 11:00 ~ 15:00 UTC, but time of day does definitely seem to be an important factor in causing the issue.

After 15:00 the service seems to operate normally. Very strange issue.

Ahmadin29 commented 1 year ago

+1

pierroo commented 1 year ago

Random question but, would all of this happen if we use the default CodePush configurations where it:

check if there is an update (in background by default I think)
launch the app regardless
if there is an update, download it in background (no loading screen etc)
only install it upon next launch, which should take almost time and no longer relies on remote fetching

I may be wrong but I feel like all the above is happening because you force a download & install right away when it detects an update, and force a checking screen while it searches for said update?

(asking these since i am considering using codepush from now on but this scares me)

goguda commented 1 year ago

Random question but, would all of this happen if we use the default CodePush configurations where it:
* check if there is an update (in background by default I think)

* launch the app regardless

* if there is an update, download it in background (no loading screen etc)

* only install it upon next launch, which should take almost time and no longer relies on remote fetching
I may be wrong but I feel like all the above is happening because you force a download & install right away when it detects an update, and force a checking screen while it searches for said update?

(asking these since i am considering using codepush from now on but this scares me)

You should be fine if you're using it this way, it's checking for an update before the app loads that it causing the issue for people.

pierroo commented 1 year ago

thank you so much @goguda .

So that confirms that by default codepush will NOT block the app while checking for an update? (nor slow it down while waiting for a response if it ever takes too long as mentioned above?)

goguda commented 1 year ago

thank you so much @goguda .

So that confirms that by default codepush will NOT block the app while checking for an update? (nor slow it down while waiting for a response if it ever takes too long as mentioned above?)

Correct

microsoft-github-policy-service[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs within 15 days of this comment.

microsoft-github-policy-service[bot] commented 1 year ago

This issue will now be closed because it hasn't had any activity for 15 days after stale. Please feel free to open a new issue if you still have a question/issue or suggestion.

microsoft / react-native-code-push