Hot Code Push broken on iOS

[EliArtist]

My iOS app can't use hot code push to get new versions I upload. After some searching I believe I can narrow down the problem to a bug in cordova-plugin-meteor-webapp which tries to link the same file twice. I'm pretty sure that is what breaks hot code push. I found a similar issue here: https://github.com/meteor/cordova-plugin-meteor-webapp But that repository is untouched since February so I decided to open a new issue here.

My current Meteor version is 1.7.1-rc.5, I'll try to update the meteor version to 1.8, test again and post an update here.

EDIT: After trying to build an iOS App with Meteor 1.8 I'm getting the same error

[dovydaskukalis]

Same issue, it stops working every time we update font files. Same error message as in https://github.com/meteor/cordova-plugin-meteor-webapp/issues/56.

[menelike]

Also related to https://github.com/meteor/cordova-plugin-meteor-webapp/pull/59

Right now we need to release a new app version on font updates.

[fizx1]

I was about to post a separate bug report, but ours seems like it could be the same issue. I'll post our details below. We are eager to see this working and willing to help out however possible. Thanks!

We are having some issues with Hot Code Push. In short, we are deploying to Galaxy using a Windows PC. We use that same PC to build the Android/Cordova apk. Hot Code Push works for these Android clients. Hot Code Push doesn't work, though, for iOS clients -- the iOS client is built on another machine. We are using the METEOR_CORDOVA_COMPAT_VERSION system, so it should work, from my understanding. There are no apparent error messages (maybe I'm not looking in the right place), but when I redeploy a new app version to Galaxy, only Android clients process the update. Thanks in advance and please let me know how I can help further. The long version... We are deploying from a Windows machine to Galaxy using the following commands:

SET DEPLOY_HOSTNAME=galaxy.meteor.com
SET METEOR_CORDOVA_COMPAT_VERSION_ANDROID=v0001
SET METEOR_CORDOVA_COMPAT_VERSION_IOS=v0001
meteor deploy appname.meteorapp.com --settings settings.json --mobile-server=https://appname.meteorapp.com

The hot code push version number is confirmed to be making it to the Galaxy server; we can see the file http://appname.meteorapp.com/__cordova/manifest.json is visible and contains the entry: "cordovaCompatibilityVersions":{"android":"v0001","ios":"v0001"}

We are building for iOS App Store with the following command: METEOR_CORDOVA_COMPAT_VERSION_IOS=v0001 meteor build build_directory --server https://appname.meteorapp.com

After building, we open the build in XCode, create an archive, and upload the archive to the App Store for approval and release. (please PM me for the actual appname, thanks!)

[rj-david]

Anybody figured how to workaround this bug?

[fizx1]

We haven't tried it yet, but you could try doing all the builds on the same machine. It's a long shot.

On Wed, Oct 24, 2018, 8:42 PM rj-david notifications@github.com wrote:

Anybody figured how to workaround this bug?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/meteor/meteor/issues/10277#issuecomment-432874511, or mute the thread https://github.com/notifications/unsubscribe-auth/AReKdBiyQ15zhK3vEuU9nVfYtFX08FB5ks5uoQj5gaJpZM4Xakek .

[dovydaskukalis]

@fizx1 We always deploy from the same machine and manually specify compatibility version in config file, same issue.

[sugoke]

Same issue here. I have an app deployed on heroku, and when I build for iOS, HCP seems to be broken. I have seen a few messages regarding this “bug” on other forums but no real explanation.

`2018-10-18 12:40:36.942203+0200 Premia[1933:140955] Serving asset bundle version: 3e53d8a74f99d185edd94272536b033999610f80

2018-10-18 12:40:39.200604+0200 Premia[1933:140955] Start downloading asset manifest from: manifest.json -- https://premia.herokuapp.com/__cordova/

2018-10-18 12:40:40.369383+0200 Premia[1933:141234] Downloaded asset manifest for version: 9c4c53cdc986cd3eb49b9f99b097a1bb76305b42

2018-10-18 12:40:40.370495+0200 Premia[1933:141234] Download failure: Skipping downloading blacklisted version

2018-10-18 12:40:40.374600+0200 Premia[1933:140955] ERROR: {"line":36,"column":30,"sourceURL":"http://localhost:12224/plugins/cordova-plugin-meteor-webapp/www/webapp_local_server.js"}`

[rj-david]

@benjamn, we would like to ask some guidance regarding this?

@RealHandy seems to have some clues on what's happening here: https://github.com/meteor/cordova-plugin-meteor-webapp/issues/56

[rj-david]

Just saw these related PRs - hope they made it to 1.8.1.beta soon

https://github.com/meteor/cordova-plugin-meteor-webapp/pull/61 https://github.com/meteor/cordova-plugin-meteor-webapp/pull/62 https://github.com/meteor/meteor/pull/10219

[Wade-BuildOtto]

I am seeing the same issue on 1.7.0.4 with METEOR_CORDOVA_COMPAT_VERSION_IOS in use, its ignoring it even though what is listed on the manifest is accurate. I might role back to see if that helps, but it's starting to get costly rolling out small fixes via app store updates.

[rj-david]

The PRs related to the cordova-plugin-meteor-webapp has been merged to master! Wohoo!

For those who wanted to follow the PR for the main meteor project related to HCP and other cordova stuff, subscribe to this PR: https://github.com/meteor/meteor/pull/10339

[RealHandy]

BTW, this is also issue #10181. See https://github.com/meteor/cordova-plugin-meteor-webapp/pull/59 for the PR. The code there works if you need it right away, but it was done in a way to highlight the problem for determining a proper fix, which @benjamn is now doing (yay!).

[Wade-BuildOtto]

has anyone tried meteor update --release 1.8.1 to see if the update fixes the issue.

"Not it."

[Wade-BuildOtto]

any movement on this?

[sugoke]

Interestingly, the problem occurs only when deployed (to Heroku) on my side. On localhost, HCP works. 1.8.0.1 did not fix.

[macrozone]

I can also confirm that this is completly broken on IOS.

I get the error

Error: Could not link to cached asset: Error Domain=NSCocoaErrorDomain Code=516 "“315ECD_9_0.ttf” couldn’t be linked to “webfonts” because an item with the same name already exists." UserInfo={NSSourceFilePathErrorKey=/var/mobile/Containers/Data/Application/239C2D9C-6CF1-4DF0-9964-B63372F459CA/Library/NoCloud/meteor/PartialDownload/app/webfonts/315ECD_9_0.ttf, NSUserStringVariant=(
    Link
), NSDestinationFilePath=/var/mobile/Containers/Data/Application/239C2D9C-6CF1-4DF0-9964-B63372F459CA/Library/NoCloud/meteor/Downloading/app/webfonts/315ECD_9_0.ttf, NSFilePath=/var/mobile/Containers/Data/Application/239C2D9C-6CF1-4DF0-9964-B63372F459CA/Library/NoCloud/meteor/PartialDownload/app/webfonts/315ECD_9_0.ttf, NSUnderlyingError=0x2812c3c90 {Error Domain=NSPOSIXErrorDomain Code=17 "File exists"}}

and i definitly did not add any assets!

[macrozone]

Interestingly, the problem occurs only when deployed (to Heroku) on my side. On localhost, HCP works. 1.8.0.1 did not fix.

yes, on develop it works, but not once you deployed it.

[bmanturner]

I upgraded from 1.6 to 1.8, and HCP didn't work even though I used the compatibility override. I had to immediately rollback.

Is this the correct issue for what I experienced?

[macrozone]

I upgraded from 1.6 to 1.8, and HCP didn't work even though I used the compatibility override. I had to immediately rollback.

Is this the correct issue for what I experienced?

yes, i think so. I noticed too late and can't roll back easily now :-(

[bmanturner]

That's unfortunate... I had a huge mongo migration (servers and schemas) as a part of this deployment, so luckily I was able to revert the merge and go back to using the old mongo instance.

On Dec 19, 2018, at 9:40 AM, Marco Wettstein notifications@github.com wrote:

I upgraded from 1.6 to 1.8, and HCP didn't work even though I used the compatibility override. I had to immediately rollback.

Is this the correct issue for what I experienced?

yes, i think so. I noticed too late and can't roll back easily now :-(

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[sugoke]

Did Meteor team comment on that?

It is quite an important feature of the framework.

Le 19 déc. 2018 à 16:45, Brendan Turner notifications@github.com<mailto:notifications@github.com> a écrit :

That's unfortunate... I had a huge mongo migration (servers and schemas) as a part of this deployment, so luckily I was able to revert the merge and go back to using the old mongo instance.

On Dec 19, 2018, at 9:40 AM, Marco Wettstein notifications@github.com<mailto:notifications@github.com> wrote:

I upgraded from 1.6 to 1.8, and HCP didn't work even though I used the compatibility override. I had to immediately rollback.

Is this the correct issue for what I experienced?

yes, i think so. I noticed too late and can't roll back easily now :-(

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/meteor/meteor/issues/10277#issuecomment-448641012, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANFS1VoHWwKKYqKQ9RS20b6Ir0jbH8Mcks5u6l79gaJpZM4Xakek.

[RealHandy]

Yes, this is all the same issue: see also #10181. The underlying issue is at https://github.com/meteor/cordova-plugin-meteor-webapp/issues/56

Note: I've just uploaded what I think is an improved workaround: meteor/cordova-plugin-meteor-webapp#59

Update: I have provided in the PR (see link above) some instructions on how to implement this. If you know how to get the forked PR version, put it on your local drive, and make that the version in your project, you can use this to fix the problem, but it does require that you push a new iOS version out to users with the fix in it.

[bmanturner]

@RealHandy So there is no way to update a server from 1.6 to 1.8 without also publishing client updates to the app stores? I've never had to do that when upgrading meteor in the past.

[RealHandy]

@bmanturner This bug is related to changes to assets. If the upgrade causes some asset change, then the bug will bite you on iOS and hot code push will fail (as described in the posts across the various related issues and as seen in @macrozone's post above (https://github.com/meteor/meteor/issues/10277#issuecomment-446917778). If HCP is failing, the workaround can only be applied by an update to the user's phone version, since the bug fix is in cordova code.

For example, "webfont.ttf" is in my /__cordova/manifest.json file twice. See the difference between the two lines in bold:

{"path":"packages/fortawesome_fontawesome/upstream/fonts/fontawesome-webfont.ttf","where":"client","type":"asset","cacheable":false, "url":"/__cordova/packages/fortawesome_fontawesome/upstream/fonts/fontawesome-webfont.ttf", "size":138204,"hash":"ae832c436a36bdb3b60c1a7fcab1349fd9a5e3d8","sri":"a+iafir/sdUB+O+VSL7/Xlb6AA45KL4fOfmsaW9Nvm1qkIgx3P+L0R6Lw4OTTLMordHfCo6rty9ZhAq8xAyBig=="},

{"path":"packages/fortawesome_fontawesome/upstream/fonts/fontawesome-webfont.ttf","where":"client","type":"asset","cacheable":false, "url":"/packages/fortawesome_fontawesome/upstream/fonts/fontawesome-webfont.ttf", "size":138204,"hash":"ae832c436a36bdb3b60c1a7fcab1349fd9a5e3d8","sri":"a+iafir/sdUB+O+VSL7/Xlb6AA45KL4fOfmsaW9Nvm1qkIgx3P+L0R6Lw4OTTLMordHfCo6rty9ZhAq8xAyBig=="},

This is causing the HCP failure, and you need the workaround to get past the failing behavior caused by these assets being in the manifest twice.

[rj-david]

Fix to iOS HCP has been committed. Excited to test this early next week

https://github.com/meteor/meteor/pull/10248#issuecomment-453613510

[benjamn]

Note: to test these changes, you'll need to run meteor update --release 1.8.1-beta.13 first.

Thanks especially to @RealHandy for finding an elegant solution! https://github.com/meteor/cordova-plugin-meteor-webapp/pull/59

[rj-david]

Thanks @benjamn and @RealHandy. Great way to start 2019. Hope for the stable release of 1.8.1 soon

[rj-david]

I deleted my previous replies due to new data we collected

Confirming that Hot Code Push now works with iOS with one caveat. There is a need for a complete uninstall of any version of the app running previous versions of Meteor with the HCP bug. When we tried updating an existing iOS app, HCP did not work even with the fix.

But when we tried to uninstall the app and did a fresh install of the iOS with the app, HCP worked.

(...testing android)

[rj-david]

HCP is not working in Android with Meteor 1.8.1-beta.13 (tested on multiple android phones)

We are using the Reload.isWaitingForResume() feature and it can detect that a new version is available. But once we did a refresh, the new version is not being successfully loaded.

[menelike]

@rj-david

If I understand https://github.com/meteor/cordova-plugin-meteor-webapp/pull/62 correctly, https://github.com/meteor/meteor/pull/10219 needs to merged to stay backward compatible.

Do you have any logs, hard to help without them?

[RealHandy]

@rj-david HCP won’t remove files from an app’s assets, it only downloads new ones and updates files that have changed, but no deletes. Any chance this is why you have to do an uninstall to get to your proper updated app version? Just a guess. Do you get any errors in the console when it doesn’t work without a reinstall?

[rj-david]

Unfortunately we were testing with production builds (through apple test flight and android release channels). So the apps were not set on debug mode. We will test 1.8.1-beta.14 today and if it won't be successful, we will build for debugging.

[rj-david]

I can confirm now that HCP works for both android and iOS using Meteor 1.8.1-beta.14

[RealHandy]

@rj-david That’s great!

[rj-david]

Is this related to any of the changes for beta.14? https://github.com/meteor/meteor/issues/10426

[RealHandy]

The HCP fix itself wouldn’t be related, it doesn’t do anything with those configs.

[rj-david]

We noticed recently that during development using an actual device, HCP is not updating the build in the actual device.

We are using Reload.isWaitingForResume() and when this changes, we trigger HCP with window.location.replace(window.location.href)

We will debug why this is happening but does any of you have any idea on what to check?

[rj-david]

@RealHandy, @benjamn

We are actively developing an app in beta and launched last Monday using Meteor 1.8.1-beta.17. We found an issue with HCP in iOS with the latest fix. The first HCP in iOS worked without problem every time. After that the HCP stopped working again.

[menelike]

@rj-david Are you able to provide some logs from the affected devices?

[rj-david]

@rj-david Are you able to provide some logs from the affected devices?

We don't have any. What is the best way to get these logs?

[RealHandy]

@rj-david You connect your iPhone to your Mac with a usb, then you run Xcode and from the top menu select Window - > Devices , then select your connected device from left side list. The current log will show scrolling at the bottom of the screen, though you may need to hit the little arrow at the bottom to expand logs. Then you reproduce the problem and it will show in the logs. A lot of other apps and system stuff may be filling the logs, so you’re likely to need to filter the log by something specific to your app (or just have good eyes).

On Feb 14, 2019, at 3:55 AM, rj-david notifications@github.com wrote:

@rj-david Are you able to provide some logs from the affected devices?

We don't have any. What is the best way to get these logs?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[rj-david]

@RealHandy, I'm not 100% sure if we can replicate this for a development setup. Our team has been developing with an iPhone connected for quite some time and nobody raised an issue (I was using Ubuntu + Android though). We just noticed the problem when we started releasing in production. In all versions, the iOS app will HCP once when a new version is deployed. For the next version, it won't.

I'll check with those in our team using iphone and mac tomorrow.

Anybody knows if APM can catch this and what should we look at in the apm logs?

[rj-david]

This is the only relevant logs that we can see as related (i.e. the BLACKLIST part)

2019-02-18 12:54:32.961450+0800 Bountee[15907:7331347] App startup confirmed
2019-02-18 12:54:41.794419+0800 Bountee[15907:7331347] Start downloading asset manifest from: manifest.json -- http://192.168.1.18:3003/__cordova/
2019-02-18 12:54:41.848020+0800 Bountee[15907:7332764] Downloaded asset manifest for version: 38a4c717e7e8c577e9eac4a97566cd3c0013001b
2019-02-18 12:54:41.848491+0800 Bountee[15907:7332764] BLACKLIST - blacklistedVersions: []
2019-02-18 12:54:41.848580+0800 Bountee[15907:7332764] Finished downloading new asset bundle version: 38a4c717e7e8c577e9eac4a97566cd3c0013001b
2019-02-18 12:54:41.883098+0800 Bountee[15907:7331347] Serving asset bundle version: 38a4c717e7e8c577e9eac4a97566cd3c0013001b
2019-02-18 12:54:42.886473+0800 Bountee[15907:7331347] App startup confirmed
2019-02-18 12:54:42.921297+0800 Bountee[15907:7331347] Start downloading asset manifest from: manifest.json -- http://192.168.1.18:3003/__cordova/
2019-02-18 12:54:42.979222+0800 Bountee[15907:7332978] Downloaded asset manifest for version: 38a4c717e7e8c577e9eac4a97566cd3c0013001b

[RealHandy]

This shows you’re successfully getting the asset manifest and the app is resetting due to having gotten it. Then it gets the asset manifest in the restarted version. All of that looks as it should be and there are no errors due to asset update. I don’t know what the blacklist means, it has no versions in its list, and the app is proceeding to restart and function after that. There’s no error here that I can see, unless it’s related in some way to the blacklist. This log has no issue related to the original #10277 bug. Bummer, not much to go on here.

On Feb 20, 2019, at 9:55 PM, rj-david notifications@github.com wrote:

This is the only relevant logs that we can see as related (i.e. the BLACKLIST part)

2019-02-18 12:54:32.961450+0800 Bountee[15907:7331347] App startup confirmed 2019-02-18 12:54:41.794419+0800 Bountee[15907:7331347] Start downloading asset manifest from: manifest.json -- http://192.168.1.18:3003/__cordova/ 2019-02-18 12:54:41.848020+0800 Bountee[15907:7332764] Downloaded asset manifest for version: 38a4c717e7e8c577e9eac4a97566cd3c0013001b 2019-02-18 12:54:41.848491+0800 Bountee[15907:7332764] BLACKLIST - blacklistedVersions: [] 2019-02-18 12:54:41.848580+0800 Bountee[15907:7332764] Finished downloading new asset bundle version: 38a4c717e7e8c577e9eac4a97566cd3c0013001b 2019-02-18 12:54:41.883098+0800 Bountee[15907:7331347] Serving asset bundle version: 38a4c717e7e8c577e9eac4a97566cd3c0013001b 2019-02-18 12:54:42.886473+0800 Bountee[15907:7331347] App startup confirmed 2019-02-18 12:54:42.921297+0800 Bountee[15907:7331347] Start downloading asset manifest from: manifest.json -- http://192.168.1.18:3003/__cordova/ 2019-02-18 12:54:42.979222+0800 Bountee[15907:7332978] Downloaded asset manifest for version: 38a4c717e7e8c577e9eac4a97566cd3c0013001b — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[KoenLav]

@rj-david you mentioned in the 1.8.1 release thread that the second HCP on iOS fails consistently.

Do others experience the same thing?

@wojtkowiak maybe you could chime in?

[rj-david]

@KoenLav, yes, we were able to confirm it multiple times in our project.

Just a few minutes ago, we released a new version without the mdg:reload-on-resume package to test if that was the one causing the issue. Unfortunately with iOS, we have an earlier release with a change in mobile-config.js which meant we have to go through the App Store review. We expect it to be approved within 12 hours from this post. Will post here if we find something useful.

[mozfet]

@rj-david you mentioned in the 1.8.1 release thread that the second HCP on iOS fails consistently.

Do others experience the same thing?

@wojtkowiak maybe you could chime in?

I am definitely experiencing it. I've been unable to deploy iOS for more than a month. Works fine locally, but not in App Store. Could not find good logs.

[rj-david]

I am definitely experiencing it. I've been unable to deploy iOS for more than a month. Works fine locally, but not in App Store. Could not find good logs.

@mozfet, are you using at least 1.8.1-beta.14 ?

[mozfet]

I am definitely experiencing it. I've been unable to deploy iOS for more than a month. Works fine locally, but not in App Store. Could not find good logs.

@mozfet, are you using at least 1.8.1-beta.14 ?

Yup - METEOR@1.8.1-beta.14

[buymybm100]

I am using METEOR@1.8.1-beta.14 and been pushing new builds to my HockeyApp Test environment. The hot push fails the first time and all subsequent times - on IOS and Android.