Open Source Hot Code Push / Over The Air Updates

alexcroox commented 4 years ago

Plugin Request

Name: Capacitor Hot Code Push Package: @capacitor-community/hot-code-push

Platform(s)

Android, iOS

Existing Solutions

MS Code Push - Free, very popular for Cordova, no Capacitor support yet

App Flow - Paid solution with very expensive plans given the low update limits. No API to upload built web directory on normal plans, not very flexible to binaries built elsewhere

Description

Over the Air Updates / Hot Code Push is the ability to update the js/css/html of your app in real time without having to go through the app store and waiting for reviews. This allows you to push critical fixes out quickly inline with your web app deployments instead of waiting days for Apple to approve entire binary updates.

It would be great if a plugin could facilitate over the air updates from a server/endpoint of our choosing where the developer controls the hosting of the OTA updates.

The key to the plugin is the ability to download a zip file of your updated dist folder, extract it, tell Capacitor to launch from this new directory until told otherwise.

There are other factors like verifying updates, choosing when to install them (forced on launch or next time the app is given focus after downloading etc), and optionally downloading only what has updated.

riderx commented 3 years ago

@mlynch Your Community and Startup pricing for indies and startups don't include Over-the-air update, your plan do not make sense for indie, tiny team do already build the app themself and will continue, what we cannot do is the OTA, who is missing.

OTA with self-hosted assets is the only feature i need from appflow

mlynch commented 3 years ago

@riderx apologies, the Startup pricing does include live updates, currently the community one only includes native builds and not live updates.

riderx commented 3 years ago

@mlynch so indie are still stuck, 49 is very expensive for indie and do not include what we need... Please take time to think about it.

amakh commented 3 years ago

If we use self-hosted option, do we still need to subscribe to a plan and pay when we exceed the limit?

Silventino commented 3 years ago

@mlynch so indie are still stuck, 49 is very expensive for indie and do not include what we need... Please take time to think about it.

Actually, it is $249 per month lol And you have to pay a whole year upfront........

levarberry commented 3 years ago

I'll say this because I know Max can not.

You have to ask yourself why as an Ionic developer you can't afford $49. What is it about your business or career that doesn't allow you to spend $49 on a tool that can help you? The amount of time and effort saved is tremendous. You probably spend $40 on hosting and spend $20 on canva and a lot of misc money on other things. $49 or even $200 is not a lot of money to ask. If you don't have a Mac and need Native builds ... pay someone to build them for you. If you can't really can't afford $49 for live updates .. that means to me your users are not paying you any money and therefore should wait on App Store updates.

In addition... if Ionic was to charge anything less ... I would be in fear that this project would fall because of lack of funding. Ionic is a business not a charity. I run a business not a charity, so paying helps me feel like they are going to put the effort in to keeping this alive.

Open your eyes to what has happened with all the other FREE / Low cost solutions. They are gone!

Is it fair to Max to fund a project that costs him a lot of money and nothing in return just for the sake of people who pay him nothing? Do you write Ionic apps for free for your clients? He's already not charging for the web components and Capacitor and Stencil... what more do you want from him?

It doesn't matter what country you are from. This is not a lot of money. If you need money ... you can always work in my company and I'll pay your $49.

Levar BERRY - CEO (o) (513)318-1360 ext.101 (c) 513-403-1210 (e) @. @.> (w) http://edrivenapps.com/

On Mon, Oct 18, 2021 at 10:25 AM Martin DONADIEU @.***> wrote:

@mlynch https://github.com/mlynch so indie are still stuck, 49 is very expensive for indie and do not include what we need... Please take time to think about it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/capacitor-community/proposals/issues/43#issuecomment-945829138, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAC7NB242QT3EI6UMANSO2DUHQU7HANCNFSM4OTVDCNQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

gustavopch commented 3 years ago

@levarberry You have a point, it's important that Ionic can fund itself and be profitable. That said, you're terribly wrong here:

It doesn't matter what country you are from. This is not a lot of money.

The impact of US$249/mo is massively different for a company based in a country where GDP per capita is US$60k/year Vs. where it's US$6k/year.

Look at Spotify. They seem to charge US$9.99/mo in the US. Do you know how much they charge in Brazil? US$3.60/mo in the current dollar rate. Is that because people here are penny-pinchers? Of course not. It's because people in this country are not as rich as people in the US. Spotify knows that and they're not a charity. While 50% of people in the US live with US$3k/month or more, it's about 2% of people in Brazil. And Brazil isn't even one of the poorest countries in the world — there are a lot of people in a much worse condition just because of where they were born. Sometimes it really seems like people in 1st world countries can't even realize how well they are compared to the rest of the world.

Please read this: https://ourworldindata.org/poverty-minimum-growth-needed.

Edit: sorry for the rant, but I think it's important for the discussion.

mlynch commented 3 years ago

@levarberry thanks for the support, we really appreciate it.

It's hard balancing making money and giving things away for free. I strongly believe that Ionic generating revenue is the best thing for the community. Without it, the services companies rely on are on shaky ground, as we see with App Center. It's great that App Center is free or very cheap, but it's not so great when that means you can't rely on it as a business.

The revenue we generate goes back into our open source offerings to make them better. We know not everyone will choose or be able to be a paying customer of our commercial services but I do think it's important to point out where that money ultimately goes.

Also we very much understand prices can be felt very differently in different countries. However, we also operate in an environment where we are paying US prices so there's a limit to how low we can go before something simply becomes not economically viable for us. I am guessing Microsoft had been losing money on Code Push bandwidth for a long time but kept it around as a loss leader. Seems that bill is coming due. We won't have that issue as we've priced our product with our costs fully in mind to make sure it's sustainable.

lincolnthree commented 3 years ago

Since I've been active on this thread, I'll just leave two main comments/ideas here.

Before I get into that, though, I will say that the folks at Ionic have worked very hard on providing solid tools for developers to use, and they have have been very generous with open-sourcing most of them and supporting the open-source community. Thank you. You guys should definitely charge money for this service, and your new pricing is MUCH more fair than what I think the prior model presented as a barrier to small/indie devs.

Okay...

First, as a proponent of open-source, and someone who's worked on the Capacitor plugin for AppCenter/CodePush, I will say this. If you are not able to pay for a $249/mo (~$3000/yr) subscription for this service, then please consider supporting this project, and consider starting an open-source project implementing the AppCenter server/API endpoints:

https://github.com/mapiacompany/capacitor-codepush

AppCenter and the APIs required to make a functional code push server are NOT complicated. Building a UI to make it all pretty and easy to use would be a bit more work, but if you are willing to implement the AppCenter APIs, and upload a few files for clients to check / download when you want to push out an update, then this would be very straightforward to implement and host yourself.

The capacitor-codepush plugin makes only two or three API calls, roughly, including the download of the new distribution, and this could fairly easily be reproduced in an open-source solution.

Client calls to check for availability / metadata of new update - provides its current version and device info.
If update is available, client requests download of zip file including new code.
Client calls to notify server update was successful or if rolled back. (This could be ignored if you want to run wild and don't care about tracking errors.)

Second, I'd also suggest contacting Microsoft support, as per their suggestion, and asking them to open-source their Cordova codepush solution.

"If you have an Apache Cordova app and need assistance evaluating other options, our team is more than happy to assist you via our support channel. Simply open a support ticket under the drop-down menu on the right top corner of the portal."

gustavopch commented 3 years ago

@mlynch To clarify my point, I don't think simply lowering prices is necessarily a sustainable solution. I can absolutely see how US$249/mo for a company based in the US can be quite cheap if you consider how much time would be spent building all of that from scratch. Country-based pricing on the other hand may make it affordable in places where it isn't today. End result may be growing the revenue by reaching potential users that are not reached today.

riderx commented 3 years ago

@levarberry thanks for your comment, you said many true things, but you forgot many point. I do build app that are free to use and open souce, in your exemple i don't have the right to have great tool, a bit unfair. Second point, 49 dollar is expensible for indie but doable as i said. But in this price we don't have the only interesting feature, the code-push . I don't know any indie who need CI/CD for building since you are alone you build on your machine, want you cannot do is sending OTA update. @mlynch thanks you made some effort and to keep the dialog open, but this offer do not fit the need, in my point of view. Even when i was in startup less than 1M i don't see the point. In the other way around for sure 49 for only OTA and no CI/CD look smarter, of course when the team got bigger i will need your CI/CD 100% of the time .

riderx commented 3 years ago

I'm on my way to build a module who allow you to download update from a given url. Something like :

import { updateApp } from 'capacitor-updater'
updateApp('URL_TO_S3_OR ANY_PLACE')

It will try to :

download data from url
unzip data it
replace the current code with the new one

The plugin will be simple, you manage on your side the update timeline and when to update. If you are interested put emoji or send me email to discuss about it.

ap1969 commented 3 years ago

I'm also dusting of the old nordnet components. CLI is done, running proper tests tonight. I can probably do the java, but it's been a long time since I worked with it, so a second pair of eyes would be great. I've never worked on native IOS apps, so a hand with those would definitely be needed, if anyone is interested?

https://cordovahotcodepush.com for updates.

riderx commented 3 years ago

@ap1969 let's join our efforts : https://github.com/riderx/capacitor-updater I just made the base of Android and IOS needs. It's still WIP but any help is welcome ! The blocking poitn now is to manage the unziping for downloaded files. I will continue next week

ap1969 commented 3 years ago

Does it only work with capacitor?

To be blunt, I'm avoiding using anything to do with ionic because I hit problems with their Appflow, and got zero help to diagnose the problem. I couldn't even see logs, and they wouldn't send any.

lincolnthree commented 3 years ago

If you're looking for a client side module that does the update for Capacitor... it's already built. Just need to build out the server end! (E.g. Just need to swap out the AppCenter URL for your own server -- but we need help with the server / CLI part!)

Here's the client code: https://github.com/mapiacompany/capacitor-codepush

o-alexandrov commented 3 years ago

@lincolnthree you don't need a server (one that you would pay for computing expense) to have OTA updates. Here's a simple scenario:

clone an example capacitor.js web app
make the example capacitor.js web app to look at a specific URL (for example AWS S3 endpoint) for updates with version.json file
build an example web app and upload to AWS S3
- together with the build files, add a version.json file

In the proposal above, all you need to determine whether to download a new version is the static version.json file:

GET https://aws-s3-endpoint/version.json
- if a version is different than what you have client-side (on the device), download the build and replace a directory with web app files you have client-side
- if needed, add authorization layer based on your preference
- later on, add hooks api client-side, so developers could add their custom logic at least for onSuccess (on successful download) and onError
- if needed, on the backend (AWS S3) in this case, a developer can add a Lambda function for AWS S3 events

That's what I believe @riderx and @ap1969 are implementing with capacitor-updater

ap1969 commented 3 years ago

Exactly that. I've got android updates working already. Moving onto iOS now.

tolutronics commented 3 years ago

Does it only work with capacitor?

To be blunt, I'm avoiding using anything to do with ionic because I hit problems with their Appflow, and got zero help to diagnose the problem. I couldn't even see logs, and they wouldn't send any.

@mlynch

riderx commented 3 years ago

@ap1969 yes the plan was to make it work in capacitor only, no server need no overload.

ap1969 commented 3 years ago

Hi @riderx Sorry, I meant would your solution work in plain Cordova as well? Andy

riderx commented 3 years ago

@ap1969 i used the capacitor start plugin from here: https://capacitorjs.com/docs/plugins/creating-plugins So it seems it's not.

jfbloom22 commented 3 years ago

@mlynch I am a huge fan of Ionic AppFlow. It has worked flawlessly for me. The reason I hope this thread continues to work on a community solution is because I am running into the 100 install limit on the Community plan and it does not make sense to spend $500 a month to deploy an app to 30 users.

I expect there are a lot of developers out there supporting apps for an internal mobile team of 20 - 200 users who will also find $500 too pricy.

Additionally, the app I am working on is an Enterprise iOS app so we don't publish to the store. I have 0 interest in building the binary on AppFlow.

It would be brilliant if the Community plan included 500 live updates. Or better yet, if there was a Live Update only package for $50/month that included 5,000 Live Updates. Bottom line is that I am happy to pay a bit more than $50 a month for more than 100 live updates, but no way can I justify $500 to distribute an app to 30 people.

ap1969 commented 3 years ago

@riderx OK. I don't know much about capacitor, hence my asking. That's a shame.

ThodorisElissaios commented 3 years ago

@mlynch i was paying $49/mo just for the live updates, now you are asking me $499/mo ?

Who is the genius behind this move?

Ionic used to be an awesome framework with reasonable priced tools. (i have been using the platform since DragNDrop builder era & Push notifications)

And now you are asking $4,999/yr ?

nonsenseless commented 3 years ago

@ThodorisElissaios I'm not crazy about their pricing model either but there's no need to be rude. They're trying their best to figure out how to make an actual functional business model and that ain't easy when everybody expects you to give your work away.

ThodorisElissaios commented 3 years ago

Please everyone excuse me if i am rude. I completely understand they're trying their best to figure out how to make an actual functional business model.

In my opinion asking for an (almost) 1000% increase sounds rude.

mlynch commented 3 years ago

Appreciate all the feedback, truly!

@ap1969 sorry you had a bad experience with Appflow, we've made a bunch of improvements to the tool and our support process recently so I hope that you give it another try some day!

@jfbloom22 I can understand if it's not a fit in that case. Most of our customers are using our other features in tandem or in larger companies that need other enterprise features. If you just need a few live-updates and nothing else I agree that's not going to fit with our current plans but we will certainly keep this in mind.

And @nonsenseless @ThodorisElissaios thanks. I understand that charging for things in the open source world can be contentious, but I do want to point out that because Ionic generates revenue, we can actively employ full time engineers to work on projects you love like Capacitor and Ionic. Our business model seems to work best when we don't charge Indies or Students much or anything, but focus on charging serious businesses because they need a lot more and they also need a lot more support and security guarantees for everything we sell. That's why I'm legitimately happy if the indie market has a path forward here but I want to be transparent about how we decide to put an official stamp on something.

At the end of the day Ionic is still here and doing well while App Center, which doesn't charge enough, is starting to crumble. It's really hard if you don't have real revenue to point to when it comes time to validate your existence in a big company it turns out.

jfbloom22 commented 3 years ago

If anyone is in a similar boat and needs a few more live updates on the Community plan, I recommend contacting Ionic support and simply asking. I asked for 500 live deploys, explained the situation (above) and they graciously granted my request.

I have built over 12 solutions with Ionic Framework, starting when it was in beta years ago. Recently I started to have my doubts because of this poor pricing with Live Updates, but now my confidence in Ionic is completely restored. Really impressed with the folks over there at Ionic.

nonsenseless commented 3 years ago

@mlynch I will say that it would be great to have more transparent pricing on the enterprise-level options. We have quite a few clients and, with all due respect to sales people, there's nothing more frustrating than having to talk to them to get basic pricing information and then navigate an endless series of "JUST WANTED TO FOLLOW UP" emails until one or the other of you finally dies and can stop using email.

mlynch commented 3 years ago

@nonsenseless agreed, we rolled out much more pricing transparency recently which has been well received. Open to feedback if we're missing the mark still! https://ionic.io/pricing

nonsenseless commented 3 years ago

@mlynch Nice, this is definitely much improved! For the indie/startup plans, what kind of live update limits are there? And what kind of documentation do you ask for from start-ups? We work with a lot of folk developing MVPs so it would be helpful to have a clearer overview of the process and options to give them.

riderx commented 3 years ago

@mlynch do you know the pricing of revenue cat ? https://www.revenuecat.com/pricing Maybe you can ask user to connect they webhook of apple store and google to check the app revenue and do a limit like revenue cat do ?

jamesyoung1337 commented 3 years ago

If you're looking for a client side module that does the update for Capacitor... it's already built. Just need to build out the server end! (E.g. Just need to swap out the AppCenter URL for your own server -- but we need help with the server / CLI part!)

Here's the client code: https://github.com/mapiacompany/capacitor-codepush

@lincolnthree What exactly is needed here? Is there some basic documentation for the endpoints and how the cli works?

I would be interested in hosting something for open source and other projects.

Can we verify this plugin works as expected, more or less, and that only the URL needs changing along with the new endpoints and cli? Since you say it's not overly complex to implement the endpoints save for a pretty interface I would like to get this up and running and perhaps work on a nicer interface later, when it works.

lincolnthree commented 3 years ago

@jamesyoung1337 Awesome! Yes, I both helped develop this iteration of the plugin, and use it in production on over 100,000 devices. (iOS and Android).

I think the best documentation is the source code of the client capacitor plugin, which is... typical for OSS, but of course not ideal :) Microsoft liked to keep their server API proprietary, and open-source the client stuff.

Essentially the current Capacitor CodePush plugin makes two API calls:

Call: What version is available for me? (GET /checkUpdate?clientVersion=1.0.0&stage=PROD), which returns info on available update for a given update stage/track.
If an update is available, then it downloads the file (GET /path/to/1.0.1.zip) and unpacks it to the device filesystem. If the update succeeds, it can reload immediately, wait until background timeout, or wait until next launch. If it fails, the update is rolled back.
Call: Then it informs the server if the update succeeded or failed (POST /updateResult?version=1.0.2&status=[success/fail]) - this is not essential and could be skipped. It is used for stats/metrics - and is useful to see if you need to pause/roll back an update.

It's really not much more complicated than any of the new solutions being proposed on this thread. It also already supports verifying the MD5 checksum and signed contents of the archive to prevent malicious updates / hacking.

So as you can see, there is a lot of great / useful functionality in here that really would be quite easy to re-use with a few tweaks. It would be a shame to lose all of this work.

What is needed is:

An update/fork of the AppCenter CLI to replace their bundle uploading to build a local bundle, sign, and upload to S3 or GitHub or wherever the file is hosted, and add the new build to the update server/service.
A service to receive the API calls from the client and do some version comparisons with the available updates and the client app version / return appropriate update info/file URL.
Updates to the client plugin depending on / made to match the changes above.
A service to track success/failure of updates (optional)

Anything beyond that, like a GUI to pause/roll back an update is just candy. I think this should be pretty easy to host straight out of GitHub or S3 using a JSON file, and possibly moving the version comparison logic to the client, to be honest.

lincolnthree commented 3 years ago

Here are some of the HTTP calls, which I believe happen solely in the TypeScript/JS, and not in native code:

Check for update (code-push package included via NPM): https://github.com/microsoft/code-push/blob/master/src/script/acquisition-sdk.ts#L97

Download the file: https://github.com/mapiacompany/capacitor-codepush/blob/main/src/remotePackage.ts#L56

jamesyoung1337 commented 3 years ago

@lincolnthree Thank you, that is very helpful!

I have been working my way through the source code, and I have the general hang of it as far as the appcenter cli goes.

I will post a link to a github repo here when I have some legwork done.

I'm interested in some details on how you have used in production: that was this plugin and maybe your own backend, localhost or your own server? I'm working my way through the commit history, will let you know how I get on.

Thanks and talk soon!

lincolnthree commented 3 years ago

@jamesyoung1337 My pleasure! Perhaps we should move this discussion somewhere else. Possibly an issue in the Codepush Plugin repo? Happy to share more details off this main thread or via email / Discord: LincolnThree num 1078

jamesyoung1337 commented 3 years ago

@lincolnthree Yes, let's do that. An issue at your repo here? Discord channel: Capacitor and CodePush

riderx commented 3 years ago

i just update capacitor-updater it's working on IOS, i improved the DX a bit :

  import { CapacitorUpdater } from 'capacitor-updater'

  const testUpdate = async () => {
    const version = await CapacitorUpdater.download({
      url: 'https://github.com/Forgr-ee/Mimesis/releases/download/0.0.1/dist.zip',
    })
    await CapacitorUpdater.set(version)
  }

This will download the version and load it. Edit: hard reload app work too ! Have fun ! if you need something please open issue

riderx commented 3 years ago

I made it !

o-alexandrov commented 3 years ago

@riderx pls consider to create some UI around it to offer a commercial version, so you could support your solution long term :)

lincolnthree commented 3 years ago

@riderx Do you have a discord server we could all chat on? Possibly set up a server to collaborate? I am happy to get this started if it would be helpful.

riderx commented 3 years ago

@lincolnthree you can join here : https://discord.gg/VnYRvBfgA6

I'm working on the Android Version

@o-alexandrov I'm not sure what I can do for commercial version.

I will setup sponsorship account, I think it can be an easy solution too. Feel free to join the Discord to discuss about it

riderx commented 3 years ago

Just released capacitor-updater V1.0.05 with Android hot reload, like IOS I need to make it persistent between app kill.

riderx commented 3 years ago

And Boom Found the way to persistency in Android with 1.0.9 thanks @jamesyoung1337 for the help !

npm i capacitor-updater and enjoy, I improved the doc too : capacitor-updater

akashdusky commented 2 years ago

https://github.com/riderx/capacitor-updater/projects

this project works flawless its dead simple you got control dont have to rely on anyone for your code push you could use static S3 or put a api in front for basic analytics perfect 🔥

riderx commented 2 years ago

Thanks @akashdusky the project have still few issue, but the idea was to make it very simple ! For example I didn't find a way to revert the app and use the original code again. Or handle launch issue, yet. But I will continue to work on it !

riderx commented 2 years ago

I foud the way to revert to original app !

hoveit commented 2 years ago

Great work! @riderx

capacitor-community / proposals