Allow more than 20 apps per github repo

[shawnvega]

Describe the solution you'd like I build multiple apps from one repo, and each app uses one github webhook. Github has a limit of 20 webhooks, therefore I can build a maximum of 20 apps from one repo. I would like app center to reuse a single webhook for multiple apps so that I can have webhooks available to do other things, and build more than 20 apps from one repo.

Describe alternatives you've considered I've considered making more branches and building from those branches, but this seems like a lot of extra work.

[dennispan]

@shawnvega Thanks for reaching out! This is indeed something we are aware of as a limitation. Some customers reached out to GitHub or Bitcket to increase their webhook limit. Didn't work for everyone but some did. So this may be something worth trying for now.

Do you mind sharing how you are using the repository and App Center? Do you have more than 20 applications in the repository? Or do you create multiple App Center apps for each application from the repository?

[shawnvega]

@DennisPan I'll reach out to git hub and ask them to increase the number of webhooks. The repository builds about 27 apps at the moment, and we are about to add 3 more. That would bring us to 30 apps.

[shawnvega]

I've written into github support about increasing my limit, and they have denied my request. below is the response from github support.

Ivan Zuzak (GitHub Developer Support)

Jun 27, 8:55 AM UTC

Hi Shawn,

Thanks for reaching out. Did I notice correctly that each of those webhooks set up on that repository is being triggered by the same exact events and making the same exact webhook deliveries to the same service? If so, then there are 20 webhooks doing the same exact thing -- they're delivering the same events 20 times to the same service. That kind of behavior is wasteful, and the webhook limits were introduced to prevent exactly that kind of wasteful behavior (which is a performance and reliability concern). We can't support a higher number of webhooks for such a use-case.

Instead, the service you're using should be able to work with just one webhook defined on the repository since the events they get from that webhook are the same as the events they get from 20 webhooks they have now. An alternative would be that you build a lightweight middleman proxy and point one webhook to that proxy. The proxy would then forward webhook deliveries it receives to any number of destination URLs that you want (since you control the proxy).

I wish I had better news for you, but I don't. If you have any other questions -- please let me know.

Best, Ivan

[dennispan]

Thanks for sharing this @shawnvega . Unfortunately this isn't something we can easily change. Technically this is achievable, but the work on our side isn't trivial. Given other priorities, I'm not sure when we will be able to get to this.

[bperreault]

@DennisPan - I've been asked to add my comments and contribute to this issue's supporters. We have ran into this same issue and I found this thread through a google search. What do we need to do to get this work item scheduled for development? Can we discuss how we might help? It is this or move on for us, unfortunately.

We have 26 apps listed in AppCenter, with 24 configured. We are not able to connect any more of them because the error is: Linking failed: Failed to configure the GitHub Webhook for the repository. Do you have admin permissions?
We have admin permissions ( as demonstrated by the previous apps which are connected )

[dennispan]

@bperreault Thanks for reaching out. An alternative would be using the GtiHub marketplace app: https://github.com/marketplace/app-center, with added benefits of PR builds. There are couple caveats like billing should go through GitHub, onboarding goes through GitHub etc. This integration doesn't need webhook, which would address your situation.

However, let me look for a quick workaround first with your current setup. The problem right now is that our webhook url has app id. I want to see how easy it is for us to remove that and make it more generic.

[dennispan]

@bperreault Would using the GitHub marketplace app as I suggested above work for you?

Looking at how things are setup on our side, there isn't a quick workaround (aside from using GitHub marketplace app). A proper implementation of a generic webhook url without app id is non-trivial.

[bperreault]

Hi, @DennisPan . We have installed the GitHub marketplace app and the devs will be working with it to see if it will do what we need it to. Thanks for your help.

[bperreault]

Hello @DennisPan . Our team is in the process of configuring AppCenter through the GitHub app as you have suggested. Is there some limitation in that app which can be resolved? We are not able to configure even as many apps as we were straight through AppCenter.

We have this issue now:
"From the dev: I have configured 5 iOS client apps and after that I can't configure more. It is showing an error screen like one attached. I have sent a chat to appcenter team and waiting for the reply."

[shawnvega]

We have moved over to bitrise.io Which will allow us to build multiple apps from one repo. Not as many webhooks are required.

On Wed, Oct 9, 2019, 10:36 PM Bobbi Perreault notifications@github.com wrote:

Hello @DennisPan https://github.com/DennisPan . Our team is in the process of configuring AppCenter through the GitHub app as you have suggested. Is there some limitation in that app which can be resolved? We are not able to configure even as many apps as we were straight through AppCenter.

We have this issue now:

"From the dev: I have configured 5 iOS client apps and after that I can't configure more. It is showing an error screen like one attached. I have sent a chat to appcenter team and waiting for the reply." [image: unnamed-4] https://user-images.githubusercontent.com/543334/66481626-4c119180-ea6f-11e9-9276-d6397a6e504c.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/appcenter/issues/591?email_source=notifications&email_token=AA5ATNHGBV4ODZ5V6TE74RLQNXF5HA5CNFSM4HZNTOLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAXXD5Q#issuecomment-539980278, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5ATNBW5PE22UNIA64RVE3QNXF5HANCNFSM4HZNTOLA .

[bperreault]

@shawnvega thank you for that update. We'll take a look.

[dennispan]

@bperreault I haven't been able to find the support conversation. You can ask support to forward the conversation to me directly, or let me know the user who initiated the conversation so that I can do a better search.

The error you are seeing typically means the user who started the process don't have the right access to the GitHub organization or user which has the GitHub App installed. But if it was the same user, which had success and now don't, that likely is something else. I will need more information like the user name, etc, to understand

[bperreault]

@DennisPan, understood on what the error normally means. and as you have stated, since the same user created all the existing projects it's not likely that the error is given for the same reason in this scenario. I made a request in my AppCenter support case to have the details forwarded to you. Thank you.

[gcarter-comcast]

Many snows ago I worked for Sybase and had many opportunities to speak directly with customers on their sites regarding what they needed from Sybase to develop their systems better. I have ever appreciated the comment made by one such customer, "Don't protect us from ourselves. After all, we might just know a little better than you how to best solve our problems."

Among professionals, I think the point should be well taken.

[karltaylor]

We've just hit the same issue here https://github.com/microsoft/appcenter/issues/591#issuecomment-537900882 and can't connect anymore apps.

Our infrastructure is based around white label apps that come from one single repository. According to @bperreault here https://github.com/microsoft/appcenter/issues/591#issuecomment-539980278 it doesn't look like configuring through GitHub will suffice.

I'd like to look into this solution here https://github.com/microsoft/appcenter/issues/591#issuecomment-506494848

Instead, the service you're using should be able to work with just one webhook defined on the repository since the events they get from that webhook are the same as the events they get from 20 webhooks they have now. An alternative would be that you build a lightweight middleman proxy and point one webhook to that proxy. The proxy would then forward webhook deliveries it receives to any number of destination URLs that you want (since you control the proxy).

Edit: It actually seems quite simple, the main repo can just have 1 webhook which is the proxy, and then that proxy can forward all the requests onto the api payloads. This could be an AWS Lambda or Google Cloud Function, but I'd have to keep all the webhook urls somewhere

[bperreault]

@karltaylor let me know if you find a solution that works with appcenter. We have had to go through the pain of setting up at another provider ( the one named above by @shawnvega ) And it does cost more and we did like appcenter better. I suppose now that we are moved over, we are staying as the cost in developer time adds up.

[dennispan]

@kranzky Unfortunately this won't work. The signature here is not generated with app specific secret. It's a combination of multiple information, which isn't supposed to be able to constructed by users, so that we avoid malicious use of the triggers.

[hytea]

@kranzky That is a great workaround. If I were a product manager at Microsoft / App Center I would be pretty embarrassed that you had to use a third party application to get a core CI feature to work with their platform.

[hytea]

Although, I do have to give @DennisPan credit; he has been a big help to our organization. He has helped us get some things sorted out that came up because of this issue. He is the redeeming factor in this whole situation.

[karltaylor]

I've got a bit stuck on the part where to build an App you need to have a Connected Repo otherwise you can't build, which makes sense.

But If I connected all the repo's I'm back to square 1 where I can't connect all of them. How can I get round this bit?

[karltaylor]

Thanks @kranzky

I've finally managed to get round to building the Lamda and now everything is working perfectly.

You do need to configure the App Center app to connect to Github, but after that you can just delete the webhook that it creates on your repository webhook settings.

Here's the repo for the workaround, which is weighted to my use case at the moment but will make it a bit more generic.

https://github.com/karltaylor/appcenter-github-proxy

[dionshihk]

Our team is experiencing exactly the same problem currently, since we cannot change the webhooks limit of GitHub, I'd like to share our solution with you guys.

We use our CI (Jenkins) to trigger app builds, instead of clicking "Build" button on AppCenter website.

In the CI pipeline, the core flow is to run a script including project creation (if not exist), repo connection, build configuration, triggering build and waiting for status change, then repo disconnection, step by step in one script. Every step can be done using AppCenter API: https://openapi.appcenter.ms/

We developed the script in TypeScript, after yarn install, simply run it by yarn ts-node ./path/to/build-script.ts project-name

The drawbacks are as below, but acceptable by our team:

• Build has to be triggered manually
• No build history retrieval, as the build history & settings are destroyed when disconnecting repo

[dionshihk]

To facilitate the build of such whitelabel apps, I made an open-source npm for this, with clear typing definition: https://www.npmjs.com/package/appcenter-auto-builder

The overall flow is like what I mentioned in the above reply. Here is an usage example:

import {startAppCenterBuilder} from "appcenter-auto-builder";
import {AppCenterBuilderConfiguration} from "appcenter-auto-builder/type";

const config: AppCenterBuilderConfiguration = {
    apiToken: "<Your API Token>",
    project: {
        name: "test-rn-ios",
        os: "iOS",
        platform: "React-Native",
        description: "My test app",
    },
    repo: {
        url: "https://github.com/dionshihk/some-ios-project",
    },
    owner: {
        type: "individual",
        name: "<Your Account Name>",
    },
    buildSetting: {
        trigger: "manual",
        artifactVersioning: {buildNumberFormat: "buildId"},
        environmentVariables: [
            {name: "TEST_ENV_1", value: "1"},
            {name: "TEST_ENV_2", value: Date.now().toString()},
        ],
        toolsets: {
            javascript: {
                packageJsonPath: "package.json",
                nodeVersion: "12.x",
            },
            xcode: {
                 ....
            },
        },
    },
    extraBuildEnvironmentVariables: [{name: "APP_SECRET", value: {type: "app-secret"}}],
    disconnectRepoOnFinish: true,
    buildEstDuration: 200,
};

startAppCenterBuilder(config);

Run this script in your CI, or even local machines (with Node environment), it will help you create and build your app step by step, with clear logs. By setting disconnectRepoOnFinish to true is the key of repo webhooks re-use.

[Martinedo]

any new plans on increasing the webhook count? the custom solutions provided by the people here are great, but its not exactly great to setup additional environments or servers.

[MinaZakaria]

Is there any news about supporting appcenter to multiple build triggers by one webhook?

[ghost]

This issue has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs within 15 days of this comment.

[ghost]

This issue will now be closed because it hasn't had any activity for 15 days after stale. Please feel free to open a new issue if you still have a question/issue or suggestion.

[KawaljeetCuelogic]

@shawnvega I am trying to connect about 10 apps using the same bitbucket repository with the FREE plan. Is it possible? While connecting the same repository to the 6th app, I got an error "Linking failed: Hit the platform limits for App Center.". Is it something to do with the FREE plan? Please reply.

[shawnvega]

@shawnvega I am trying to connect about 10 apps using the same bitbucket repository with the FREE plan. Is it possible? While connecting the same repository to the 6th app, I got an error "Linking failed: Hit the platform limits for App Center.". Is it something to do with the FREE plan? Please reply.

I do not know. I ended up switching to bitrise

[osamaqarem]

I wish the idea of using a single webhook for all apps from the same repo would get revisited.

I understand this can be alleviated by instead using the GitHub marketplace app, however that app forces on you Pull Request Build Checks which delay your deployments if you don't have a concurrent minutes plan. The marketplace app would be suitable if those checks could be disabled.

[ruscon]

Can App Center add an account configuration or app configuration to disable adding a new github webhook?

[AndersonVanzo]

@KawaljeetCuelogic did you find out any solution? I'm trying to do the same thing and got the same error

[KawaljeetRepublic]

@AndersonVanzo Sorry for the late reply. We had changed the architecture of the app and configured most of the things inside of a single application. For android, we went on to create product flavours and for iOS we created targets for multiple apps. However, that being said, we just had to connect a single repo.

I am not sure whether you need to deploy your app in different regions with different configurations, but if you have to, you can give a thought to it. All the best and happy coding!