Support private submodules (github)

[AlexSenchenko]

Hi, let me know please how to deploy repository containing private submodules like this

"name":   
  "version": "0.0.1",
  "private": true,
  "dependencies": {
    "bluebird": "^2.10.2",
    "classnames": "^2.1.3",
    "efu-core": "git+https://longToken:x-oauth-basic@github.com:OrgName/RepositoryName.git",
    "express": "^4.13.3",
    "jquery": "^2.1.4",
    "jquery-mousewheel": "^3.1.13",
    "lodash": "^3.10.1",
    "malihu-custom-scrollbar-plugin": "^3.1.0",
    "node-uuid": "^1.4.3",
    "path-to-regexp": "^1.2.1",
    "react": "^15.0.2",
    "react-dom": "^15.0.2",
    "slick-carousel": "1.5.9",
    "validator": "^4.2.1"
  },
  "devDependencies": {

[davidebbo]

Same as here. Best guidance is @suwatch's post: http://blogs.suwatbodin.com/2013/07/deploy-to-windows-azure-using-git-with.html

[AlexSenchenko]

Would changes here in near future?

[davidebbo]

Before we talk about changes, did you try going through the guidance in the post?

[AlexSenchenko]

Yes, I follow steps and stopped here:

Simply copy the deployment key (d:\home.ssh\id_rsa.pub) and paste to this user as user key. Because GitHub only allows one unique key, you may have to delete the deployment key from repo before add as a user key.

So, it means app service FTP? And where needed to be added user key?

[AlexSenchenko]

Is this way it will work or not?

[davidebbo]

@suwatch maybe tweak your blog post to make the points above more explicit?

[suwatch]

I have updated the blog with link instruction how to copy SSH key.

Simply copy the deployment key (d:\home.ssh\id_rsa.pub) from your webapp (using FTP or Kudu Console) and paste to this user as user SSH key.

BTW, I admitted that I only experimented with submodule (.gitmodules). In your case, it is npm packages to private git repo and I think this should work the same. Let us know one way or another.

[ghost]

Dear @suwatch and @davidebbo I've create integration account at GitHub. Added it to organization account. When setup deployment source - azure don't see organization repositories. Why? (From github I can easy fork) As a workaround - I've created fork and setup deployment from it.

Follow by you article - after copy publis id_rsa.pub to github account ssh, preliminary - delete deployment key from forked repo. I used following schema - "packageName": "git+ssh://git@github.com:<Organization>/<Repo>.git", And got: ERROR: Repository not found.\r\nnpm ERR! fatal: Could not read from remote repository.\r\nnpm ERR! \r\nnpm ERR! Please make sure you have the correct access rights\r\nnpm ERR! and the repository exists.\r\nnpm ERR! \r\nnpm ERR! \r\nnpm ERR! If you need help, you may report this error at:\r\nnpm ERR! <https://github.com/npm/npm/issues>\r\n\r\nnpm ERR! Please include the following file with any support request:\r\nnpm ERR! D:\home\site\wwwroot\npm-debug.log\r\nC:\Program Files (x86)\SiteExtensions\Kudu\55.50526.2249\bin\Scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd"

[davidebbo]

@eyefitu-integration see this page to investigate why you don't see the repos listed.

For the second issue, to isolate from anything npm related, as a test, try just cloning the repo in Kudu console using the ssh url. Does that work?

[AlexSenchenko]

Thank for link - will try.

@davidebbo you mean cloning repo?

[davidebbo]

Yes, sorry, typo (corrected). And you can do this in a test folder. e.g. d:\home\TestRepo. This will tell us whether the issue is npm specific, or more generally a problem with ssh key.

[ghost]

When I granted owner permission to integration account - I can to deploy via such url git+ssh://git@github.com:<Organization>/<Repo>.git with tip: (copy/paste id_pub.rsa and delete deployment key).

Will continue to play with it.

Have some questions now -

1. how to automate/scale such solution?
2. Why I need to give integration account owner permission, is it GitHub restriction?
3. Why it's not working via HTTPS OAuth url like here http://stackoverflow.com/questions/28728665/how-to-use-private-github-repo-as-npm-dependency

I think it's pretty straightforward case to use npm dependencies as private github repos.

Also FYI, some outdated info: at Github due this https://azure.microsoft.com/en-us/blog/using-app-service-web-apps-continuous-deployment-with-github-organizations/ - they renamed menu item Applications -> OAuth Applications

[suwatch]

Will try to answer one by one

how to automate/scale such solution?

You could use the same SSH key for all sites. On Github, you have to manually setup the user SSH key once. Then, for each sites, you can propagate the SSH private key across using /api/sshkey.

Why I need to give integration account owner permission, is it GitHub restriction?

For set up the continuous deployment (say thru portal), the account need admin/owner permission since it needs to set up hook and deployment key with Github. If you do the key manually (like you did above), the account will only need READ permission since it only performs git fetch operation.

Why it's not working via HTTPS OAuth url like here http://stackoverflow.com/questions/28728665/how-to-use-private-github-repo-as-npm-dependency

If you generate the OAuth token and hardcode it with packages json file, it should just work although it may not be wise in term of security (persist and check in security token to a file).

[ghost]

Thanks for answer.

@suwatch can you please sample of using [/api/sshkey]. (https://github.com/projectkudu/kudu/wiki/REST-API#sshkey) to setup deployment across several sites without using portal?

Don't want to have an integration account with full permissions.

Another one question, does kudu supports private npm registry?

[suwatch]

@suwatch can you please sample of using [/api/sshkey]. (https://github.com/projectkudu/kudu/wiki/REST-API#sshkey) to setup deployment across several sites without using portal?

You can find the instruction here.

Another one question, does kudu supports private npm registry?

Does private npm registry mean only reachable from specific machine? If so, then it may not be possible since the npm install step happens on (and must be accessible from) Azure WebApp VM.

[wilsonson]

I've tried to use some submodules from private repro with HTTP connection, but it's stucked at "git credential-manager get" and looks like waiting for input. Is it a expected behavior ?

[suwatch]

No, it should just fail fast. We added this code to avoid cred hanging issue. Could you check if the repo (use Kudu Debug Console) to see if git cred hook exists?

[wilsonson]

Yes, the hook "git-credential-invalid.sh" exists and the content is the same as the code you mentioned.

[rasmus91]

Looking at Suwat Bodin's site I get an idea of how to get the SSH keys. but even with the find -name '.ssh' I can not locate a file with an SSH key. Any idea on how i can proceed?

I have a solution that i am trying to deploy. It consists currently of two projects. Everything is in private repositories.

[jvano]

Hi

If the problem persists and is related to running it on Azure App Service, please open a support incident in Azure: https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

This way we can better track and assist you on this case

Thanks,

Joaquin Vano Azure App Service