microsoft / oxa-tools

Open edX on Azure Tools
MIT License
36 stars 103 forks source link

After successful execution of Deploy Script, we cannot receive the provisioning related mails like "OXA Bootstrap – [Cluster Name]" #335

Closed egonid closed 6 years ago

egonid commented 6 years ago

Hello

Today, I tried to deploy with oxa-tools (oxa/master.fic) several times.

Every times, The Deploy script was successful, but I could not received the provisioning status mail like "OXA Bootstrap – [Cluster Name]"

Just only I received mail like "Cron root@XXXvmss000001 [ ( ! -f /etc/opt/omi/creds/omi.keytab ) -o "

I waited for more than 3 hours, I still cannot receive the bootstrap mail.

How can I resolve this?

In addition, the script and provisioning was successful in a few days ago (last week)

Thanks in advance.

egonid commented 6 years ago

I have researched the bootstrap.csx.log. I found GPG error like below.

05/23/18 18:55:01 :: update package(s)... W: GPG error: https://apt-mo.trafficmanager.net/repos/azure-cli wheezy InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF W: The repository 'https://apt-mo.trafficmanager.net/repos/azure-cli wheezy InRelease' is not signed.

So I go to the https://apt-mo.trafficmanager.net/repos/azure-cli . The update date is 22-May-2018 17:28 .

I doubt that the update date of azure-cli repository have some clues for this issue.

So I run the command like below in jbVM.

lexoxaadmin@DMW09jb:/var/log$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EB3E94ADBE1229CF

Executing: /tmp/tmp.6UmSyMUpoJ/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys EB3E94ADBE1229CF gpg: requesting key BE1229CF from hkp server keyserver.ubuntu.com gpg: key BE1229CF: public key "Microsoft (Release signing) gpgsecurity@microsoft.com" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1)

lexoxaadmin@DMW09jb:/var/log$ sudo apt-get update Hit:1 http://azure.archive.ubuntu.com/ubuntu xenial InRelease Get:2 http://azure.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB] Get:3 http://azure.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB] Ign:4 http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.2 InRelease Hit:5 http://security.ubuntu.com/ubuntu xenial-security InRelease Hit:6 http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.2 Release Get:8 https://apt-mo.trafficmanager.net/repos/azure-cli wheezy InRelease [2,808 B] Fetched 219 kB in 1s (194 kB/s) Reading package lists... Done

after execution of those command .

I can receive the bootstrap mail except VMSS related mail.

and the lms site was not opend.

Did you see this article? https://github.com/dotnet/cli/issues/2189

How do you think about this?

eltoncarr commented 6 years ago

I've had one other report of this. I will start an investigation hopefully later today or tomorrow and provide a feedback.

egonid commented 6 years ago

Hi Elton

You can see that the signing key update(May 2018) warning in below link.

https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-apt?view=azure-cli-latest

I think that maybe the new key is the solution for this issue.

Thanks in advance.

kevmax1 commented 6 years ago

Hi Egonid I use azur-cli on windows and I have this deployment problem since yesterday, I have the same errors as you.

kevmax1 commented 6 years ago

I solve problem by adding new function in /oxa-tools/templates/stamp/utilities.sh

add-key-adv()
{
    sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EB3E94ADBE1229CF
    sudo apt-get update
}

and i call this function inside install-tools() function in the same file (utilities.sh) i also change oxaToolsGithubProjectName param inside parameters.json to specifie new github repo. once all this is done I restart the installation and now it's work

EliasPI commented 6 years ago

What is the new github repo? isn't it the same "oxa-tools" ?

egonid commented 6 years ago

EliasPI

You have to fork https://github.com/Microsoft/oxa-tools to your own GitHub account. after then, you can see the oxa-tools repository in your own github account which is replicated from Microsoft/oxa-tools. Maybe, It's url is EliasPi/oxa-tools.

After fork, you can modify the utility.sh in your own forked repository(EliasPi/oxa-tools)

And then, you have to modify local(in your PC) parameter.json file.

In Parameter.json

oxaToolsGithubAccountName": { "value": "Microsoft" -> EliasPI(your GitHub account name) }, "oxaToolsGithubProjectName": { "value": "oxa-tools" -> oxa-tools(your GitHub Project name) }, "oxaToolsGithubBranch": { "value": "oxa/master.fic" -> oxa/master.fic (your GitHub branch name)

Thanks

eltoncarr commented 6 years ago

We are synchronizing a fix for this. I put forth an initial pass in PR#338 to address this issue.

egonid commented 6 years ago

Hello Elton

How long it takes time that the PR#338 is applied to oxa/master.fic?

Thanks.

eltoncarr commented 6 years ago

338 is already merged into our dev.fic branch. There was another we wanted to consider as part of promoting community contribution (#337). But due to delays, we will proceed with #338 as a fix to this and propagate it. I will start that process today and hopefully complete within 24 hrs or thereabouts depending on validation.

EliasPI commented 6 years ago

I deployed yesterday and I still get the same emails. ( I used the oxa.master branch)

eltoncarr commented 6 years ago

You may want to double check because I have validated this scenario as resolved.

shadowdancerx3 commented 6 years ago

We finally manage to successfully delploy EDX, but we also receiving the same error.

After successful provisioning we first receive 5 emails:

Cron root@XXXX000001 [ ( ! -f /etc/opt/omi/creds/omi.keytab ) -o ( /etc/krb5.keytab -nt /etc/opt/omi/creds/omi.keytab ) ] && /opt/omi/bin/support/ktstrip /etc/krb5.keytab /etc/opt/omi/creds/omi.keytab

src kt = /etc/krb5.keytab File /etc/krb5.keytab does not exist /opt/omi/bin/support/ktstrip: 38: exit: Illegal number: -1

and after that we manage to receive 5 out of 5 emails for EDX deployment. We are also using oxa.master.fic branch. Is deployment successful or not? We are able to login to jb, and to cms and lms.

eltoncarr commented 6 years ago

Did you continue to get those email? Regarding status of deployment, you should get a final notification email indication completion of bootstrap process. Also, your TM endpoints will come online and your site will be accessible.

amoswachanga commented 4 years ago

Please advise if this issue is related to #476