Nova compute installation failed on multi-node environment

[JianlanWang]

I met a strange problem. When I tried to install the component of nova-compute on more than 4 nodes, intermittenly some of nodes would reported one puppet error: "Error 400 on SERVER: Could not find class ...". Here is a log segment from /var/log/syslog for your reference:

Oct 24 09:33:11 puppetcilent2 puppet-agent[41721]: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find class nova_e::nova_compute::install for puppetcilent2.dodai.com on node puppetcilent2.dodai.com Oct 24 09:33:11 puppetcilent2 puppet-agent[41721]: Not using cache on failed catalog Oct 24 09:33:11 puppetcilent2 puppet-agent[41721]: Could not retrieve catalog; skipping run Oct 24 09:35:05 puppetcilent2 puppet-agent[42131]: Caching catalog for puppetcilent2.dodai.com Oct 24 09:35:05 puppetcilent2 puppet-agent[42131]: Applying configuration version '1351042243'

[guanxiaohua2k6]

Can you confirm whether there is file /etc/puppet/modules/nova_e/nova_compute/install.pp in dodai-deploy server?

[JianlanWang]

Sure. I can install nova-compute on 2 nodes successfully. But if the node number is more than 4 (including 4), this error would appear on some nodes (not all nodes.), which means some nodes installed nova-compute well, but some failed. It seems a Puppet defect.

[guanxiaohua2k6]

OK, thank you. I will confirm the problem.

[aimonb]

Hi, the error makes me think its a server issue. Look at your puppetmaster setup.. Are you Reverse Proxying it? Do workers match balancers? If just webrick.. do you have enough listeners? If you can share some of the puppet-master setup I may be able to lend more directed help...

[JianlanWang]

What kind of info on puppet master do you need?

[guanxiaohua2k6]

I am trying to repeat the error in my environment. If I need some info later, I will ask you. Thanks in advance.

[aimonb]

Info about the web server.. Which you are using etc.. If u can provide.

[JianlanWang]

I just set up web server using setup.sh in dodai-deploy. No change on it. So it only used Webrick? Please tell me how I can get the info if you need. Thanks :)

[guanxiaohua2k6]

Yes, your puppet server is using the default web server webrick. And if you want to use other web server, you can refer the page http://projects.puppetlabs.com/projects/puppet/wiki/Using_Mongrel_On_Debian.

BTW, I tried in my environment, but couldn't repeat the error.

[JianlanWang]

All right. Maybe I need more tests. From google, I found someone got the same Puppet error, but still didn't get the root cause and resolution.

[aimonb]

In general webrick can only handle one connection at a time.. most likely there is more to it.. I'll take a look at the script and see if I can lend a hand..

[JianlanWang]

I'm planning to use other web server for puppet master. If any update, I'll let you know.Thanks a lot. BTW, what's your plan for OpenStack Folsom release?

[guanxiaohua2k6]

I've added support for folsom (except swift), you can see the options in the "new proposal" page if you updated dodai-deploy server. But tests of it are not enough.

[aimonb]

We successfully use Apache w/ Mod_Passenger as a web server for puppet. It seems to scale well. If you are on Ubuntu the package setup is VERY easy.

[habuka036]

aimon, thank you for your advice. :)

[JianlanWang]

Hi Xiaohua, here's great news to let you know that I replaced webrick with mongrel + Apache2 to resolve this issue. Thanks so much. I think you can close the issue.

[guanxiaohua2k6]

Thank you for your news.