Push-job server and client communication problem.

[laxmanrapolu]

No messages being received on command port in 4s. Possible encryption problem?

We have recently started using Chef in our organization. We have spinned up our own chef server in Azure(Chef in azure market place) and chef-clients in Azure VM's (with load balancer). Everything working fine and our client runs for every five minutes. Recently we wanted to implement chef-push-job server so that we can send command to run client immediately after we push our artifacts to azure blob(so we don't have to wait for 5 min for deployments to happen). I have installed push-job server(2.1.1) and push job client (2.1.1) successfully. But when ever I check for status it says unavailable and when I go and see logs, I have found above possible encryption error. Since the install client haven't revived any heart beat from server. I have also tried to open the ports on both ends, but still no luck. Any help would be really appreciated.

Push Jobs Client Version

2.1.1

Platform Version

push job server - Ubuntu 14.04, push job client - Windows 2012r2. Cook book version 3.2.2. Chef-client 12.17.44

Replication Case

Install push job server and push job client.

Stacktrace

[2016-12-12T19:42:08+00:00] INFO: Chef Push Jobs Client Service initialized [2016-12-12T19:42:12+00:00] INFO: Push Client version: 2.1.1 [2016-12-12T19:42:12+00:00] INFO: Push Client started as service with parameters: [] [2016-12-12T19:42:12+00:00] INFO: Push Client passed: -c, C:\chef/push-jobs-client.rb [2016-12-12T19:42:12+00:00] INFO: Push Client using default config file path: 'C:\chef/push-jobs-client.rb' [2016-12-12T19:42:14+00:00] INFO: [AzureMachine] using config file path: 'C:\chef/push-jobs-client.rb' [2016-12-12T19:42:25+00:00] INFO: [AzureMachine] Using node name: AzureMachine [2016-12-12T19:42:25+00:00] INFO: [AzureMachine] Using org name: myorg [2016-12-12T19:42:25+00:00] INFO: [AzureMachine] Using Chef server: https://mychefserver.azure.com/organizations/myorg [2016-12-12T19:42:25+00:00] INFO: [AzureMachine] Using private key: c:\chef\client.pem [2016-12-12T19:42:25+00:00] INFO: [AzureMachine] Incarnation ID: e8b3c954-725b-4f5c-b1f5-d4d403f1f817 [2016-12-12T19:42:25+00:00] INFO: [AzureMachine] Allowing fallback to unencrypted connection: false [2016-12-12T19:42:25+00:00] INFO: [AzureMachine] Starting client ... [2016-12-12T19:42:25+00:00] INFO: [AzureMachine] Retrieving configuration from https://mychefserver.azure.com/organizations/myorg//pushy/config/AzureMachine: ... [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Resolved Mychefserver.lhrscxud0tcelo5owsb0j0bggc.bx.internal.cloudapp.net to '' and -1 others [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Starting ZMQ version {:major=>4, :minor=>0, :patch=>6} [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Listening for server heartbeat at tcp://Mychefserver.lhrscxud0tcelo5owsb0j0bggc.bx.internal.cloudapp.net:10000 [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Connecting to command channel at tcp://Mychefserver.lhrscxud0tcelo5owsb0j0bggc.bx.internal.cloudapp.net:10002 [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Considering server online, and starting to heartbeat [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Started client. [2016-12-12T19:43:55+00:00] INFO: pushy-client is started... [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Starting heartbeat / offline detection thread on interval 10.0 ... [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Starting command / server heartbeat receive thread ... [2016-12-12T19:43:55+00:00] INFO: [AzureMachine] Starting reconfigure thread. Will reconfigure / reload keys after 3600 seconds, less up to splay 0.1. [2016-12-12T19:43:59+00:00] ERROR: [AzureMachine] No messages being received on command port in 4s. Possible encryption problem? [2016-12-12T19:43:59+00:00] INFO: [AzureMachine] Reconfiguring client / reloading keys ... [2016-12-12T19:43:59+00:00] INFO: [AzureMachine] Retrieving configuration from https://mychefserver.azure.com/organizations/myorg//pushy/config/AzureMachine: ... [2016-12-15T19:02:58+00:00] ERROR: [AZ1DFOFE0] No messages being received on command port in 3323s. Possible encryption problem? [2016-12-15T19:02:58+00:00] ERROR: [AZ1DFOFE0] Error in command / server heartbeat receive thread: can't create Thread: Not enough space c:/opscode/push-jobs-client/embedded/lib/ruby/gems/2.1.0/gems/opscode-pushy-client-2.1.1/lib/pushy_client.rb:144:in initialize' c:/opscode/push-jobs-client/embedded/lib/ruby/gems/2.1.0/gems/opscode-pushy-client-2.1.1/lib/pushy_client.rb:144:innew' c:/opscode/push-jobs-client/embedded/lib/ruby/gems/2.1.0/gems/opscode-pushy-client-2.1.1/lib/pushy_client.rb:144:in trigger_reconfigure' c:/opscode/push-jobs-client/embedded/lib/ruby/gems/2.1.0/gems/opscode-pushy-client-2.1.1/lib/pushy_client/protocol_handler.rb:294:inblock in start_receive_thread'

[thpert]

This bug is fixed with Push Job Client 2.1.4

[laxmanrapolu]

I am not sure about it. When I have first tried, I tried with 2.1.4 only. But anyhow now we are planning to give the deployment responsibility to Octopus deploy again. This way, for sure we will get to know when the deployment is done and run the subsequent steps.

Thanks Guys.

[iennae]

@laxmanrapolu it sounds like this may not be a problem for you anymore based on your use. If it still is please do reopen this issue. Thanks.