Deploy Block commands getting Error

[mattjay]

Thought I was having an issue with the kind of Docker deploy I was trying to do with connecting to my private registry. Got frustrated and during debugging tried to just run any command.

As you can see, even a basic ls command gets an error while its in the Custom Scripts -> Deploy block.

[jaredly]

Wow. What's your setup? version of strider, version of node, OS?

[mattjay]

Ubuntu 12.04 Node 10.5 Strider 1.2.18

[mattjay]

Hmm noticing https://github.com/Strider-CD/strider/blob/master/ROADMAP.md looks like we are on a pretty old version. Weird because I just npm installed it 3 days ago and before that was using the docker registry.

[jaredly]

;) yeah the latest version has a ton of awesome features. It's also good idea to back up your db before upgrading this one time --- 1.4 introduced some major DB schema changes. It should be fine, but it's good to be safe.

[mattjay]

1.4.3 sorry. had wrong output on npm strider -v first time.

[mattjay]

Specifically: ─┬ strider@1.4.3 ├─┬ strider-bitbucket@0.1.0 ├─┬ strider-custom@0.5.0 ├── strider-env@0.4.4 ├─┬ strider-extension-loader@0.4.1 ├─┬ strider-git@0.1.0 ├─┬ strider-github@0.1.2 ├─┬ strider-heroku@0.1.0 ├─┬ strider-node@0.3.3 │ └─┬ strider-detection-rules@0.0.1 ├─┬ strider-python@0.2.0 │ └─┬ strider-detection-rules@0.0.1 ├─┬ strider-sauce@0.6.1 ├─┬ strider-simple-runner@0.11.2 │ └─┬ strider-runner-core@0.1.2 ├─┬ strider-webhooks@0.1.0

[jaredly]

ahhm. does the error only happen in the deploy block?

[mattjay]

yup. workaround right now is scripting it all in the test block:

docker build -t localhost:$PORT/$REPO .
container=$(docker run -d -e NODE_ENV=test -p 5555:8080 -dns $DNS_IP localhost:$PORT/$REPO)
sleep 5
jasmine-node $TESTS.spec.js --captureExceptions
result=$?
docker stop $container

if [$result -eq 1]; then
    exit $result
else 
    docker push localhost:$PORT/$REPO
    $result = $?
    if [$result -eq 1]; then
        exit $result
    else
        exit 0
    fi
fi    

Not the most elegant I'm aware but it's what we ended up with after a few days of banging our heads. This appears to work at the moment. Anything in deploy just fails.

[jaredly]

ok. I just did a test and it worked fine for me -- is it this way with all of your projects? What plugins do you have enabled?

[ProZachJ]

This issue occurs when NODE_ENV=production. Not when NODE_ENV=development.

I was lying. This is now occurring with development too.

20 Dec 00:03:32 - info: Job 52b38941191568dc2200000d domain error caught connect ECONNREFUSED Error: connect ECONNREFUSED at errnoException (net.js:901:11) at Object.afterConnect as oncomplete 20 Dec 00:03:32 - error: Job 52b38941191568dc2200000d connect ECONNREFUSED Error: connect ECONNREFUSED at errnoException (net.js:901:11) at Object.afterConnect as oncomplete

[ProZachJ]

Ok I chased this down and it looks like we had a web-hook failing causing this. Disabled the web-hooks and it appears to be working.

[jaredly]

good to know! Do we need to wrap the webhook fire call in a try/catch? https://github.com/Strider-CD/strider-webhooks/blob/master/webapp.js#L17

[ProZachJ]

I'd vote yes. It would be nice for the build not to fail just because my web-hook listener isn't up.

[jaredly]

:) right; could you confirm that's the right place to catch though?

[ProZachJ]

Yea, I'll take a look at it. Right now I'm just happy I finally got an auto-deploy to my registry ;)

[ProZachJ]

What about just making this more robust to handle connection errors?

https://github.com/Strider-CD/strider-webhooks/blob/9faf28edc3e8bb65bd7cdc89488686fb2d0ec094/lib/utils.js#L25

[jaredly]

I think req.end is throwing an error synchronously, which is why it breaks everything. I've seen this happen before

[ProZachJ]

I've never used superagent, does request.post(url) return something that you could inspect to know if the connection was successful before calling req.body or req.end?

[jaredly]

nope. req.end is when the request is sent off, and there are some errors that are apparently blocking. I don't think it's a superagent-specific thing. It just uses a normal http request http://nodejs.org/api/http.html

[ProZachJ]

This appears to have fixed it

req.end(function (err, req) {
    if(err){
      console.error('failed to fire webhook', err);
    }
    else if (req.status >= 300 || req.status < 100) {
      console.error('failed to fire webhook', req.status, req.text, url, secret, payload)
    }
  })

[jaredly]

Oh awesome. Care to PR? On Dec 19, 2013 7:02 PM, "ProZachJ" notifications@github.com wrote:

This appears to have fixed it

req.end(function (err, req) { if(err){ console.error('failed to fire webhook', err); } else if (req.status >= 300 || req.status < 100) { console.error('failed to fire webhook', req.status, req.text, url, secret, payload) } })

— Reply to this email directly or view it on GitHubhttps://github.com/Strider-CD/strider/issues/319#issuecomment-30983904 .

[niallo]

Fixed in https://github.com/Strider-CD/strider-webhooks/pull/1

[mattjay]

Thanks guys. This was killing us, glad we chased it down.