search

apache / openwhisk

Apache OpenWhisk is an open source serverless cloud platform
https://openwhisk.apache.org/
Apache License 2.0
6.52k stars 1.17k forks source link

Unable to create trigger #2221

Closed Katiyman closed 7 years ago

Katiyman commented 7 years ago

Environment details:

Steps to reproduce the issue:

  1. try to create trigger using the below command
  2. wsk -i trigger create every-20-seconds --feed /whisk.system/alarms/alarm --param cron '/10 ' --param maxTriggers 6

Provide the expected results and outputs:

Trigger should have got created successfully

Provide the actual results and outputs:

error: Unable to create trigger 'every-20-seconds': Unable to invoke trigger 'every-20-seconds' feed action '/whisk.system/alarms/alarm'; feed is not configured: Unable to invoke action 'alarms/alarm': The supplied authentication is not authorized to access this resource. (code 22040)

Additional information you deem important:

Katiyman commented 7 years ago

Output of wsk -i package list /whisk.system is packages /whisk.system/watson-translator shared /whisk.system/github shared /whisk.system/weather shared /whisk.system/watson-speechToText shared /whisk.system/websocket shared /whisk.system/utils shared /whisk.system/watson-textToSpeech shared /whisk.system/combinators shared /whisk.system/samples shared /whisk.system/slack shared

markusthoemmes commented 7 years ago

You haven't installed the alarms package. For more information on the package, see the respective repo: https://github.com/apache/incubator-openwhisk-package-alarms. See https://github.com/apache/incubator-openwhisk-package-alarms/issues/51 for information on how to get it to run locally.

Katiyman commented 7 years ago

@markusthoemmes Upon following the above links I am able to install the alarm package but still when I run the below command I get wsk -i rule create \

invoke-periodically \
every-20-seconds \
handler

Error : error: Unable to create rule 'invoke-periodically': trigger /guest/every-20-seconds does not exist (code 22747) why is it reffering to guests still when I have created the trigger in /whisk.system/alarms/alarm. also in the list I can see the alarms package created packages /whisk.system/alarms shared /whisk.system/watson-translator shared /whisk.system/github shared /whisk.system/weather shared /whisk.system/watson-speechToText shared /whisk.system/websocket shared /whisk.system/utils shared /whisk.system/watson-textToSpeech shared /whisk.system/combinators shared /whisk.system/samples shared /whisk.system/slack shared

Katiyman commented 7 years ago

One more info upon trigger creation the output which is coming is

wsk -i trigger create every-20-seconds \

--feed  /whisk.system/alarms/alarm \
--param cron '*/20 * * * * *' \
--param maxTriggers 6

ok: invoked /whisk.system/alarms/alarm, but the request has not yet finished, with id 23d80bf91ec7458183b419a73d5a3e76 ok: invoked /whisk.system/alarms/alarm, but the request has not yet finished, with id 1b05a3a9664d4713b296dc03733666f6

Katiyman commented 7 years ago

Also following is the error noted wsk -i activation result 23d80bf91ec7458183b419a73d5a3e76 { "error": "The action exceeded its time limits of 60000 milliseconds." }

jasonpet commented 7 years ago

Does your feed action reach the alarms provider? Do you see any messages in the alarms provider logs after the message "resetting system from last state" after you attempt to create your trigger?

Katiyman commented 7 years ago

no I don't see any log after that. @jasonpet you meam to say that if action was able to reach the alarms provider it will generate some logs in the container?

Katiyman commented 7 years ago

Lemme also put the commads used to install the alarms export AUTH_KEY=cat ~/openwhisk/ansible/files/auth.whisk.system export COUCHDB_HOST=awk -F "=" '/db_host/ {print $2}' ~/openwhisk/ansible/db_local.ini export COUCHDB_PORT=awk -F "=" '/db_port/ {print $2}' ~/openwhisk/ansible/db_local.ini export API_HOST=${COUCHDB_HOST} export API_PORT=11001 ./installCatalog.sh $AUTH_KEY $API_HOST $API_HOST $API_PORT ./gradlew :distDocker mkdir -p /home//wsklogs/alarmsTrigger

for running the docker

export COUCHDB_PROTOCOL=awk -F "=" '/db_protocol/ {print $2}' ~/openwhisk/ansible/db_local.ini export COUCHDB_HOST=awk -F "=" '/db_host/ {print $2}' ~/openwhisk/ansible/db_local.ini export COUCHDB_PORT=awk -F "=" '/db_port/ {print $2}' ~/openwhisk/ansible/db_local.ini export COUCHDB_USER=awk -F "=" '/db_username/ {print $2}' ~/openwhisk/ansible/db_local.ini export COUCHDB_PWD=awk -F "=" '/db_password/ {print $2}' ~/openwhisk/ansible/db_local.ini sudo docker run -it -p 11001:8080 -v /home//wsklogs/alarmsTrigger:/logs \ -e PORT=8080 -e ROUTER_HOST=172.17.0.1 \ -e DB_PREFIX=whisk_alarms -e DB_USERNAME=${COUCHDB_USER} \ -e DB_PASSWORD=${COUCHDB_PWD} -e DB_HOST=${COUCHDB_HOST}:${COUCHDB_PORT} \ -e DB_PROTOCOL=${COUCHDB_PROTOCOL} whisk/catalog_alarms

is there any issue with the above

jasonpet commented 7 years ago

Yeah, you would see more log messages in the container if the feed action was able to reach the alarms provider. Something in your install steps probably does not have the right value. I know these steps have worked for others, but without knowing the value in these environment variables it is hard for me to tell what could be wrong. I have created a commit on one of my openwhisk branches that will use ansible to install the alarms feed action and start the alarms provider for you. Can you please cherry pick my commit below and then run the providers.yml playbook I created? You would run it just as you run openwhisk.yml. You need to make sure openwhisk is deployed when you run it.

https://github.com/jasonpet/openwhisk/commit/db0202c8a975a91efc77b03d0b5de8609088e9bc

Katiyman commented 7 years ago

@jasonpet thanks for the help Jason.. I coudnt cherry pick your code due to some git(don't wanna debug that now) so I took your code manually and ran the providers.yml with the command below sudo ansible-playbook -i environments/local providers.yml

but I am getting still the same error "error": "The action exceeded its time limits of 60000 milliseconds."

Just wanna say that that the DB connection should not be the problem since I am able to see the DB getting created.. Can you help me in which direction I should debug..

Katiyman commented 7 years ago

Activation log poll output 

Activation: alarm (089710acfa8b4bb4aba82c6351723147)
[
    "2017-05-11T06:28:58.170457688Z stdout: alarm:  { maxTriggers: 6,",
    "2017-05-11T06:28:58.170531236Z stdout: authKey: '23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP',",
    "2017-05-11T06:28:58.170536675Z stdout: trigger_payload: '',",
    "2017-05-11T06:28:58.170538691Z stdout: cron: '*/20 * * * * *',",
    "2017-05-11T06:28:58.170540761Z stdout: package_endpoint: '172.17.0.1:11001',",
    "2017-05-11T06:28:58.170542829Z stdout: lifecycleEvent: 'CREATE',",
    "2017-05-11T06:28:58.170545621Z stdout: triggerName: '/_/every-20-seconds' }"
]

Activation: alarm (d79e0801a3b84f1a8d7c8bcd6723d253)
[
    "2017-05-11T06:29:57.609733862Z stdout: alarm:  { maxTriggers: 6,",
    "2017-05-11T06:29:57.609768104Z stdout: authKey: '23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP',",
    "2017-05-11T06:29:57.609772625Z stdout: trigger_payload: '',",
    "2017-05-11T06:29:57.609774929Z stdout: cron: '*/20 * * * * *',",
    "2017-05-11T06:29:57.609782847Z stdout: package_endpoint: '172.17.0.1:11001',",
    "2017-05-11T06:29:57.609784997Z stdout: lifecycleEvent: 'DELETE',",
    "2017-05-11T06:29:57.609787088Z stdout: triggerName: '/_/every-20-seconds' }"
]
Katiyman commented 7 years ago

log of controller

[2017-05-11T06:28:56.923Z] [INFO] [#tid_34760] PUT /api/v1/namespaces/_/triggers/every-20-seconds overwrite=false
[2017-05-11T06:28:56.923Z] [INFO] [#tid_34760] [RestAPIVersion_v1] authenticate: 23bc46b1-71f6-4ed5-8c54-816aa4f8c502
[2017-05-11T06:28:56.924Z] [INFO] [#tid_34760] [Identity] [GET] serving from cache: 23bc46b1-71f6-4ed5-8c54-816aa4f8c502 [marker:database_cacheHit_count:0]
[2017-05-11T06:28:56.924Z] [INFO] [#tid_34760] [RestAPIVersion_v1] authentication valid
[2017-05-11T06:28:56.924Z] [INFO] [#tid_34760] [LocalEntitlementProvider] checking user 'guest' has privilege 'PUT' for 'triggers/guest/every-20-seconds'
[2017-05-11T06:28:56.924Z] [INFO] [#tid_34760] [LocalEntitlementProvider] authorized
[2017-05-11T06:28:56.924Z] [INFO] [#tid_34760] [WhiskTrigger] [GET] serving from datastore: id: guest/every-20-seconds [marker:database_cacheMiss_count:1]
[2017-05-11T06:28:56.925Z] [INFO] [#tid_34760] [CouchDbRestStore] [GET] 'root_manu-150_whisks' finding document: 'id: guest/every-20-seconds' [marker:database_getDocument_start:1]
[2017-05-11T06:28:56.937Z] [INFO] [#tid_34760] [CouchDbRestStore] [GET] 'root_manu-150_whisks', document: 'id: guest/every-20-seconds'; not found. [marker:database_getDocument_finish:13:12]
[2017-05-11T06:28:56.937Z] [INFO] [#tid_34760] [WhiskTrigger] invalidating id: guest/every-20-seconds
[2017-05-11T06:28:56.937Z] [INFO] [#tid_34760] [TriggersApi] [PUT] entity does not exist, will try to create it
[2017-05-11T06:28:56.937Z] [INFO] [#tid_34760] [TriggersApi] [PUT] entity created/updated, writing back to datastore
[2017-05-11T06:28:56.937Z] [INFO] [#tid_34760] [WhiskTrigger] write initiated on new cache entry
[2017-05-11T06:28:56.937Z] [INFO] [#tid_34760] [CouchDbRestStore] [PUT] 'root_manu-150_whisks' saving document: 'id: guest/every-20-seconds, rev: null' [marker:database_saveDocument_start:14]
[2017-05-11T06:28:56.990Z] [INFO] [#tid_34760] [CouchDbRestStore] [PUT] 'root_manu-150_whisks' completed document: 'id: guest/every-20-seconds, rev: null', response: '{"ok":true,"id":"guest/every-20-seconds","rev":"1-6f53ca26bd8527740843c143d6fba3ec"}' [marker:database_saveDocument_finish:67:53]
[2017-05-11T06:28:56.990Z] [INFO] [#tid_34760] [WhiskTrigger] write all done, caching id: guest/every-20-seconds Cached
[2017-05-11T06:28:56.990Z] [INFO] [#tid_34760] [TriggersApi] [PUT] entity success
[2017-05-11T06:28:56.990Z] [INFO] [#tid_34760] [BasicHttpService] [marker:http_put.200_count:67:67]
[2017-05-11T06:28:56.994Z] [INFO] [#tid_34761] POST /api/v1/namespaces/whisk.system/actions/alarms/alarm blocking=true&result=false
[2017-05-11T06:28:56.995Z] [INFO] [#tid_34761] [RestAPIVersion_v1] authenticate: 23bc46b1-71f6-4ed5-8c54-816aa4f8c502
[2017-05-11T06:28:56.995Z] [INFO] [#tid_34761] [Identity] [GET] serving from cache: 23bc46b1-71f6-4ed5-8c54-816aa4f8c502 [marker:database_cacheHit_count:1]
[2017-05-11T06:28:56.995Z] [INFO] [#tid_34761] [RestAPIVersion_v1] authentication valid
[2017-05-11T06:28:56.996Z] [INFO] [#tid_34761] [LocalEntitlementProvider] checking user 'guest' has privilege 'READ' for 'packages/whisk.system/alarms'
[2017-05-11T06:28:56.996Z] [INFO] [#tid_34761] [WhiskPackage] [GET] serving from datastore: id: whisk.system/alarms [marker:database_cacheMiss_count:2]
[2017-05-11T06:28:56.996Z] [INFO] [#tid_34761] [CouchDbRestStore] [GET] 'root_manu-150_whisks' finding document: 'id: whisk.system/alarms' [marker:database_getDocument_start:2]
[2017-05-11T06:28:57.041Z] [INFO] [#tid_34761] [CouchDbRestStore] [GET] 'root_manu-150_whisks' completed: found document 'id: whisk.system/alarms' [marker:database_getDocument_finish:44:41]
[2017-05-11T06:28:57.041Z] [INFO] [#tid_34761] [PackageCollection] entitlement check on package, 'READ' allowed?: true
[2017-05-11T06:28:57.041Z] [INFO] [#tid_34761] [LocalEntitlementProvider] authorized
[2017-05-11T06:28:57.041Z] [INFO] [#tid_34761] [WhiskPackage] [GET] serving from cache: id: whisk.system/alarms [marker:database_cacheHit_count:46]
[2017-05-11T06:28:57.041Z] [INFO] [#tid_34761] [ActionsApi] [GET] entity success
[2017-05-11T06:28:57.041Z] [INFO] [#tid_34761] [ActionsApi] merged package parameters and rebased action to 'whisk.system/alarms
[2017-05-11T06:28:57.041Z] [INFO] [#tid_34761] [WhiskAction] [GET] serving from datastore: id: whisk.system/alarms/alarm [marker:database_cacheMiss_count:46]
[2017-05-11T06:28:57.041Z] [INFO] [#tid_34761] [CouchDbRestStore] [GET] 'root_manu-150_whisks' finding document: 'id: whisk.system/alarms/alarm' [marker:database_getDocument_start:46]
[2017-05-11T06:28:57.048Z] [INFO] [#tid_34761] [CouchDbRestStore] [GET] 'root_manu-150_whisks' completed: found document 'id: whisk.system/alarms/alarm' [marker:database_getDocument_finish:54:8]
[2017-05-11T06:28:57.049Z] [INFO] [#tid_34761] [ActionsApi] [GET] entity success
[2017-05-11T06:28:57.049Z] [INFO] [#tid_34761] [ActionsApi] [marker:controller_blockingActivation_start:55]
[2017-05-11T06:28:57.049Z] [INFO] [#tid_34761] [ActionsApi] [POST] action activation id: 089710acfa8b4bb4aba82c6351723147 [marker:controller_loadbalancer_start:55]
[2017-05-11T06:28:57.049Z] [INFO] [#tid_34761] [LoadBalancerService] [marker:controller_kafka_start:55]
[2017-05-11T06:28:57.050Z] [INFO] [#tid_34761] [LoadBalancerService] posting topic 'invoker0' with activation id '089710acfa8b4bb4aba82c6351723147'
[2017-05-11T06:28:57.060Z] [INFO] [#tid_34761] [LoadBalancerService] Posted to invoker0[0][34] [marker:controller_kafka_finish:66:11]
[2017-05-11T06:28:57.060Z] [INFO] [#tid_34761] [ActionsApi] [marker:controller_loadbalancer_finish:66:11]
Katiyman commented 7 years ago

Except the starting logs no logs generated in the alarmsTrigger log on creation on trigger

markusthoemmes commented 7 years ago 
"2017-05-11T06:29:57.609782847Z stdout: package_endpoint: '172.17.0.1:11001',",

That's your problem. The 172.17.0.1 needs to be the IP of your VM.

Edit: Ignore this, it's nonsense.

Katiyman commented 7 years ago

172.17.0.1 is my api host is that also wrong and how to change this package_endpoint property

output of wsk -i property get whisk auth 23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP whisk API host 172.17.0.1 whisk API version v1 whisk namespace _ whisk CLI version 2017-04-27T13:54:41+00:00 whisk API build 2017-05-05T09:12:39Z whisk API build number latest

@jasonpet is it related to ROUTER_HOST, it is coming from edge host I feel whose value is 172.17.0.1 so is It correct to changed to local vm ip addr 10.71.11.150.. why I am asking this is I want to ascertain wat more impacts can be there because this would have been the value in the whisk.property for whole of my installation. I am also asking this ques to understand the config and working more.

Katiyman commented 7 years ago

whisk.property: host list

consulserver.host=172.17.0.1 controller.host=172.17.0.1 edge.host=172.17.0.1 kafka.host=172.17.0.1 loadbalancer.host=172.17.0.1 router.host=172.17.0.1 zookeeper.host=172.17.0.1 invoker.hosts=172.17.0.1

Katiyman commented 7 years ago

In browser when I hit 172.17.0.1:11001 or 10.71.11150:11001 both case I get Error Cannot GET.. are they both not supposed to work?

markusthoemmes commented 7 years ago

Do you get no response at all or Cannot GET /?

Katiyman commented 7 years ago

Cannot GET/

I get the above response

jasonpet commented 7 years ago

Try 172.17.0.1:11001/health

rabbah commented 7 years ago

Small update: I worked with @Katiyman (via slack) - we verified that the alarm service responds to /ping, and is reachable from inside an action container (by curling the same ping endpoint).

rabbah commented 7 years ago

We tried another action - one that uses npm openwhisk to see if the action can even reach the edge router. And it can't.

I suggested using this action https://github.com/openwhisk/openwhisk/blob/master/tests/dat/actions/helloOpenwhiskPackage.js to create and invoke the action. The logs show:

2017-05-12T06:28:30.058371216Z stdout: error Error: Error encountered calling OpenWhisk: Error: tunneling socket could not be established, statusCode=504
2017-05-12T06:28:30.058434242Z stdout: at Client.handle_errors (/nodejsAction/node_modules/openwhisk/lib/client.js:105:11)
2017-05-12T06:28:30.058443476Z stdout: at rp.catch.err (/nodejsAction/node_modules/openwhisk/lib/client.js:43:38)
2017-05-12T06:28:30.058449927Z stdout: at tryCatcher (/nodejsAction/node_modules/bluebird/js/main/util.js:26:23)
2017-05-12T06:28:30.05845619Z  stdout: at Promise._settlePromiseFromHandler (/nodejsAction/node_modules/bluebird/js/main/promise.js:510:31)
2017-05-12T06:28:30.058462692Z stdout: at Promise._settlePromiseAt (/nodejsAction/node_modules/bluebird/js/main/promise.js:584:18)
2017-05-12T06:28:30.058468819Z stdout: at Promise._settlePromises (/nodejsAction/node_modules/bluebird/js/main/promise.js:700:14)
2017-05-12T06:28:30.058474879Z stdout: at Async._drainQueue (/nodejsAction/node_modules/bluebird/js/main/async.js:123:16)
2017-05-12T06:28:30.058480779Z stdout: at Async._drainQueues (/nodejsAction/node_modules/bluebird/js/main/async.js:133:10)
2017-05-12T06:28:30.058486833Z stdout: at Immediate.Async.drainQueues (/nodejsAction/node_modules/bluebird/js/main/async.js:15:14)
2017-05-12T06:28:30.05849292Z  stdout: at runCallback (timers.js:637:20)
2017-05-12T06:28:30.058499047Z stdout: at tryOnImmediate (timers.js:610:5)
2017-05-12T06:28:30.058505004Z stdout: at processImmediate [as _immediateCallback] (timers.js:582:5)

@jeremiaswerner could this be a network isolation issue in his deployment between the action containers and the rest of the deployment?

Katiyman commented 7 years ago

@jeremiaswerner how should i go about debugging this issue...

Katiyman commented 7 years ago

If network seems to be the issue can we check where we need to provide no_proxy etc properly.. i dont have much clue where all to change that

Katiyman commented 7 years ago

I wanted to check if the call is reaching kafka containers or not.. how to check that?

Katiyman commented 7 years ago

Updating the findings till now:

we tried the following sample code and got the same issue Code: var request = require('request-promise'); function main() { return request('http://172.17.0.1:11001') } issue: { "error": "The action exceeded its time limits of 60000 milliseconds." }

Then we changed the code to below Code: process.env['NO_PROXY'] = '*'; var request = require('request-promise'); function main() { return request('http://172.17.0.1:11001') } Result: Then is seemed to connect and timeout was not there

-->Then we checked below docker exec -it kafka bash -c 'echo $NO_PROXY' docker exec -it kafka bash -c 'echo $HTTP_PROXY' both were giving empty values -->Thenk we changed the action code as below to check the HTTP proxy value var request = require('request-promise'); function main() { console.log(process.env['NO_PROXY']); console.log(process.env['HTTP_PROXY']); return request('http://172.17.0.1:11001') }

result 2017-05-18T04:21:06.078097702Z stdout: undefined 2017-05-18T04:21:06.078164789Z stdout: http://10.xxx.xxx.69:8080

Now need to understand from where it is taking the http_proxy and where no_proxy has to be setup in order for this to work

Katiyman commented 7 years ago

All containers env details ################################## CONTROLLER container env variables sudo docker exec 8249067ec7cd printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=controller SERVICE_CHECK_HTTP=/ping WHISK_VERSION_NAME=local KAFKA_NUMPARTITIONS=2 CONSULSERVER_HOST=172.17.0.1 WHISK_VERSION_BUILDNO=latest CONSUL_HOST_PORT4=8500 SERVICE_CHECK_TIMEOUT=2s SERVICE_CHECK_INTERVAL=15s PORT=8080 WHISK_VERSION_DATE=2017-05-05T09:13:36Z COMPONENT_NAME=controller DEBIAN_FRONTEND=noninteractive LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8 VERSION=8 UPDATE=121 BUILD=13 JAVA_HOME=/usr/lib/jvm/java-8-oracle JRE_HOME=/usr/lib/jvm/java-8-oracle/jre HTTP_PROXY=http://10.xxx.xxx.69:8080 HTTPS_PROXY=http://10.xxx.xxx.69:8080 HOME=/root ##################################################### KAFKA sudo docker exec 83eba3961247 printenv PATH=/kafka/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=83eba3961247 ZOOKEEPER_PORT=tcp://172.17.0.3:2181 ZOOKEEPER_PORT_2181_TCP=tcp://172.17.0.3:2181 ZOOKEEPER_PORT_2181_TCP_ADDR=172.17.0.3 ZOOKEEPER_PORT_2181_TCP_PORT=2181 ZOOKEEPER_PORT_2181_TCP_PROTO=tcp ZOOKEEPER_PORT_2888_TCP=tcp://172.17.0.3:2888 ZOOKEEPER_PORT_2888_TCP_ADDR=172.17.0.3 ZOOKEEPER_PORT_2888_TCP_PORT=2888 ZOOKEEPER_PORT_2888_TCP_PROTO=tcp ZOOKEEPER_PORT_3888_TCP=tcp://172.17.0.3:3888 ZOOKEEPER_PORT_3888_TCP_ADDR=172.17.0.3 ZOOKEEPER_PORT_3888_TCP_PORT=3888 ZOOKEEPER_PORT_3888_TCP_PROTO=tcp ZOOKEEPER_NAME=/kafka/zookeeper ZOOKEEPER_ENV_LANG=C.UTF-8 ZOOKEEPER_ENV_JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk/jre ZOOKEEPER_ENV_JAVA_VERSION=8u121 ZOOKEEPER_ENV_JAVA_ALPINE_VERSION=8.121.13-r0 ZOOKEEPER_ENV_ZOO_USER=zookeeper ZOOKEEPER_ENV_ZOO_CONF_DIR=/conf ZOOKEEPER_ENV_ZOO_DATA_DIR=/data ZOOKEEPER_ENV_ZOO_DATA_LOG_DIR=/datalog ZOOKEEPER_ENV_ZOO_PORT=2181 ZOOKEEPER_ENV_ZOO_TICK_TIME=2000 ZOOKEEPER_ENV_ZOO_INIT_LIMIT=5 ZOOKEEPER_ENV_ZOO_SYNC_LIMIT=2 ZOOKEEPER_ENV_ZOOCFGDIR=/conf KAFKA_ADVERTISED_HOST_NAME=172.17.0.1 JAVA_VER=8 JAVA_HOME=/usr/lib/jvm/java-8-oracle KAFKA_VERSION=0.10.0.1 KAFKA_SCALA_VERSION=2.11 JMX_PORT=7203 KAFKA_RELEASE_ARCHIVE=kafka_2.11-0.10.0.1.tgz HOME=/kafka ###################### INVOKER sudo docker exec b54337ee477f printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=invoker0 COMPONENT_NAME=invoker0 CONSULSERVER_HOST=172.17.0.1 CONSUL_HOST_PORT4=8500 PORT=8080 SELF_DOCKER_ENDPOINT=localhost SERVICE_CHECK_HTTP=/ping SERVICE_CHECK_TIMEOUT=2s SERVICE_CHECK_INTERVAL=15s DEBIAN_FRONTEND=noninteractive LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8 VERSION=8 UPDATE=121 BUILD=13 JAVA_HOME=/usr/lib/jvm/java-8-oracle JRE_HOME=/usr/lib/jvm/java-8-oracle/jre DOCKER_VERSION=1.12.0 HOME=/root ########################## WSK0_warmjscontainer

sudo docker exec c4791239c64b printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=c4791239c64b __OW_API_HOST=https://172.17.0.1:443 SERVICE_IGNORE=true DEBIAN_FRONTEND=noninteractive HTTP_PROXY=http://10.xxx.xxx.69:8080 HTTPS_PROXY=http://10.xxx.xxx.69:8080 NODE_VERSION=6.9.1 HOME=/root ############################## ALARM

sudo docker exec 29f219fc1015 printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=alarmstrigger DB_PASSWORD=Nokia123 ROUTER_HOST=172.17.0.1 DB_PROTOCOL=http DB_HOST=10.71.11.150:5984 DB_USERNAME=admin DB_PREFIX=rootmanu-150 PORT=8080 DEBIAN_FRONTEND=noninteractive HOME=/root #################################################################### NGINX

sudo docker exec a2696bf51899 printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=nginx NGINX_VERSION=1.11.13-1~jessie HOME=/root ########################## ZOOKEEPER

sudo docker exec 947fa689a7ef printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin:/zookeeper-3.4.10/bin HOSTNAME=947fa689a7ef LANG=C.UTF-8 JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk/jre JAVA_VERSION=8u121 JAVA_ALPINE_VERSION=8.121.13-r0 ZOO_USER=zookeeper ZOO_CONF_DIR=/conf ZOO_DATA_DIR=/data ZOO_DATA_LOG_DIR=/datalog ZOO_PORT=2181 ZOO_TICK_TIME=2000 ZOO_INIT_LIMIT=5 ZOO_SYNC_LIMIT=2 ZOOCFGDIR=/conf HOME=/root ################################################################ REGISTRATOR

sudo docker exec 80ab0a589766 printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=registrator HOME=/root ########################################## CONSULSERVER

sudo docker exec 0061355f02cc printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=0061355f02cc CONSUL_VERSION=0.7.1 DOCKER_BASE_VERSION=0.0.4 HOME=/root ######################################## COUCHDB

sudo docker exec 625a842c1481 printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=625a842c1481 GPG_KEYS=15DD4F3B8AACA54740EB78C7B7B7C53943ECCEE1 1CFBFA43C19B6DF4A0CA3934669C02FFDF3CEBA3 25BBBAC113C1BFD5AA594A4C9F96B92930380381 4BFCA2B99BADC6F9F105BEC9C5E32E2D6B065BFB 5D680346FAA3E51B29DBCB681015F68F9DA248BC 7BCCEB868313DDA925DF1805ECA5BCB7BB9656B0 C3F4DFAEAD621E1C94523AEEC376457E61D50B88 D2B17F9DA23C0A10991AF2E3D9EE01E47852AEE4 E0AF0A194D55C84E4A19A801CDB0C0F904F4EE9B COUCHDB_VERSION=1.6.1 HOME=/root

Katiyman commented 7 years ago

Finally we found that the issue was becoz of HTTP_PROXY and HTTPS_PROXY variable set for controller and nodejs container Once these settings were removed new build done and new deployement. this particular issue was resolved. @markusthoemmes @rabbah @jasonpet Thanks for all the help. :)

hsane2001 commented 7 years ago

I am running in the same issue, how did you remove the proxy and rebuild? Is it set in the Docker.conf file or somewhere in the build within ansible for you?

hsane2001 commented 7 years ago

I was unable to build "gradle distDocker" as is, hence manually inserted the company proxy into each Dockerfile which allowed apt and npm to install packages. Thats how the build worked, but if I remove proxy, build will fail - chicken and egg problem?

Katiyman commented 7 years ago

@hsane2001 Actually we need not expose it as the ENV variable but rather in each command in the Dockerfile then this will not be the issue.

hsane2001 commented 7 years ago

I kept it as ENV in each docker file, but build this time without "ansible-playbook API-gateway". Then Pinged externally from the controller, consul, Kafka, invoked containers and now I can get actions that reach external sites. However I still can't get trigger from whisk/alarm to work, it fails as soon as I provide the feed on socket,connect On failure

Katiyman commented 7 years ago

I dont think putting proxy as a ENV variable is required.. that was the mistake i did, becoz of which i got issue export proxy in individual commands where ever proxy is required. like below

RUN export http_proxy=http://1x.xx.xx.6:8080 && \ export https_proxy=https://1x.xx.xx.6:8080 && \ apk add --no-cache \ bzip2-dev \ gcc \ libc-dev \ libxslt-dev \ libxml2-dev \ libffi-dev \ linux-headers \ openssl-dev

hsane2001 commented 7 years ago

OK that makes sense, I can try that. However I cant build the api-gateway (an additional component right now) if the http_proxy variable in my native ubuntu ENV is set - know any reason why? I cant find the Docker file for api-gateway since its not part of the "gradle" build.

Katiyman commented 7 years ago

I am not sure about api-gateway... but it can be a no_proxy issue.

hsane2001 commented 7 years ago

I have no_proxy set in native environment. I can create actions that get external content. Problem is with creating a trigger using the alarms feed. How to verify my install is correct using the variables mentioned in the posts above, and if the alarms feed is actually reaching the action? Creating triggers is no problem, using the feed gives errors.

hsane2001 commented 7 years ago

I rebuilt with your advise on Proxy and it works now - thanks Katiyman