search

apache / openwhisk-devtools

Development tools for building and deploying Apache OpenWhisk
https://openwhisk.apache.org/
Apache License 2.0
179 stars 126 forks source link

Makefile quick-start error #322

Closed marquies closed 4 years ago

marquies commented 4 years ago

Hey, thanks for your awesome work. I want to use the quick-start on an Amazon Linux EC2 instance.

I get the same error like the travis-ci:

pinging minio...
 ... OK
openwhisk_minio_1 is up-to-date
[dumb-init] rclone: No such file or directory
make: *** [setup] Fehler 2

Looks like the following command causes the error

docker-compose --project-name openwhisk run --rm --no-deps apigateway rclone mkdir minio:api-gateway
WARNING: The DOCKER_COMPOSE_HOST variable is not set. Defaulting to a blank string.
[dumb-init] rclone: No such file or directory
marquies commented 4 years ago

Travis Log https://travis-ci.org/github/apache/openwhisk-devtools/jobs/712576264

shiipou commented 4 years ago

Got the same I think it come from this lines : https://github.com/apache/openwhisk-devtools/blob/a7f2a99910885b17493a36f23ab19adb67146ee0/docker-compose/Makefile#L257-L258

The command apigateway say the directory rclone does not exist. This directory must contain the rclone.conf file that contain : 

[minio]
type = s3
provider = Minio
env_auth = false
access_key_id = 5VCTEQOQ0GR0NV1T67GN
secret_access_key = 8MBK5aJTR330V1sohz4n1i7W5Wv/jzahARNHUzi3
region = us-east-1
endpoint = http://minio.docker:9000
location_constraint =
server_side_encryption =
acl = public-read

Here logs : 

root@k8s-worker-01:/# cat /root/tmp/openwhisk/controller/logs/controller-local_logs.log 
Listening for transport dt_socket at address: 9222
[2020-08-03T09:39:52.045Z] [INFO] Slf4jLogger started
[2020-08-03T09:39:52.121Z] [INFO] Starting remoting
[2020-08-03T09:39:52.468Z] [INFO] Remoting started; listening on addresses :[akka.tcp://controller-actor-system@controller:2551]
[2020-08-03T09:39:52.470Z] [INFO] Remoting now listens on addresses: [akka.tcp://controller-actor-system@controller:2551]
[2020-08-03T09:39:52.511Z] [INFO] Cluster Node [akka.tcp://controller-actor-system@controller:2551] - Starting up, Akka version [2.5.26] ...
[2020-08-03T09:39:52.638Z] [INFO] Cluster Node [akka.tcp://controller-actor-system@controller:2551] - Registered cluster JMX MBean [akka:type=Cluster]
[2020-08-03T09:39:52.639Z] [INFO] Cluster Node [akka.tcp://controller-actor-system@controller:2551] - Started up successfully
[2020-08-03T09:39:52.697Z] [INFO] Cluster Node [akka.tcp://controller-actor-system@controller:2551] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/cluster-usage.html#joining-to-seed-nodes
[2020-08-03T09:39:53.408Z] [WARN] Failed to attach the instrumentation because the Kamon Bundle is not present on the classpath
[2020-08-03T09:39:53.719Z] [INFO] Started the Kamon StatsD reporter
[2020-08-03T09:39:54.588Z] [INFO] [#tid_sid_unknown] [] [Config] environment set value for limits.triggers.fires.perMinute
[2020-08-03T09:39:54.589Z] [INFO] [#tid_sid_unknown] [] [Config] environment set value for limits.actions.sequence.maxLength
[2020-08-03T09:39:54.589Z] [INFO] [#tid_sid_unknown] [] [Config] environment set value for limits.actions.invokes.concurrent
[2020-08-03T09:39:54.589Z] [INFO] [#tid_sid_unknown] [] [Config] environment set value for limits.actions.invokes.perMinute
[2020-08-03T09:39:54.589Z] [INFO] [#tid_sid_unknown] [] [Config] environment set value for runtimes.manifest
[2020-08-03T09:39:54.589Z] [INFO] [#tid_sid_unknown] [] [Config] environment set value for kafka.hosts
[2020-08-03T09:39:54.589Z] [INFO] [#tid_sid_unknown] [] [Config] environment set value for port
[2020-08-03T09:39:56.996Z] [INFO] [#tid_sid_unknown] [] [KafkaMessagingProvider] created topic completed0
[2020-08-03T09:39:58.241Z] [INFO] [#tid_sid_unknown] [] [KafkaMessagingProvider] created topic health
[2020-08-03T09:39:59.795Z] [INFO] [#tid_sid_unknown] [] [KafkaMessagingProvider] created topic cacheInvalidation
[2020-08-03T09:40:00.635Z] [INFO] [#tid_sid_unknown] [] [KafkaMessagingProvider] created topic events
[2020-08-03T09:40:01.351Z] [INFO] [#tid_sid_controller] [] [Controller] starting controller instance 0 [marker:controller_startup0_counter:6792]
[2020-08-03T09:40:03.245Z] [INFO] [#tid_sid_dispatcher] [] [MessageFeed] handler capacity = 128, pipeline fill at = 128, pipeline depth = 256
[2020-08-03T09:40:03.872Z] [INFO] [#tid_sid_loadbalancer] [] [ShardingContainerPoolBalancerState] managedFraction = 0.9, blackboxFraction = 0.1
[2020-08-03T09:40:04.310Z] [INFO] [#tid_sid_dispatcher] [] [MessageFeed] handler capacity = 128, pipeline fill at = 128, pipeline depth = 256
[2020-08-03T09:40:05.591Z] [INFO] [#tid_sid_loadbalancer] [] [WhiskAction] [GET] serving from datastore: CacheKey(whisk.system/invokerHealthTestAction0) [marker:database_cacheMiss_counter:11033]
[2020-08-03T09:40:05.670Z] [INFO] [#tid_sid_loadbalancer] [] [CouchDbRestStore] [GET] 'local_whisks' finding document: 'id: whisk.system/invokerHealthTestAction0' [marker:database_getDocument_start:11112]
[2020-08-03T09:40:05.910Z] [INFO] [#tid_sid_loadbalancer] [] [CouchDbRestStore]  [marker:database_getDocument_finish:11353:240]
[2020-08-03T09:40:05.915Z] [INFO] [#tid_sid_loadbalancer] [] [WhiskAction] invalidating CacheKey(whisk.system/invokerHealthTestAction0)
[2020-08-03T09:40:06.036Z] [INFO] [#tid_sid_loadbalancer] [] [WhiskAction] write initiated on new cache entry
[2020-08-03T09:40:06.042Z] [INFO] [#tid_sid_loadbalancer] [] [CouchDbRestStore] [PUT] 'local_whisks' saving document: 'id: whisk.system/invokerHealthTestAction0, rev: null' [marker:database_saveDocument_start:11485]
[2020-08-03T09:40:06.066Z] [INFO] [#tid_sid_loadbalancer] [] [InvokerPool] test action for invoker health now exists
[2020-08-03T09:40:06.106Z] [ERROR] [#tid_sid_loadbalancer] [] [CouchDbRestStore] [PUT] 'local_whisks' failed to put document: 'id: whisk.system/invokerHealthTestAction0, rev: null'; http status: '404 Not Found (details: {"error":"not_found","reason":"Database does not exist."}
)' [marker:database_saveDocument_error:11547:62]
[2020-08-03T09:40:06.107Z] [INFO] [#tid_sid_loadbalancer] [] [WhiskAction] invalidating CacheKey(whisk.system/invokerHealthTestAction0)
[2020-08-03T09:40:06.314Z] [INFO] [#tid_sid_controller] [] [Controller] loadbalancer initialized: ShardingContainerPoolBalancer
[2020-08-03T09:40:06.317Z] [INFO] [#tid_sid_dispatcher] [] [MessageFeed] handler capacity = 128, pipeline fill at = 128, pipeline depth = 256
[2020-08-03T09:40:06.532Z] [INFO] [#tid_sid_controller] [] [KindRestrictor] all kinds are allowed, the white-list is not specified
[2020-08-03T09:40:07.560Z] [INFO] [#tid_sid_controller] [] [ActionsApi] actionSequenceLimit '20'
root@k8s-worker-01:/# cat /root/tmp/openwhisk/setup.log 
Attaching to openwhisk_minio_1
minio_1            | Created minio configuration file successfully at /root/.minio
minio_1            | 
minio_1            |  You are running an older version of Minio released 2 years ago 
minio_1            |  Update: docker pull minio/minio:RELEASE.2020-07-31T03-39-05Z 
minio_1            | 
minio_1            | 
minio_1            | Endpoint:  http://172.18.0.2:9000  http://127.0.0.1:9000
minio_1            | AccessKey: 5VCTEQOQ0GR0NV1T67GN 
minio_1            | SecretKey: 8MBK5aJTR330V1sohz4n1i7W5Wv/jzahARNHUzi3 
minio_1            | 
minio_1            | Browser Access:
minio_1            |    http://172.18.0.2:9000  http://127.0.0.1:9000
minio_1            | 
minio_1            | Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
minio_1            |    $ mc config host add myminio http://172.18.0.2:9000 5VCTEQOQ0GR0NV1T67GN 8MBK5aJTR330V1sohz4n1i7W5Wv/jzahARNHUzi3
minio_1            | 
minio_1            | Object API (Amazon S3 compatible):
minio_1            |    Go:         https://docs.minio.io/docs/golang-client-quickstart-guide
minio_1            |    Java:       https://docs.minio.io/docs/java-client-quickstart-guide
minio_1            |    Python:     https://docs.minio.io/docs/python-client-quickstart-guide
minio_1            |    JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
minio_1            |    .NET:       https://docs.minio.io/docs/dotnet-client-quickstart-guide
minio_1            | Exiting on signal terminated
minio_1            | 
minio_1            |  You are running an older version of Minio released 2 years ago 
minio_1            |  Update: docker pull minio/minio:RELEASE.2020-07-31T03-39-05Z 
minio_1            | 
minio_1            | 
minio_1            | Endpoint:  http://172.18.0.2:9000  http://127.0.0.1:9000
minio_1            | AccessKey: 5VCTEQOQ0GR0NV1T67GN 
minio_1            | SecretKey: 8MBK5aJTR330V1sohz4n1i7W5Wv/jzahARNHUzi3 
minio_1            | 
minio_1            | Browser Access:
minio_1            |    http://172.18.0.2:9000  http://127.0.0.1:9000
minio_1            | 
minio_1            | Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
minio_1            |    $ mc config host add myminio http://172.18.0.2:9000 5VCTEQOQ0GR0NV1T67GN 8MBK5aJTR330V1sohz4n1i7W5Wv/jzahARNHUzi3
minio_1            | 
minio_1            | Object API (Amazon S3 compatible):
minio_1            |    Go:         https://docs.minio.io/docs/golang-client-quickstart-guide
minio_1            |    Java:       https://docs.minio.io/docs/java-client-quickstart-guide
minio_1            |    Python:     https://docs.minio.io/docs/python-client-quickstart-guide
minio_1            |    JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
minio_1            |    .NET:       https://docs.minio.io/docs/dotnet-client-quickstart-guide
shiipou commented 4 years ago

I've tested it this morning on MacOs X 10.15.6 (19G73) with Docker Desktop for Mac, it fails too.

The first try I send was on Ubuntu 20.04.

rabbah commented 4 years ago

Thanks for the updates. Have you tried the standalone openwhisk stack? https://github.com/apache/openwhisk/#quick-start if you're deploying a small scale/single VM installation it may be easier to deploy.

I don't have any insight into the failing rclone yet.

shiipou commented 4 years ago

The standalone version didn't support persistence of the created actions. For my project I really need it to not redeploy each of my base action each time I restart my computer to allow me to develop. But thank you for the suggestion.

rabbah commented 4 years ago

The standalone stack can persist the db to disk - see https://github.com/apache/openwhisk/tree/master/core/standalone#using-couchdb.

You can enable couchdb with --couchdb for example:

./gradlew core:standalone:bootRun --args=--couchdb

or if using bin/wskdev:

bin/wskdev standalone --a="--args=--couchdb"
rabbah commented 4 years ago

Regardless, need to get to bottom of the rclone issue being missing.

mhamann commented 4 years ago

The apigateway builds are throwing errors like this: 

installing rclone sync ... 
package github.com/putdotio/go-putio/putio: cannot find package "github.com/putdotio/go-putio/putio" in any of:
        /usr/lib/go/src/github.com/putdotio/go-putio/putio (from $GOROOT)
        /usr/lib/go/bin/src/github.com/putdotio/go-putio/putio (from $GOPATH)
package github.com/yunify/qingstor-sdk-go/v3/config: cannot find package "github.com/yunify/qingstor-sdk-go/v3/config" in any of:
        /usr/lib/go/src/github.com/yunify/qingstor-sdk-go/v3/config (from $GOROOT)
        /usr/lib/go/bin/src/github.com/yunify/qingstor-sdk-go/v3/config (from $GOPATH)
package github.com/yunify/qingstor-sdk-go/v3/request/errors: cannot find package "github.com/yunify/qingstor-sdk-go/v3/request/errors" in any of:
        /usr/lib/go/src/github.com/yunify/qingstor-sdk-go/v3/request/errors (from $GOROOT)
        /usr/lib/go/bin/src/github.com/yunify/qingstor-sdk-go/v3/request/errors (from $GOPATH)
package github.com/yunify/qingstor-sdk-go/v3/service: cannot find package "github.com/yunify/qingstor-sdk-go/v3/service" in any of:
        /usr/lib/go/src/github.com/yunify/qingstor-sdk-go/v3/service (from $GOROOT)
        /usr/lib/go/bin/src/github.com/yunify/qingstor-sdk-go/v3/service (from $GOPATH)
mv: can't rename '/usr/lib/go/bin/rclone': No such file or directory

Interestingly enough, this doesn't fail the build (which seems odd as well).

I don't know why offhand, but I'll see if I can't investigate further and PR a fix.

rabbah commented 4 years ago

@mhamann provided a fix https://github.com/apache/openwhisk-apigateway/pull/381.

marquies commented 4 years ago

I removed all images and restated the installation. Works fine now. Thank you ❤️