awslabs / serverless-photo-recognition

A collection of 3 lambda functions that are invoked by Amazon S3 or Amazon API Gateway to analyze uploaded images with Amazon Rekognition and save picture labels to ElasticSearch (written in Kotlin)
https://aws.amazon.com/blogs/ai/use-amazon-rekognition-to-build-an-end-to-end-serverless-photo-recognition-system/
Apache License 2.0
383 stars 105 forks source link

some erros with setup script #15

Closed s53zo closed 5 years ago

s53zo commented 5 years ago

I am running this on Debian, java8, python 2.7 and awscli installed... It outputs some erros and then goes on forever with "Waiting for the ES domain rekognition20190124114720 to finish processing. Sleeping.."


./setupEnvironment.sh Using the following 'root name' for your resources, such as S3 bucket: 20190124114720 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 15 100 15 0 0 45 0 --:--:-- --:--:-- --:--:-- 45 Creating the Cognito resources { "Role": { "Arn": "arn:aws:iam::179810983012:role/20190124114720rek-unauthenticated-role", "AssumeRolePolicyDocument": { "Statement": [ { "Condition": { "ForAnyValue:StringLike": { "cognito-identity.amazonaws.com:amr": "unauthenticated" }, "StringEquals": { "cognito-identity.amazonaws.com:aud": "us-east-1:9be46eae-5a3d-4a18-8b38-9b9e4ff8803c" } }, "Action": "sts:AssumeRoleWithWebIdentity", "Principal": { "Federated": "cognito-identity.amazonaws.com" }, "Effect": "Allow" } ], "Version": "2012-10-17" }, "RoleName": "20190124114720rek-unauthenticated-role", "RoleId": "AROAJN66ZUIEFDLUYRREA", "CreateDate": "2019-01-24T10:47:24Z", "Path": "/" } } { "Role": { "Arn": "arn:aws:iam::179810983012:role/20190124114720rek-authenticated-role", "RoleId": "AROAJLGGGEFVRQ3E227SO", "AssumeRolePolicyDocument": { "Statement": [ { "Condition": { "ForAnyValue:StringLike": { "cognito-identity.amazonaws.com:amr": "authenticated" }, "StringEquals": { "cognito-identity.amazonaws.com:aud": "us-east-1:9be46eae-5a3d-4a18-8b38-9b9e4ff8803c" } }, "Effect": "Allow", "Principal": { "Federated": "cognito-identity.amazonaws.com" }, "Action": "sts:AssumeRoleWithWebIdentity" } ], "Version": "2012-10-17" }, "CreateDate": "2019-01-24T10:47:27Z", "Path": "/", "RoleName": "20190124114720rek-authenticated-role" } } usage: aws [options] [ ...] [parameters] To see help text, you can run:

aws help aws help aws help

Unknown options: --schema, Name=email,Required=true usage: aws [options] [ ...] [parameters] To see help text, you can run:

aws help aws help aws help aws: error: argument --user-pool-id: expected one argument

Parameter validation failed: Invalid length for parameter CognitoIdentityProviders[0].ClientId, value: 0, valid range: 1-inf { "Role": { "RoleName": "lambda-to-es-rek-s3-20190124114720", "Arn": "arn:aws:iam::179810983012:role/lambda-to-es-rek-s3-20190124114720", "RoleId": "AROAJTEJ7B6OPZMPIA3QA", "Path": "/", "CreateDate": "2019-01-24T10:47:34Z", "AssumeRolePolicyDocument": { "Statement": [ { "Effect": "Allow", "Action": "sts:AssumeRole", "Principal": { "Service": "lambda.amazonaws.com" } } ], "Version": "2012-10-17" } } } make_bucket: rekognition-20190124114720 The Task.leftShift(Closure) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use Task.doLast(Action) instead. at build_cr2xxzh92n7ciadgvde49npdd.run(/home/openhab/serverless-photo-recognition/build.gradle:48) :compileKotlin w: /home/openhab/serverless-photo-recognition/src/main/kotlin/com/budilov/db/ESPictureService.kt: (91, 23): 'getSourceAsObjectList(Class<T!>!): (Mutable)List<T!>!' is deprecated. Deprecated in Java :compileJava UP-TO-DATE :copyMainKotlinClasses :processResources UP-TO-DATE :classes UP-TO-DATE :jar :startScripts :distTar :distZip :assemble :compileTestKotlin UP-TO-DATE :compileTestJava UP-TO-DATE :copyTestKotlinClasses :processTestResources UP-TO-DATE :testClasses UP-TO-DATE :test UP-TO-DATE :check UP-TO-DATE :build

BUILD SUCCESSFUL

Total time: 13.657 secs Setting up the S3 Lambda events Adding the ADD event Lambda permissions { "Statement": "{\"Sid\":\"20190124114720-rekognition\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"s3.amazonaws.com\"},\"Action\":\"lambda:InvokeFunction\",\"Resource\":\"arn:aws:lambda:us-east-1:179810983012:function:rekognition-add-picture-20190124114720\",\"Condition\":{\"StringEquals\":{\"AWS:SourceAccount\":\"179810983012\"},\"ArnLike\":{\"AWS:SourceArn\":\"arn:aws:s3:::rekognition-20190124114720\"}}}" } Adding the DEL event Lambda permissions { "Statement": "{\"Sid\":\"20190124114720-rekognition\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"s3.amazonaws.com\"},\"Action\":\"lambda:InvokeFunction\",\"Resource\":\"arn:aws:lambda:us-east-1:179810983012:function:rekognition-del-picture-20190124114720\",\"Condition\":{\"StringEquals\":{\"AWS:SourceAccount\":\"179810983012\"},\"ArnLike\":{\"AWS:SourceArn\":\"arn:aws:s3:::rekognition-20190124114720\"}}}" } Preparing the S3 Notifications json definitions file Creating the events Setup Elasticsearch with a domain of rekognition20190124114720 { "DomainStatus": { "AdvancedOptions": { "rest.action.multi.allow_explicit_index": "true" }, "DomainId": "179810983012/rekognition20190124114720", "Deleted": false, "DomainName": "rekognition20190124114720", "SnapshotOptions": { "AutomatedSnapshotStartHour": 0 }, "AccessPolicies": "", "Processing": true, "EBSOptions": { "VolumeType": "standard", "VolumeSize": 10, "EBSEnabled": true }, "ElasticsearchVersion": "5.1", "ARN": "arn:aws:es:us-east-1:179810983012:domain/rekognition20190124114720", "Created": true, "ElasticsearchClusterConfig": { "InstanceCount": 1, "InstanceType": "t2.small.elasticsearch", "DedicatedMasterEnabled": false, "ZoneAwarenessEnabled": false } } } { "DomainConfig": { "ElasticsearchVersion": { "Status": { "State": "Processing", "UpdateDate": 1548326914.886, "PendingDeletion": false, "UpdateVersion": 2, "CreationDate": 1548326914.886 }, "Options": "5.1" }, "ElasticsearchClusterConfig": { "Status": { "State": "Processing", "UpdateDate": 1548326914.886, "PendingDeletion": false, "UpdateVersion": 2, "CreationDate": 1548326914.886 }, "Options": { "InstanceCount": 1, "InstanceType": "t2.small.elasticsearch", "ZoneAwarenessEnabled": false, "DedicatedMasterEnabled": false } }, "AccessPolicies": { "Status": { "State": "Processing", "UpdateDate": 1548326918.297, "PendingDeletion": false, "UpdateVersion": 4, "CreationDate": 1548326918.297 }, "Options": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::179810983012:role/lambda-to-es-rek-s3-20190124114720\"},\"Action\":\"es:\",\"Resource\":\"arn:aws:es:us-east-1:179810983012:domain/rekognition20190124114720/\"},{\"Sid\":\"\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"\"},\"Action\":\"es:\",\"Resource\":\"arn:aws:es:us-east-1:179810983012:domain/rekognition20190124114720/*\",\"Condition\":{\"IpAddress\":{\"aws:SourceIp\":\"84.255.242.202\"}}}]}" }, "EBSOptions": { "Status": { "State": "Processing", "UpdateDate": 1548326914.886, "PendingDeletion": false, "UpdateVersion": 2, "CreationDate": 1548326914.886 }, "Options": { "VolumeSize": 10, "VolumeType": "standard", "EBSEnabled": true } }, "SnapshotOptions": { "Status": { "State": "Processing", "UpdateDate": 1548326914.886, "PendingDeletion": false, "UpdateVersion": 2, "CreationDate": 1548326914.886 }, "Options": { "AutomatedSnapshotStartHour": 0 } }, "AdvancedOptions": { "Status": { "State": "Active", "UpdateDate": 1548326918.47, "PendingDeletion": false, "UpdateVersion": 4, "CreationDate": 1548326918.47 }, "Options": { "rest.action.multi.allow_explicit_index": "true" } } } } Waiting for the ES domain rekognition20190124114720 to finish processing. Sleeping... Waiting for the ES domain rekognition20190124114720 to finish processing. Sleeping.. Waiting for the ES domain rekognition20190124114720 to finish processing. Sleeping... Waiting for the ES domain rekognition20190124114720 to finish processing. Sleeping...

vbudilov commented 5 years ago

Try running some sample commands manually to make sure you have the appropriate permissions. Also, make sure you have the latest cli installed and the default output is set to json (and the region supports all of the services).

vbudilov commented 5 years ago

@s53zo so what was it?

s53zo commented 5 years ago

I decided to take another way and used https://github.com/awsdocs/amazon-rekognition-developer-guide/blob/master/code_examples/python_examples/image/python-detect-labels.py

Thanks anyway!