Creates a CloudFormation template that uses AWS StepFunctions to automate the building and training of Sagemaker custom models based on S3 and GitHub events
When running npm run up I receive the following error:
> sagebuild@3.0.0 up /home/ubuntu/environment/aws-sagemaker-build
> make && ./bin/upload.sh && ./templates/main/bin/launch.js
mkdir -p build; mkdir -p build/lambda; mkdir -p build/templates
for l in templates/activo templates/main; do \
make -C $l; \
done;
make[1]: Entering directory '/home/ubuntu/environment/aws-sagemaker-build/templates/activo'
./bin/check.js
StepFunction:CrawlerStart 372/4096
StepFunction:CrawlerStatus 414/4096
{ ResponseMetadata: { RequestId: '0b457362-9215-11e9-ba71-b70688c9a33c' },
Parameters:
[ { ParameterKey: 'UiTemplateS3Uri', NoEcho: false },
{ ParameterKey: 'AssetPrefix', NoEcho: false },
{ ParameterKey: 'WorkteamArn', NoEcho: false },
{ ParameterKey: 'MaxConcurrentTaskCount', NoEcho: false },
{ ParameterKey: 'NumberOfHumanWorkersPerDataObject',
NoEcho: false },
{ ParameterKey: 'TaskAvailabilityLifetimeInSeconds',
NoEcho: false },
{ ParameterKey: 'PreHumanTaskLambdaArn', NoEcho: false },
{ ParameterKey: 'MaxPercentageOfInputDatasetLabeled',
NoEcho: false },
{ ParameterKey: 'TaskTimeLimitInSeconds', NoEcho: false },
{ ParameterKey: 'MaxHumanLabeledObjectCount', NoEcho: false },
{ ParameterKey: 'AssetBucket', NoEcho: false } ],
Description: '',
Capabilities: [ 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ],
CapabilitiesReason: 'The following resource(s) require capabilities: [AWS::IAM::Role, AWS::CloudFormation::Stack]',
DeclaredTransforms: [] }
Resources: 15
make[1]: Leaving directory '/home/ubuntu/environment/aws-sagemaker-build/templates/activo'
make[1]: Entering directory '/home/ubuntu/environment/aws-sagemaker-build/templates/main'
rm build/notebooks.zip ; cd SageMakerNotebook/notebooks && zip -r ../../../../build/notebooks.zip .
rm: cannot remove 'build/notebooks.zip': No such file or directory
adding: mxnet/ (stored 0%)
adding: mxnet/input.html (deflated 65%)
adding: mxnet/mxnet_mnist.ipynb (deflated 67%)
adding: mxnet/mnist.py (deflated 61%)
adding: amazon/ (stored 0%)
adding: amazon/linear_learner_mnist.ipynb (deflated 67%)
adding: tensorflow/ (stored 0%)
adding: tensorflow/utils.py (deflated 59%)
adding: tensorflow/setup.sh (deflated 61%)
adding: tensorflow/mnist.py (deflated 66%)
adding: tensorflow/daemon.json (deflated 42%)
adding: tensorflow/tensorflow_distributed_mnist.ipynb (deflated 69%)
adding: HyperParameterOptimization/ (stored 0%)
adding: HyperParameterOptimization/hpo_mxnet_mnist.ipynb (deflated 67%)
adding: HyperParameterOptimization/input.html (deflated 65%)
adding: HyperParameterOptimization/mnist.py (deflated 62%)
./bin/check.js
(node:3990) UnhandledPromiseRejectionWarning: Error: Cannot find module './stepfunctions'
at Function.Module._resolveFilename (module.js:548:15)
at Function.Module._load (module.js:475:25)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/ubuntu/environment/aws-sagemaker-build/templates/main/step_function/stateMachines/DockerTrainDeploy/index.js:6:10)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at next (/home/ubuntu/environment/aws-sagemaker-build/templates/main/step_function/stateMachines/index.js:30:39)
at Object.<anonymous> (/home/ubuntu/environment/aws-sagemaker-build/templates/main/step_function/stateMachines/index.js:18:32)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
(node:3990) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:3990) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
make[1]: Leaving directory '/home/ubuntu/environment/aws-sagemaker-build/templates/main'
make -C ./lambda
make[1]: Entering directory '/home/ubuntu/environment/aws-sagemaker-build/lambda'
for l in ./util; do \
make -C $l; \
done;
make[2]: Entering directory '/home/ubuntu/environment/aws-sagemaker-build/lambda/util'
echo "Building util"; cd nodejs && npm install && cp -r lib/* node_modules && cd .. && zip -r -q ../../build/lambda/util.zip .
Building util
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN unzip@1.0.0 No description
npm WARN unzip@1.0.0 No repository field.
added 20 packages from 75 contributors and audited 23 packages in 1.761s
found 0 vulnerabilities
make[2]: Leaving directory '/home/ubuntu/environment/aws-sagemaker-build/lambda/util'
make[1]: Leaving directory '/home/ubuntu/environment/aws-sagemaker-build/lambda'
bootstrap bucket is jasbarto-virginia-ml-bucket/artifacts/blog
upload: build/notebooks.zip to s3://jasbarto-virginia-ml-bucket/artifacts/blog/notebooks.zip
upload: build/lambda/util.zip to s3://jasbarto-virginia-ml-bucket/artifacts/blog/lambda/util.zip
console launch url:
https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?stackName=SageBuild&templateURL=http://s3.amazonaws.com/jasbarto-virginia-ml-bucket/artifacts/blog/templates/main.json¶m_AssetBucket=jasbarto-virginia-ml-bucket¶m_AssetPrefix=artifacts/blog¶m_Username=Admin
(node:4070) UnhandledPromiseRejectionWarning: Error: Cannot find module './stepfunctions'
at Function.Module._resolveFilename (module.js:548:15)
at Function.Module._load (module.js:475:25)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/ubuntu/environment/aws-sagemaker-build/templates/main/step_function/stateMachines/DockerTrainDeploy/index.js:6:10)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at next (/home/ubuntu/environment/aws-sagemaker-build/templates/main/step_function/stateMachines/index.js:30:39)
at Object.<anonymous> (/home/ubuntu/environment/aws-sagemaker-build/templates/main/step_function/stateMachines/index.js:18:32)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
(node:4070) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 3)
(node:4070) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
When running
npm run upI receive the following error: