Amazon

[drniiken]

User story

As a developer I want to have the prototype and the associated technologies running on AWS. I want to get a feeling for prices and network latency when transferring large docker images and models.

Definition of done

• [x] SIM-CORE platform as defined in D1.1 is setup on AWS
• [x] Private SIM-CORE docker registry stored on AWS
• [x] Cost estimates for operating the full platform
• [x] Documentation
• [x] Deploy in house: Reproducible hardware/software configurations (Ansible)

[mguidon]

Setup computational backend as a docker swarm with workers using different hardware (gpu/cpu) if possible

[mguidon]

Definition of Done:

• This is part of D5.2
• computational backend demo
• qx demo
• Documentation
• s4l solver

[mguidon]

1. Familiarize with EC2 and get docker swarm up and running
2. Move docker images from masu to dockerhub
3. Run workflow (might need some changes due to hardcoded IPs)
4. Hybrid clusters? Very easy, we could have a service with the XRpcWorker and its thrift interface almost for free.
5. Familiarize with amazon image registry: Oh buy this is a bit difficult due to the 12 h expiration of credentials. Maybe this helps: https://github.com/awslabs/amazon-ecr-credential-helper
6. qx
7. iSolve

[drniiken]

1. Amazon MaG

• huge amount of services, options
• example
• 3 nodes
• simcore.io domain, purchased
• redirect and connection to Amazon
• now we use docker swarm, might change to other tool
• complete py sdk available on amazon
• linux normally
• can also run win containers, could run win super S4L service
• blackfinn has both ... virtual machines and real hardware
• test Win desktop S4L running on amazon
• setup business model
• charges for hardware, charges for usage
• check payment options

[drniiken]

• MaG

• we now have private sim core docker registry

• cost estimates, coming

• deploy in house up and running

• 4 linux machines

• whole setup which is on Amazon is in basement, mirrored

• cost estimates

• see summary by MaG https://github.com/mguidon/oSPARC_Test/blob/workflow/demos/workflow/aws/docs/pricing.md

• you can pay upfront for several years

• price per hour usage also possible

• pay for hardware, network managmenet

• ...

• see example setup and pricing

• in MaG's file

• have a basic setup for the website

• and then on demand items for computing