Set up scene information cache.

jumbatm / cloud-path-tracer

A toy online path tracing service.

GNU General Public License v3.0

0 stars 1 forks source link

Set up scene information cache. #5

Closed jumbatm closed 5 years ago

jumbatm commented 5 years ago

[x] Set up ~Amazon ElastiCache for Redis~ Azure Cache so that we can store data into it.
[x] Figure out what format scene information should be in.

peter2380123 commented 5 years ago

Due to student tier AWS being overly-strict on ElastiCache for Redis creation, we will resort to using Azure Cache for Redis (tested, deployment successful). Notes: Azure-for-students accounts has 100AUD credit. Using standard cache size (250MB) costs about 2.2cents/hr (see pricing here)

Here is the tutorial to connect Azure Cache for Redis to Node.js app.

jumbatm commented 5 years ago

Set up is pretty easily. Add it as a normal resource, and then you can access the Access Key to put data on. Cheapest tier is pretty solid, seems that it's interacted with through get and set async methods.

jumbatm commented 5 years ago

We decided not to close this until we confirm exactly what format to put into scene information in.

jumbatm commented 5 years ago

EC2 has health probes - periodically checks status

Information must goes into the scene information cache:

Scene information
(Maybe) which instances are running the render for the scene.

How to deal with individual RUs failing?

Idea: Wait for % of rendering units to complete. Then set a timeout for the remainder to complete, and give up. Use tile images as indicator of % complete. Hard limit a timeout on the rendering time for any rendering unit.
Possible alteration: Record which rendering units are allocated to a render. Then have collator check those (perhaps with SDK?). When it sees all are offline (for whatever reason), collator stitches the image, leaving black tiles for any missing tiles.
Final decision: We will check to see if checking EC2 status is possible. If not, we'll fall back to using tile images in bucket as the indicator of completion.

jumbatm commented 5 years ago

We can get instance status information: see https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/EC2.html#describeInstanceStatus-property

peter2380123 commented 5 years ago

Example of aws-cli returning instance states under specified (my) account: aws ec2 describe-instances --filters Name=instance-state-name,Values=running --query Reservations[].Instances[].State.Name Returns the following:

[
    "running"
]

Similarly, we can check by instance ID: aws ec2 describe-instances --instance-ids i-06fd9f6bf6343fd46 --query Reservations[].Instances[].State.Name

peter2380123 commented 5 years ago

This is now completed. Comments related to Phase 3 will be moved to ther respective issues.