jamesyonan / brenda

Blender render farm software for Amazon Web Services
Other
245 stars 67 forks source link

Is there a way to specify an availability zone? #30

Closed sundriftproductions closed 7 years ago

sundriftproductions commented 7 years ago

I'm using the Brenda console script (https://github.com/rider-rebooted/win_brenda_console ) to run Brenda. I've successfully run tests with the Brenda console script in the past. I stopped using it several months ago but now am ready to start running it again, this time with a real project.

Unfortunately, I am having difficulty acquiring a reasonably-priced instance because the Brenda console, Brenda itself, or AWS is always picking an expensive availability zone that I don't want, and thus I can never get my render started.

I'm in US-East (Virginia). When I check prices on the Brenda console, I get something that looks like this:

Spot price data for instance c1.xlarge us-east-1a 2017-07-22T09:28:44.000Z $0.106 us-east-1b 2017-07-22T10:14:18.000Z $0.106 us-east-1c 2017-07-22T13:00:14.000Z $0.1075 us-east-1d 2017-07-22T09:28:44.000Z $5.2 us-east-1e 2017-07-22T09:28:44.000Z $0.1066

price data for instance c3.large us-east-1a 2017-07-22T12:28:47.000Z $0.0156 us-east-1b 2017-07-22T12:28:47.000Z $0.0158 us-east-1c 2017-07-22T12:28:47.000Z $0.0159 us-east-1d 2017-07-22T12:28:47.000Z $1.05 us-east-1e 2017-07-22T13:31:02.000Z $0.016 us-east-1f 2017-07-22T12:28:47.000Z $1.05

Spot price data for instance c3.xlarge us-east-1a 2017-07-22T09:00:40.000Z $0.0428 us-east-1b 2017-07-22T13:04:01.000Z $0.0426 us-east-1c 2017-07-22T12:16:19.000Z $0.0427 us-east-1d 2017-07-22T07:12:25.000Z $2.1 us-east-1e 2017-07-22T11:42:37.000Z $0.0436 us-east-1f 2017-07-22T07:12:25.000Z $2.1

Spot price data for instance c3.2xlarge us-east-1a 2017-07-22T13:39:42.000Z $0.1049 us-east-1b 2017-07-22T13:41:00.000Z $0.1047 us-east-1c 2017-07-22T13:20:00.000Z $0.1047 us-east-1e 2017-07-22T13:41:16.000Z $0.1092

I'll pick one instance of c3.large, for, say, $0.10 or $0.20, which -- based on the prices under c3.large, you'd think it would start up an instance. Instead, in the AWS console I get the status of "capacity-not-available" and there it sits forever because, according to the console, the job is scheduled to run on us-east-1d -- the most expensive availability zone ($1.05) for an instance. I want to run on any availability zone other than us-east-1d but can't figure out how to change it. Even when I choose a different instance type, like c3,xlarge, it still defaults to run on us-east-1d -- the most expensive option and way above my bid price.

I'm starting to pick apart what the Brenda console script is doing, and it looks like it's sending a command like this to Brenda:

python brenda-run -i c3.large -N 1 -p 0.20 spot

I looked through the Brenda documentation but didn't see a way to specify, say, us-east-1a as the availability zone. Is that possible? If so, what would be the syntax to do that?

If that's not possible, is there a way to set up things in AWS so that it defaults to using us-east-1a?

sundriftproductions commented 7 years ago

Never mind -- I figured it out. The solution is here: https://stackoverflow.com/questions/45258778/aws-ec2-spot-market-run-instance-on-a-particular-availabilty-zone

BelleNottelling commented 7 years ago

You mean like https://github.com/jamesyonan/brenda/pull/18 I've merged this PR into my fork

sundriftproductions commented 7 years ago

Hah! I didn't notice that. Oh well, I wrote my own code that does the same thing (haven't uploaded it to GitHub yet).