search

nrdg / cloudknot

A python library to run your existing code on AWS Batch
https://nrdg.github.io/cloudknot/
Other
70 stars 17 forks source link

Problem getting started: "ck.Knot(...)" fails with 'Waiter StackCreateComplete failed' #205

Closed CarlKCarlK closed 4 years ago

CarlKCarlK commented 4 years ago

That for providing cloudknot.

I'm trying to run the 00 and 01 examples, but get this message.

~/anaconda3/lib/python3.7/site-packages/botocore/waiter.py in wait(self, **kwargs)
    321                     name=self.name,
    322                     reason='Waiter encountered a terminal failure state',
--> 323                     last_response=response,
    324                 )
    325             if num_attempts >= max_attempts:

WaiterError: Waiter StackCreateComplete failed: Waiter encountered a terminal failure state

I can't figure out how to figure out what is wrong. (There is nothing in the log. I set the logging level to logging.INFO.)

I'm running under an AWS administrator account that (on AWS) let me run little batch jobs.

My configuration may be new, but should work. It is Ubuntu under Windows. (Docker seems fine.). I've run Anaconda on Ubuntu under Windows routinely with other project.

Thanks for any suggestions you can provide.

welcome[bot] commented 4 years ago

👋 Thanks for opening your first issue here! We appreciate your help making Cloudknot better.

arokem commented 4 years ago

Hi Carl,

Thanks for checking in here. I believe that this is a duplicate of #155 and should be resolved on master. Did you install from pypi? If so, could you please upgrade to current master and give it a try?

On Sun, Apr 19, 2020 at 3:45 PM welcome[bot] notifications@github.com wrote:

👋 Thanks for opening your first issue here! We appreciate your help making Cloudknot better.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/richford/cloudknot/issues/205#issuecomment-616238059, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA46NTXJOZOXSEGSX53DUTRNN5JXANCNFSM4ML7PEGA .

CarlKCarlK commented 4 years ago

That worked. (Namely:

pip uninstall cloudknot pip install git+https://github.com/richford/cloudknot.git ). Thanks! Thanks!

arokem commented 4 years ago

On Sun, Apr 19, 2020 at 5:07 PM Carl Kadie notifications@github.com wrote:

That worked. (Namely:

pip uninstall cloudknot pip install git+https://github.com/richford/cloudknot.git ). Thanks! Thanks!

  • Carl

Awesome. I will close this issue. More below:

  • p.s. Minor thing I noticed: When I ran this from example_01:

result_futures = knot.map(range(10), env_vars=[{'name': 'MY_ENV_VAR', 'value': 'foo'}])

  1. It only ran on one machine. I assume that is some AWS batch configuration I need to change.

Batch tends to pack as many jobs as it can onto one machine. You can configure the instance type and memory requirements as key-word arguments to ck.Knot

  1. Instead of returning a list of one future, it returned just one future. I was surprised the return type (list vs singleton) seemed to depend on my AWS configuration, not my knot.map command. [If this seems interesting to track. Let me know and I'll post it as a new issue.]

Please do post an issue about this. I am curious what exactly you meant here.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/richford/cloudknot/issues/205#issuecomment-616248688, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA46NVWCDLO5U5XEZJOWRLRNOG53ANCNFSM4ML7PEGA .

arokem commented 4 years ago

Also, please note that we just moved the repo to https://github.com/nrdg/cloudknot, so do not be alarmed/confused by that.