Allow executing a script in userdata

[gianlucaborello]

Hi,

Not really an issue but more of a custom workflow problem.

So I use and love the Datastax AMI, and I launch it using a set of cloudformation templates, along with the other instances in my tiers. I use ansible in pull mode for configuring my instances, so what I do is embedding a two-line script in the userdata section, that will just install ansible-pull on my instance. Ansible will then take care of the rest of the configuration (for cassandra, it will do some minor things like setting the right seed node, increasing the JVM size, registering the IP address in Route 53).

Of course with the Datastax AMI I can't do this because the userdata is reserved to the various switches.

I understand this is not a big problem as I can find another way to do it (e.g. launching ansible from another machine once the deploy is done, or creating my own AMI), but I was looking if there's some "hidden" way (e.g. using a switch like "--base64-script" to be executed after cassandra gets installed) to do that with your AMI.

Thanks for the good work.

[joaquincasares]

Glad you like and use the AMI! Yeah, I'd be happy to hear what you have in mind.

If all you need is a --base64-script parameter and the ability to execute the command later, feel free to code up a pull request and I'll ensure that it doesn't mess with the rest of the AMI before accepting it.

One question before we head down that path. When would you need this script executed? Before Cassandra gets installed, after, or when it's already running? The first two would be the easiest as the latter, as I'm sure you're able to tell, is sometimes finicky for some EC2 disk-reasons I haven't yet tracked down. :)

Looking forward to hearing from you!

[gianlucaborello]

So I would say after Cassandra is installed is fine, as I need to instruct ansible to do something like "go in cassandra.yaml and populate the seed nodes with these values fetched from a boto script", so it's important that at this step all the configuration done by your code are all done (and all the Cassandra files are unpacked).

I'm pretty flexible though, I don't have any problem in complicating my custom script enough to make it work, what I really needed is just the ability of running some custom code at some point in your AMI.

I'll see if I can write some code, it sounds easy but it might take a while as I'm at full capacity :(

Thank you!

[joaquincasares]

Sure, np!

Would this line be sufficient? Or were you looking for some additional functionality that I might be missing?

logger.exe(base64.b64decode(options.base64script))

where options.base64script is your script?

[gianlucaborello]

I think you'll have to use the shell parameter, otherwise logger.exe will fail in executing a shell script as the example:

>>> subprocess.Popen("echo test1 && echo test2").communicate()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/subprocess.py", line 709, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1326, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
>>> subprocess.Popen("echo test1 && echo test2", shell=True).communicate()
test1
test2
(None, None)
>>>

And while we're there, base64 was the first suggestion that came to my mind, if there was actually a better way to pass this script (e.g. by using a properly formatted multi-line parameter) that would be great, so I could easily compare different versions of the cloudformation templates in git :)

[joaquincasares]

Thanks for the info on the shell parameter!

Sorry, I had already finished adding the current options before reading your request. Using backslashes may work, but if you wouldn't mind playing around with the parser and logger, that'd be helpful on my end.

Would you mind adding --forcecommit origin/dev-2.5 to your standard deployment along with --base64postscript <script> to see if that works for you?

If so, I'll merge 9c33d41f6d32639b94048739653692dbc1222d72 into the main 2.5 branch.

Cheers!

[gianlucaborello]

It seems like ec2 userdata is parsed before switching commit:

[ERROR] One of the options was not set correctly. Aborting installation.

Please verify your settings:
--forcecommit origin/dev-2.5 --base64postscript IyEvYmluL2Jhc2gNCmVjaG8gY2lhbyA+IC90bXAvdGVzdA0KZWNobyBjaWFvMiA+IC90bXAvdGVzdDINCg==
[ERROR] Exception seen in ds1_launcher.py:
Traceback (most recent call last):
  File "/home/ubuntu/datastax_ami/ds1_launcher.py", line 22, in initial_configurations
    ds2_configure.run()
  File "/home/ubuntu/datastax_ami/ds2_configure.py", line 990, in run
  File "/home/ubuntu/datastax_ami/ds2_configure.py", line 210, in parse_ec2_userdata
  File "/home/ubuntu/datastax_ami/ds2_configure.py", line 56, in exit_path
AttributeError

On a side note, I was just realizing it will be challenging to encode base64 in my cloudformation templates, mainly because my script will have to take at least one parameter from outside, so I need some kind of parameter replacement at cloudformation launch time.

I can see if there's something I can do to encode it on the fly.

[joaquincasares]

Thanks for pointing that out. I turns out there is a bug with the forcecommit code, but I'll just patch that later with a new AMI release.

I've moved the base64postscript code to the normal 2.5 branch, so you should have it now. It doesn't seem like output is being captured correctly, but hopefully that's enough to get you started in the right direction.

Cheers!

[gianlucaborello]

Hi,

Sorry to get back to you so late but I've been quite busy. I have another error:

EXEC] 03/03/14-17:37:57 sudo rm -rf /etc/motd
[EXEC] 03/03/14-17:37:57 sudo touch /etc/motd
[INFO] Started with user data set to:
[INFO] --base64postscript IyEvYmluL2Jhc2gNCmVjaG8gY2lhbyA+IC90bXAvdGVzdA0KZWNobyBjaWFvMiA+IC90bXAvdGVzdDINCg==
[INFO] Using instance type: c3.large
[INFO] meta-data:instance-type: c3.large
[INFO] meta-data:local-ipv4: 10.45.1.144
[INFO] meta-data:public-hostname: ec2-107-22-75-156.compute-1.amazonaws.com
[INFO] meta-data:ami-launch-index: 0
[INFO] meta-data:reservation-id: r-c3e1d7e3
[ERROR] Exception seen in ds1_launcher.py:
Traceback (most recent call last):
  File "/home/ubuntu/datastax_ami/ds1_launcher.py", line 22, in initial_configurations
    ds2_configure.run()
  File "/home/ubuntu/datastax_ami/ds2_configure.py", line 997, in run
  File "/home/ubuntu/datastax_ami/ds2_configure.py", line 221, in parse_ec2_userdata
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

IIRC it used to work without providing any parameter (which of course I don't do in my production environment but just now for the sake of testing).

Thank you!

[joaquincasares]

No worries. We all are. :)

That should error out in a better fashion, but because you used the --base64postscript option you will need to also add the --version and --totalnode parameters.

I'll hopefully provide better error handling there soon.

[gianlucaborello]

I'm running with:

--clustername mytest --totalnodes 1 --version community --base64postscript IyEvYmluL2Jhc2gNCmVjaG8gInRlc3QxIg0KZWNobyAidGVzdDIi

Where the encoded script is:

#!/bin/bash
echo "test1"
echo "test2"

But in ami.log I see:

[EXEC] 03/04/14-00:34:15 sudo service ntp start:
 * Starting NTP server ntpd
   ...done.

[INFO] base64postscript response:

[INFO] ds2_configure.py completed!

And when I login, the motd contains:

These notices occurred during the startup of this instance:
[ERROR] Scripts using a non-signed commit. Please ensure commit is valid.
[ERROR]     If it was a missed signature, feel free to open a ticket at https://github.com/riptano/ComboAMI.

Thanks

[joaquincasares]

Yeah, I'm not sure why the response is blank, but apparently it's executed. Could you try perhaps touching a file, or changing a configuration?

As far as the motd, it's because I merged a commit straight from Github. I've added a trivial signed commit just right now, so it should clear things up.

Thanks for letting me know!

[gianlucaborello]

It seems to work, thanks so far, I'll see if there's a way to specify the script in plain text so that I can make version control more useful in this case.

Thanks!

[joaquincasares]

Glad to hear! I'll close this issue, but feel free to open another one.

Cheers!