Open agile-jtdressel opened 3 years ago
Hey @agile-jtdressel, thanks for opening an issue to address these unexpected results you've been seeing.
My first guess as to why you might be seeing this behavior is that the userdata
file on the instance got spliced when the UserDataModule
ran. This probably happened due to the local edits to userdata
while the original userdata
tied to the instance remained unchanged. The UserDataModule
then downloaded the original userdata
and wrote it on top of the local changes that were made. But if this is the case then I'm curious as to why date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt-4.txt
was not overwritten with date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt
.
One thought I had is that we can prevent this by adding another option to the Execute-User-Data
module to control the overwrite behavior. This option would toggle between overwriting with the latest and never overwriting. Please let me know your thoughts on this option and if it's something you think would be useful.
This is all just initial speculation based on the information you've provided. The issue is going to require a bit more investigation on our side in order to determine what's actually going on. We'll keep this issue updated as we learn more about the problem.
Also, please post any new data, comments, and/or questions you have along the way :).
My first guess as to why you might be seeing this behavior is that the userdata file on the instance got spliced when the UserDataModule ran. This probably happened due to the local edits to userdata while the original userdata tied to the instance remained unchanged. The UserDataModule then downloaded the original userdata and wrote it on top of the local changes that were made. But if this is the case then I'm curious as to why date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt-4.txt was not overwritten with date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt.
I wish I saved the exact file (or dumped the hex values with xxd
), I wonder if there's a weird end-of-file character issue here. Granted, I don't know how userdata
is encoded, but I would have expected something like date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt.txt
based on the number of characters
date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt # original userdata
date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch-4.txt # manually updated userdata
date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt-4.txt # actual result modifed by ec2-macos-init
date -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt.txt # what I would have expected based on the number of characters.
One thought I had is that we can prevent this by adding another option to the Execute-User-Data module to control the overwrite behavior. This option would toggle between overwriting with the latest and never overwriting. Please let me know your thoughts on this option and if it's something you think would be useful.
This would be tremendously useful to me. I was also toying with the idea of adding an option with a path to an arbitrary userdata.
Thanks for your feedback about that idea! I'll make a note of the possible option while we look to resolve this issue.
I was also toying with the idea of adding an option with a path to an arbitrary userdata.
That's an interesting idea. What would your use case look like for this option?
Thanks for your feedback about that idea! I'll make a note of the possible option while we look to resolve this issue.
I was also toying with the idea of adding an option with a path to an arbitrary userdata.
That's an interesting idea. What would your use case look like for this option?
I should note an assumption: that specifying the path would disable overwriting userdata.
It would be very similar to setting the Overwrite = never
/ Overwrite = always
that you described. It would make it easier to experiment on a non-ec2 system. Being able to specify the path would make it easier to experiment on non-ec2 systems.
Right now, I think I could experiment with userdata in a local VM (or just on my development MacBook) by faking the metadata service and hosting a simple server on 169.254.169.254. If I could configure ec2-macos-init
to read userdata from an arbitrary existing file, I could save some time spinning up a server and configuring it to serve my desired userdata.
/usr/local/aws/ec2-macos-init/init.toml
:
[[Module]]
Name = "Execute-User-Data"
PriorityGroup = 4 # Fourth group
RunPerInstance = true # Run once per instance
UserdataPath = /Users/jtdressel/ci_template/userdata_a
FatalOnError = false # Best effort, don't fatal on error
[Module.UserData]
ExecuteUserData = true # Execute the userdata
Thanks for detailing the information about what that option might look like. Please do create a separate issue where we can discuss this option and your use case further.
For what it is worth, for my purposes I just added another module at the end:
[[Module]]
Name = "BogusUserData"
PriorityGroup = 4 # Second group
RunPerBoot = true
FatalOnError = false # Best effort, don't fatal on error
[Module.Command]
Cmd = ["/bin/zsh", "-c", "/usr/local/aws/ec2-macos-init/userdata"]
and iterated on that in place. Maybe this will be helpful for future explorers!
I was experimenting with the capabilities of the
Execute-User-Data
module, and I did not want to wait for the dedicated host to be wiped between invocations./usr/local/aws/ec2-macos-init/instances/i-0a9977235e17ff7f1/userdata
/usr/local/aws/ec2-macos-init/init.toml
to only run theExecute-User-Data
modulesudo ec2-macos-init run
What I expected:
I expected
ec2-macos-init
to either run my newly editeduserdata
or to run the originaluserdata
.What happened:
ec2-macos-init
seems to have merged the originaluserdata
with my editeduserdata
, and then ran the merged userdata.Versions
Details
I started with the below userdata, entered in the aws web console.
This worked pretty much how I expected.
/tmp/launch.txt
and/Users/Shared/launch.txt
both existed and had reasonable timestamps in them. I don't recall, but I think/Users/ec2-user/Desktop/launch.txt
did not exist after the first run. (I suspect it ran before the Desktop directory was created. )I wanted to try a few edits of
userdata
to see how they were executed. (see #6 ). I had edited/usr/local/aws/ec2-macos-init/init.toml
because some of the initial tasks failed. I believe those failed because I had changed the password forec2-user
./usr/local/aws/ec2-macos-init/instances/i-0a9977235e17ff7f1/history.json
.userdata
writeablesudo chmod 755 userdata
userdata
, then I made it read only andcat
ed to verify my changes.Afterward, I looked at the contents of
userdata
, and was surprised to see it was a mix of my original and editeduserdata
. Note thedate -u +"%Y-%m-%dT%H:%M:%SZ" >> /Users/ec2-user/Desktop/launch.txt-4.txt
line, which is not in either the original or editeduserdata
.The combined contents were executed.
/tmp/launch.txt
,/Users/Shared/launch.txt
,/Users/ec2-user/Desktop/launch.txt-4.txt
andtouch /tmp/last-line
were all either created or updated.