provided user data is not executable (detected type: text/plain; charset=utf-8)

[dpreble-cisco]

Hello, I am seeing this error in the following file: /var/log/amazon/ec2/ec2-macos-init.log

I do see a file written to disk located at: /usr/local/aws/ec2-macos-init/instances/<instance-id>/ named: userdata (No extension).

I can take this script and execute it directly and it will run to completion without failure. Additionally, I have added another module to the init.toml after copying the userdata script to another location. This module runs my script successfully without issue. (I would use the same location but it gets overwritten as new userdata is pulled freshly from Amazon for each run it seems.).

Please let me know what I can do to help troubleshoot this issue.

[dpakpak]

Hi @dpreble-cisco, thanks for opening this issue to address this unexpected behavior you have been seeing.

Could you provide some more information to help us troubleshoot this issue? Some useful information would be:

• AMI ID you used to launch the instance
• What method you used to launch the instance with user-data
• What user-data is being included (please scrub any sensitive data)
• Output of sudo /usr/local/bin/ec2-macos-init version
• The module you added to copy the userdata file
• How the user-data is being manually executed

[dpreble-cisco]

Thanks for getting back to me. I am executing on two different instance families/ami IDs so I will answer for each separately using mac1.metal and mac2.metal

mac1.metal

---

• AMI ID: ami-0665a0b2ea8636d1d
• Launch Method: Auto-Scaling Group using Dedicated host provisioning based on the following method https://aws.amazon.com/blogs/compute/implementing-autoscaling-for-ec2-mac-instances/
• User Data: I'd rather not provide this immediately. I can take a user data script you provide and attempt to run the process with it.
• Version output:

  EC2 macOS Init
  Version: 1.5.4 [2022-10-18 10:48:47 -0700]
  https://github.com/aws/ec2-macos-init
  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

• The module you added to copy the userdata file: My understanding is the init.toml is already configured based on this AMI provided by amazon to execute a UserData block. The block I find in my unmodified init.toml is

  
  ### Group 5 ###
  ## Finally, run user data.

Attempt to execute userdata, if provided

[[Module]] Name = "ExecuteUserData" PriorityGroup = 5 # Fifth group RunPerInstance = true # Run once per instance FatalOnError = false # Best effort, don't fatal on error [Module.UserData] ExecuteUserData = true # Execute the userdata

I modified the `init.toml` by adding the following block

[[Module]] Name = "TestUserData" PriorityGroup = 4 # Second group RunPerBoot = true FatalOnError = false # Best effort, don't fatal on error [Module.Command] Cmd = ["/bin/sh", "-c", "/Users/ec2-user/userdata.sh"]

Then I run the following commands:

ec2-macos-init clean -all ec2-macos-init run

I can then see the successful execution of my script reflected in the log located at: `/var/log/amazon/ec2/ec2-macos-init.log`

* **How the user-data is being manually executed:**

cp /usr/local/aws/ec2-macos-init/instances/i-080313263a0577882/userdata . cp userdata userdata.sh ./userdata.sh >> userdata_exec.log & tail -f userdata_exec.log

### mac2.metal
---
*  **AMI ID:** ami-013846afc111c94b0
*  **Launch Method:** Same as **mac1.metal**
*  **User Data:** Same as **mac1.metal**
*  **Version output:** Same as **mac1.metal**
* **The module you added to copy the userdata file:** N/A, I didn't test this method on the mac2.metal yet.
* **How the user-data is being manually executed:** Same as **mac1.metal**

[thimslugga]

@dpreble-cisco

Make sure the userdata script starts with either #!/bin/bash or #!/bin/zsh. The executeCommand function that is used by the UserData module attempts to detect the content mimetype and the underlying go/http implementation is somewhat limited. The test you've implemented which uses the Command module does not attempt to perform this same type of check.

https://github.com/aws/ec2-macos-init/blob/9460c024ac68de3bfca4e78102a691b4b7f1a22e/lib/ec2macosinit/userdata.go#L65

out, err := executeCommand([]string{userDataFile}, "", []string{})
    if err != nil {
        if strings.Contains(err.Error(), "exec format error") {
            contentType := http.DetectContentType([]byte(ud))

• https://pkg.go.dev/net/http#DetectContentType
• https://mimesniff.spec.whatwg.org/

[dpakpak]

@dpreble-cisco

I tried to recreate this issue using the same information that you provided, but I was not able to. I used the following script for my userdata:

#!/bin/bash
whoami > /tmp/userdata.log

I was able to test that the userdata works as expected.

ec2-user ~ %  cat /tmp/userdata.log 
root

I believe the problem could be in the userdata you are using, like the other comment mentioned. Could you try running with this userdata and let me know if this works? Please let me know if this solution will work for you. Thank you!

[dpreble-cisco]

@thimslugga Hi! Sorry for the late reply, testing this is a difficult process due to the boot time of mac os ec2 instances!

This was indeed the issue, as I had the correct directive but my templating of the script resulted in an empty line at the top of the file. Sorry for such a simple issue!

Once this resolved the issue I ran into other issues as the userData script executes as root. It's all working now, thanks for your help!

In regards to iteration time, is it possible to edit the userData without incurring the cost of booting a new mac ec2 instance? My current iteration workflow is to update my launch template with my new user data, scale up a new instance to test the user data's success. This takes at least 40+ minutes. Of course, I try to use the instance to debug as much as I can, but it seems there are differences between calling the Module.UserData and Module.Command. If you directly edit the userdata file located under /usr/local/... whenever you re-execute the init process with ec2-macos-init run it downloads the userData from what is configured on the instance. I can only change this if I reboot which takes a significant amount of time for mac instances and cannot be done for auto-scaling as it will timeout and a new instance will be allocated. Any help reducing the iteration time here would be greatly appreciated!