Open papaneeds opened 11 months ago
crw
commented
11 months ago Thanks for this bug report, @papaneeds. While a maintainer may weigh in on the issue, I'll note that provisioners are not receiving active development, per https://github.com/hashicorp/terraform/blob/main/.github/CONTRIBUTING.md#provisioners.
This looks like a fairly sophisticated use case, you may also find help from the community forum.
Thanks!
mattlqx
commented
5 months ago I found this to be a manifestation of what is described in this SO post. In general, processes spawned from sshd are cleaned up on disconnection.
The workaround described in it however did work for me: using Cygwin's start nohup to call the script. One caveat was that I had to sleep long enough after calling the script to prevent disconnection until the long-running process was spawned.
Terraform Version
Terraform Configuration Files
Debug Output
Expected Behavior
The powershell scripts looper1.ps1 and looper2.ps1 should have been run as separate processes on the EC2 Windows instance.
Actual Behavior
The powershell scripts looper1.ps1 and looper2.ps1 do not appear to run at all.
Steps to Reproduce
Run "terraform apply"
Additional Context
I am trying to execute a powershell script as a separate process on an EC2 windows instance using "remote-exec". However, terraform does not seem to be able to launch the powershell script as a separate process - the powershell script does not run.
To illustrate the problem I have created two long-running powershell scripts called "looper1.ps1" and "looper2.ps1" which simply loop forever and write to log files in C:\Users\Administrator" every 5s (please see scripts below. I can tell whether the scripts are running or not by the presence (or absence) of the log files (called "Looper1.log" and "Looper2.log" respectively.
In the terraform snippet from my "main.tf" (which is included above) I try to execute the "looper1.ps1" script as a separate process on the EC2 windows instance using remote-exec and the command:
start /b "looper1" "powershell.exe" "C:\Users\Administrator\looper1.ps1"
I alsoI try to execute the "looper2.ps1" script as a separate process on the EC2 windows instance using remote-exec and the command:
powershell.exe Start-Process -FilePath "powershell" -ArgumentList C:\Users\Administrator\looper2.ps1
In the terraform debug output it looks like both of these commands execute successfully (they both exit with code 0). However, when I subsequently log in to the EC2 windows instance I do not see the log files "Looper1.log" and "Looper2.log" (which should be updating every 5s if looper1.ps1 and looper2.ps1 are running), nor do I see any evidence that they are running if I execute a "tasklist" command.
When I am logged in to the EC2 instance, if I open a cmd window and manually issue the command:
start /b "looper1" "powershell.exe" "C:\Users\Administrator\looper1.ps1"
then I immediately see the "Looper1.log" file created and updated every 5s. Issuing a "tasklist" command shows the presence of a new powershell.exe process belonging to "looper1.ps1".
In the same cmd window if I manually issue the command:
powershell.exe Start-Process -FilePath "powershell" -ArgumentList C:\Users\Administrator\looper2.ps1
Then a new powershell window opens, the command returns control to the cmd window, and I immediately see the "Looper2.log" file created and updated every 5s. Issuing a "tasklist" command shows the presence of a new powershell process belonging to "looper2.ps1".
So, it seems that I can manually issue these commands and start the powershell scripts as separate processes, but issuing the same commands through terraform "remote-exec" does nothing.
I have also tried starting the powershell scripts from terraform using "remote-exec" but not as separate processes (ie; in the foreground). The relevant commands are:
start "looper1" "powershell.exe" "C:\Users\Administrator\looper1.ps1"
and
powershell.exe C:\Users\Administrator\looper2.ps1
The first command is (start "looper1"...) is executed by terraform and returns with exit code 0. However, as before, it does not seem to do anything on the EC2 windows instance (in the sense that when I log in to the remote EC2 windows instance there is no "Looper1.log" file created), and the "looper1.ps1" powershell script does not appear to be running on the EC2 windows instance.
When terraform hits the second command (powershell.exe C:\Users\Administrator\looper2.ps1) terraform executes it on the remote EC2 windows instance and then waits until the command finishes executing (which means that it waits forever because the powershell script is running in the foreground and loops forever and never returns). When I log in to the EC2 windows instance I do see the presence of the "Looper2.log" file and I do see a powershell process running in tasklist associated with "looper2.ps1". So, it seems like terraform is able to run "looper2.ps1" in the foreground (but of course, terraform never returns and keeps running forever too!).
So, it looks like, for some reason, terraform is not successful launching the powershell scripts as separate processes using "remote-exec" (even though I can issue exactly the same commands in a cmd window on the EC2 instance to successfully launch the powershell scripts as separate processes).
You may ask why I want to run the powershell scripts as separate processes in the first place. It is because I the powershell scripts are executing and monitoring long-running installation processes and I want to build several EC2 windows instances in parallel (and not serially).
And, finally, I have been successful launching the powershell scripts as separate processes upon EC2 instance instantiation by embedding the powershell scripts in the AMI and then using EC2Launch (and properly configuring agent-config.yml). However, embedding the powershell scripts in the AMI and using EC2Launch is a brittle and error-prone workaround. I would much rather do it flexibly using terraform and "remote-exec" - it just doesn't seem to work.
this is looper1.ps1
$looperLogFile = "C:\Users\Administrator\Looper1.log" $sleepTime = 5 while ($true) { Add-Content -Path $looperLogFile -Value "Looper1. Waiting $sleepTime." Start-Sleep -s $sleepTime }
this is looper2.ps2
$looperLogFile = "C:\Users\Administrator\Looper2.log" $sleepTime = 5 while ($true) { Add-Content -Path $looperLogFile -Value "Looper2. Waiting $sleepTime." Start-Sleep -s $sleepTime }
References
No response