Getting SCP error when building HELK environment

[Johndpete316]

Issue

Unable to copy public key to velocihelk machine, consistently getting the same error when running this with --helk enabled. Tested on multiple different machines.

Process to reproduce

python3 ad.py --domain_controller --ad_domain MrRobot.local --admin xyz --password xyz! --ad_users 50 --endpoints 3 --location eastus --domain_join --helk --resource_group testing-helk

Output during terraform apply:

Seeing this status pop up a few times during creation . At this point it has yet to stop the process and error out.

null_resource.velociraptor-config_upload1[0] (local-exec): Executing: ["cmd" "/C" "sleep 60"]
null_resource.velociraptor-config_upload1[0] (local-exec): 'sleep' is not recognized as an internal or external command,
null_resource.velociraptor-config_upload1[0] (local-exec): operable program or batch file

After letting it run for ~20 minutes I ended the process manually, here is the output from that:

╷
│ Error: waiting for creation of Linux Virtual Machine: (Name "velocihelk" / Resource Group "testing-helk-91nxi"): Code="RetryableError" Message="A retryable error occurred."
│
│   with azurerm_linux_virtual_machine.vh_vm,
│   on helk.tf line 53, in resource "azurerm_linux_virtual_machine" "vh_vm":
│   53: resource "azurerm_linux_virtual_machine" "vh_vm" {
│
╵
╷
│ Error: local-exec provisioner error
│
│   with null_resource.velociraptor-config_upload1[0],
│   on win10-1.tf line 138, in resource "null_resource" "velociraptor-config_upload1":
│  138:   provisioner "local-exec" {
│
│ Error running command 'sleep 60': exit status 1. Output: 'sleep' is not recognized as an internal or external command,
│ operable program or batch file

Tried running the apply one more time. Just noticing this output in the terminal.

null_resource.helk-scp-velociraptor-config: Provisioning with 'remote-exec'...
null_resource.helk-scp-velociraptor-config (remote-exec): Connecting to remote host via SSH...
null_resource.helk-scp-velociraptor-config (remote-exec):   Host: 20.185.36.176
null_resource.helk-scp-velociraptor-config (remote-exec):   User: helk
null_resource.helk-scp-velociraptor-config (remote-exec):   Password: false
null_resource.helk-scp-velociraptor-config (remote-exec):   Private key: true
null_resource.helk-scp-velociraptor-config (remote-exec):   Certificate: false
null_resource.helk-scp-velociraptor-config (remote-exec):   SSH Agent: false
null_resource.helk-scp-velociraptor-config (remote-exec):   Checking Host Key: false
null_resource.helk-scp-velociraptor-config (remote-exec):   Target Platform: unix
null_resource.helk-scp-velociraptor-config (remote-exec): Connected!
azurerm_windows_virtual_machine.azurerm-vm-win10-1: Still creating... [50s elapsed]
azurerm_windows_virtual_machine.azurerm-vm-win10-3: Still creating... [50s elapsed]
null_resource.helk-scp-velociraptor-config (remote-exec): Hello World
null_resource.helk-scp-velociraptor-config: Provisioning with 'local-exec'...
null_resource.helk-scp-velociraptor-config (local-exec): Executing: ["cmd" "/C" "scp -o StrictHostKeyChecking=no -i ./ssh_key.pem helk@20.185.36.176:/home/helk/config.yaml ./files/Velociraptor.config.yaml"]
null_resource.helk-scp-velociraptor-config (local-exec): 'scp' is not recognized as an internal or external command,

I would like to put emphasis on this line

null_resource.helk-scp-velociraptor-config (local-exec): 'scp' is not recognized as an internal or external command,

I confirmed the machine I am running the commands from does have SCP (pretty sure this wasn't a requirement just wanted to double check.

I manually ended the procss again, this time I am seeing this error reguarding running the following command. (

│ Error: local-exec provisioner error
│
│   with null_resource.helk-scp-velociraptor-config,
│   on helk.tf line 124, in resource "null_resource" "helk-scp-velociraptor-config":
│  124: provisioner "local-exec" {
│
│ Error running command 'scp -o StrictHostKeyChecking=no -i ./ssh_key.pem helk@20.185.36.176:/home/helk/config.yaml ./files/Velociraptor.config.yaml': exit
│ status 1. Output: 'scp' is not recognized as an internal or external command,
│ operable program or batch file.

Complete output

logoutput.txt

As always if more details are needed let me know

P.S. shoot me a follow back on twitter, would love to shoot you a couple bucks for a drink

[iknowjason]

Hey @Johndpete316

Thanks for sharing. I'm actually out of the country now here in Germany, so my response might be spotty.

So this definitely looks like an issue with scp running on the same system running terraform. What OS are you running this from? What happens when you just type scp? What output do you get?

Yes, I should have mentioned that scp was a requirement. Assumed that most users would be running from Linux or MacOS. I believe with a powershell prompt in windows you can run scp by default too. Just let me know. The local-exec provisioner error means it is coming from the local machine. Also odd about sleep as sleep should be included in Linux. To make it easier if you can just try running all of your builds from stock Ubuntu Linux (i used 20.04) you should be gtg.

Thanks on the offer for a beer. I followed you back. I noticed you are in/from Michigan. Born in Detroit and spent several years in Ann Arbor.

[iknowjason]

@Johndpete316

Also, I have on my ToDo list to completely refactor the code for Velociraptor and HELK, getting rid of the local-exec provisioner and Ansible playbooks. Everything will be implemented with user-data and bash, making it much more reliable. I have an open issue to get this done. I hope you can get around your current issues as I'm still a couple of weeks away from starting this:

https://github.com/iknowjason/PurpleCloud/issues/12

[Johndpete316]

Confirmed issue is on our side. When switching over to a Ubuntu 20.04 local-exec was able to scp into remote machine with no issue.

Thank you for taking the time to look into it!