Closed haffa008 closed 2 years ago
I am having some issues above. logger: The Splunk web interface is at https://logger:8000 logger: logger: Init script installed at /etc/init.d/splunk. logger: Init script is configured to run at boot. logger: [05:50:20]: Downloading Palantir osquery configs... logger: Cloning into 'osquery-configuration'... logger: [05:50:21]: Installing Fleet... logger: 127.0.2.1 logger logger logger: mysql: [Warning] Using a password on the command line interface can be insecure. logger: No URLs found in -. logger: unzip: cannot find or open fleet.zip, fleet.zip.zip or fleet.zip.ZIP. logger: cp: cannot stat 'fleet/linux/fleetctl': No such file or directory logger: cp: cannot stat 'fleet/linux/fleet': No such file or directory logger: /tmp/vagrant-shell: line 301: fleet: command not found logger: cp: target '/opt/fleet/' is not a directory logger: Created symlink /etc/systemd/system/multi-user.target.wants/fleet.service → /etc/systemd/system/fleet.service. logger: [05:50:22]: Waiting for fleet service to start...
vagrant@logger:~$ service fleet status ● fleet.service - fleet Fleet Loaded: loaded (/etc/systemd/system/fleet.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Wed 2022-03-09 07:51:51 UTC; 7s ago Process: 66651 ExecStart=/usr/local/bin/fleet serve --mysql_address=127.0.0.1:3306 --mysql_database=fleet --mysq> Main PID: 66651 (code=exited, status=203/EXEC)
Mar 09 07:51:51 logger systemd[1]: Started fleet Fleet. Mar 09 07:51:51 logger systemd[66651]: fleet.service: Failed to execute command: No such file or directory Mar 09 07:51:51 logger systemd[66651]: fleet.service: Failed at step EXEC spawning /usr/local/bin/fleet: No such fil> Mar 09 07:51:51 logger systemd[1]: fleet.service: Main process exited, code=exited, status=203/EXEC Mar 09 07:51:51 logger systemd[1]: fleet.service: Failed with result 'exit-code'.
Same here. Looks similar to #604
Same here. Looks similar to #604
Yup! Looks to be a broken download URL / filename call in the script for Fleet. @clong Please advise.
Vagrant/logger_bootstrap.sh L294
Quick fix:
# fleetctl
wget -O fleetctl.zip https://github.com/fleetdm/fleet/releases/download/fleet-v4.11.0/fleetctl_v4.11.0_linux.zip
unzip fleetctl.zip
cp fleetctl_v4.11.0_linux/fleetctl /usr/local/bin/fleetctl && chmod +x /usr/local/bin/fleetctl
# fleet
wget -O fleet.tgz https://github.com/fleetdm/fleet/releases/download/fleet-v4.11.0/fleet_v4.11.0_linux.tar.gz
tar xfvz fleet.tgz
cp fleet_v4.11.0_linux/fleet /usr/local/bin/fleet && chmod +x /usr/local/bin/fleet
# create fleet dir for config files
mkdir fleet
Vagrant/logger_bootstrap.sh L294
Quick fix:
# fleetctl wget -O fleetctl.zip https://github.com/fleetdm/fleet/releases/download/fleet-v4.11.0/fleetctl_v4.11.0_linux.zip unzip fleetctl.zip cp fleetctl_v4.11.0_linux/fleetctl /usr/local/bin/fleetctl && chmod +x /usr/local/bin/fleetctl # fleet wget -O fleet.tgz https://github.com/fleetdm/fleet/releases/download/fleet-v4.11.0/fleet_v4.11.0_linux.tar.gz tar xfvz fleet.tgz cp fleet_v4.11.0_linux/fleet /usr/local/bin/fleet && chmod +x /usr/local/bin/fleet # create fleet dir for config files mkdir fleet
Awesome stuff @mrpew Thanks for that quick fix! It worked for me!
All seems to be good as per the Post build checks script!
PS C:\Users......\Documents\DetectionLab\Vagrant> .\post_build_checks.ps1 [*] Verifying that Splunk is reachable... [ √ ] Splunk is running and reachable!
[*] Verifying that Fleet is reachable... [ √ ] Fleet is running and reachable!
[*] Verifying that Microsoft ATA is reachable... [ √ ] Microsoft ATA is running and reachable!
[*] Verifying that Velociraptor is reachable... [ √ ] Velociraptor is running and reachable!
[*] Verifying that Guacamole is reachable... [ √ ] Guacamole is running and reachable!
@mrpew @clong everything seems fine now except that I don't have any devices reporting into the Fleet.
I noticed that /usr/bin/yq is erroring out in applying the enrollment secret with the below error during the "logger" setup:
" logger: mysql: [Warning] Using a password on the command line interface can be insecure. logger: Updated enrollment secret logger: Error: unknown command "w" for "yq" logger: Run 'yq --help' for usage. logger: Error: unknown command "w" for "yq" logger: Run 'yq --help' for usage. logger: Error: no spec field on "" document "
Did you also encounter this error by any chance?
Use "yq [command] --help" for more information about a command. vagrant@logger:~$ /usr/bin/yq -i '/tmp/options.yaml' Error: unknown command "/tmp/options.yaml" for "yq" Run 'yq --help' for usage. vagrant@logger:~$ /usr/bin/yq -i "/tmp/options.yaml" Error: unknown command "/tmp/options.yaml" for "yq" Run 'yq --help' for usage. vagrant@logger:~$ /usr/bin/yq -i </tmp/options.yaml> -bash: syntax error near unexpected token `newline' vagrant@logger:~$ /usr/bin/yq -i 'spec.config.options.enroll_secret= ""enrollmentsecret""' Error: unknown command "spec.config.options.enroll_secret= \"\"enrollmentsecret\"\"" for "yq" Run 'yq --help' for usage. vagrant@logger:~$ /usr/bin/yq -i 'spec.config.options.enroll_secret' 'enrollmentsecret' Error: unknown command "spec.config.options.enroll_secret" for "yq" Run 'yq --help' for usage.
Not able to figure out the syntax for "yq" to commit the config to YAML file!!
Just pushed a fix. Update git and try again?
Did not work @clong
logger: --2022-03-11 10:33:29-- https://github.com/fleetdm/fleet/releases/download/fleet-v4.11.0/fleet_v4.11.0_linux.tar.gz
logger: Resolving github.com (github.com)... 13.234.176.102
logger: Connecting to github.com (github.com)|13.234.176.102|:443... connected.
logger: HTTP request sent, awaiting response... 302 Found
logger: Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/309820286/38fd4833-7775-4071-a138-3a5c5a1714fd?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220311%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220311T103330Z&X-Amz-Expires=300&X-Amz-Signature=d50522864b2d7ce0d6f20f46f8909ed3c8dfb811d3595a14f08c91273c5db420&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=309820286&response-content-disposition=attachment%3B%20filename%3Dfleet_v4.11.0_linux.tar.gz&response-content-type=application%2Foctet-stream [following]
logger: --2022-03-11 10:33:29-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/309820286/38fd4833-7775-4071-a138-3a5c5a1714fd?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220311%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220311T103330Z&X-Amz-Expires=300&X-Amz-Signature=d50522864b2d7ce0d6f20f46f8909ed3c8dfb811d3595a14f08c91273c5db420&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=309820286&response-content-disposition=attachment%3B%20filename%3Dfleet_v4.11.0_linux.tar.gz&response-content-type=application%2Foctet-stream
logger: Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.111.133, ...
logger: Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
logger: HTTP request sent, awaiting response... 200 OK
logger: Length: 28278881 (27M) [application/octet-stream]
logger: Saving to: ‘fleet_v4.11.0_linux.tar.gz’
logger:
fleet_v4.11.0_linux 100%[===================>] 26.97M 3.49MB/s in 7.7s logger: logger: 2022-03-11 10:33:38 (3.50 MB/s) - ‘fleet_v4.11.0_linux.tar.gz’ saved [28278881/28278881] logger: logger: FINISHED --2022-03-11 10:33:38-- logger: Total wall clock time: 10s logger: Downloaded: 1 files, 27M in 7.7s (3.50 MB/s) logger: --2022-03-11 10:33:38-- https://github.com/fleetdm/fleet/releases/download/fleet-v4.11.0/fleetctl_v4.11.0_linux.tar.gz logger: Resolving github.com (github.com)... 13.234.176.102 logger: Connecting to github.com (github.com)|13.234.176.102|:443... connected. logger: HTTP request sent, awaiting response... 302 Found logger: Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/309820286/c13ae8a0-73c0-4646-a45f-4cf912d5fd36?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220311%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220311T103340Z&X-Amz-Expires=300&X-Amz-Signature=ce9d3ab972cca83a2859b748e0bfe438489ab13dbcfdf8de6cd434bb5c6485ce&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=309820286&response-content-disposition=attachment%3B%20filename%3Dfleetctl_v4.11.0_linux.tar.gz&response-content-type=application%2Foctet-stream [following] logger: --2022-03-11 10:33:38-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/309820286/c13ae8a0-73c0-4646-a45f-4cf912d5fd36?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220311%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220311T103340Z&X-Amz-Expires=300&X-Amz-Signature=ce9d3ab972cca83a2859b748e0bfe438489ab13dbcfdf8de6cd434bb5c6485ce&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=309820286&response-content-disposition=attachment%3B%20filename%3Dfleetctl_v4.11.0_linux.tar.gz&response-content-type=application%2Foctet-stream logger: Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.109.133, 185.199.108.133, 185.199.110.133, ... logger: Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.109.133|:443... connected. logger: HTTP request sent, awaiting response... 200 OK logger: Length: 16350769 (16M) [application/octet-stream] logger: Saving to: ‘fleetctl_v4.11.0_linux.tar.gz’ logger: fleetctl_v4.11.0_li 100%[===================>] 15.59M 11.9MB/s in 1.3s logger: logger: 2022-03-11 10:33:40 (11.9 MB/s) - ‘fleetctl_v4.11.0_linux.tar.gz’ saved [16350769/16350769] logger: logger: FINISHED --2022-03-11 10:33:40-- logger: Total wall clock time: 2.6s logger: Downloaded: 1 files, 16M in 1.3s (11.9 MB/s) logger: fleet_v4.11.0_linux/CHANGELOG.md logger: fleet_v4.11.0_linux/LICENSE logger: fleet_v4.11.0_linux/README.md logger: fleet_v4.11.0_linux/fleet logger: fleetctl_v4.11.0_linux/CHANGELOG.md logger: fleetctl_v4.11.0_linux/LICENSE logger: fleetctl_v4.11.0_linux/README.md logger: fleetctl_v4.11.0_linux/fleetctl logger: Migrations completed. logger: cp: target '/opt/fleet/' is not a directory logger: Created symlink /etc/systemd/system/multi-user.target.wants/fleet.service → /etc/systemd/system/fleet.service. logger: [10:33:50]: Waiting for fleet service to start...
Same issue here, getting the same output of @haffa008
looks like yq
changed its syntax in v4: https://mikefarah.gitbook.io/yq/v/v4.x/upgrading-from-v3#updating-writing-documents
I'm testing the fix now
Fleet also no longer supports fleetctl get options
and is now fleetctl get config
Awesome stuff @clong I did see that the previous Fleet command was not exporting the config properly since the syntax changed with the upgrade and I am glad you figured that and the YQ as well for a permanent fix!
Attached is the confirmation that the Fleet config was updated successfully and logger deployment was successful too!
@clong I think we might need to reopen this :( The ansible task is failing now as "curl -s https://api.github.com/repos/fleetdm/fleet/releases | grep 'https://github.com' | grep "/fleet.zip"" returns an empty string, so the service never gets installed.
@clong here my pull request fixing this: https://github.com/clong/DetectionLab/pull/789
Oh, whoops, I forgot to port the fixed code over to the ESXi section. Thanks for the PR!
Please verify that you are building from an updated Master branch before filing an issue: Yes, cloned the latest Master branch via GitHub desktop
Description of the issue:
While building the logger host, I'm running into the following error message that causes the build to hang:
The actual file name in the URL can be seen below:
Is because there is no actual "fleet.zip" in the URL: https://api.github.com/repos/fleetdm/fleet/releases being referred to in the "logger_bootstrap" script as seen in the code here:
Link to Gist Containing Build Logs:
NA. No local logs found in my Vagrant folder.