search

telekom-security / tpotce

🍯 T-Pot - The All In One Multi Honeypot Platform 🐝
GNU General Public License v3.0
6.84k stars 1.08k forks source link

When deploying T-Pot 22.04 on AWS using Terraform, it appears that apt-fast is trying to use an empty $myINSTALLPACKAGES. #1280

Closed tadashi-oya closed 1 year ago

tadashi-oya commented 1 year ago

Discussed in https://github.com/telekom-security/tpotce/discussions/1279

Originally posted by **tadashi-oya** March 16, 2023 When deploying T-Pot 22.04 on AWS using Terraform, it appears that apt-fast is trying to use an empty $myINSTALLPACKAGES. As a result, the installation proceeds without docker, docker-compose, etc., and ends up incomplete. ...it seems so. https://github.com/telekom-security/tpotce/blob/668a4d91a7c2d8d5cd50d9efd0d025057419d58b/iso/installer/install.sh#L307-L309 $myINSTALLPACKAGES is created from line 25 to line 30, but if install.sh is launched from cloud-init, the file $myINSTALLPACKAGESFILE does not seem to be accessible at this point. https://github.com/telekom-security/tpotce/blob/668a4d91a7c2d8d5cd50d9efd0d025057419d58b/iso/installer/install.sh#L25-L30 It seems that /opt/tpot will become available after line 707, which is the git clone. https://github.com/telekom-security/tpotce/blob/668a4d91a7c2d8d5cd50d9efd0d025057419d58b/iso/installer/install.sh#L702-L708 ## ⚠️ Basic support information (commands are expected to run as `root`) - What version of the OS are you currently using `lsb_release -a` and `uname -a`? I am trying to deploy to Debian 11. ``` [root@smoothbrake:~]# lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 11 (bullseye) Release: 11 Codename: bullseye [root@smoothbrake:~]# uname -a Linux smoothbrake 5.10.0-21-cloud-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux ``` - What T-Pot version are you currently using? ``` [root@smoothbrake:~]# cat /opt/tpot/version 22.04.0 ``` - What edition (Standard, Nextgen, etc.) of T-Pot are you running? ``` admin@ip-172-31-28-200:~/work/terraform/aws$ grep -A 5 -e "tpot_flavor" variables.tf variable "tpot_flavor" { default = "STANDARD" description = "Specify your tpot flavor [STANDARD, HIVE, HIVE_SENSOR, INDUSTRIAL, LOG4J, MEDICAL, MINI, SENSOR]" } variable "web_user" { ``` - What architecture are you running on (i.e. hardware, cloud, VM, etc.)? ``` [root@smoothbrake:~]# curl http://169.254.169.254/latest/meta-data/ami-id ami-08dbbf1c0485a4aa8 [root@smoothbrake:~]# curl http://169.254.169.254/latest/meta-data/instance-type t3.large ``` - Did you have any problems during the install? If yes, please attach `/install.log` `/install.err`. ``` tadashi@S1056419:~$ grep -A 10 -e "### Installing T-Pot dependencies." install.log ### Installing T-Pot dependencies. [apt-fast 16:18:26] [apt-fast 16:18:26]Working... this may take a while. Reading package lists... Building dependency tree... Reading state information... 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. ### Removing and holding back problematic packages ... Reading package lists... Building dependency tree... ``` [install.log](https://github.com/telekom-security/tpotce/files/10988720/install.log) [install.err.log](https://github.com/telekom-security/tpotce/files/10988724/install.err.log) - How long has your installation been running? The installation has not been executed perfectly. - Did you install upgrades, packages or use the update script? cloud-init is executing install.sh with runcmd. ``` admin@ip-172-31-28-200:~/work/terraform$ grep -A 5 -e "runcmd" cloud-init.yaml runcmd: - curl -sS --retry 5 https://github.com - git clone https://github.com/telekom-security/tpotce /root/tpot - /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf - rm /root/tpot.conf - /sbin/shutdown -r now ``` - Did you modify any scripts or configs? If yes, please attach the changes. The security group specification has been changed to use an existing security group. ``` admin@ip-172-31-28-200:~/work/terraform-test/aws$ cat main.tf provider "aws" { access_key = "${jsondecode(file("credentials.json")).AccessKeyId}" secret_key = "${jsondecode(file("credentials.json")).SecretAccessKey}" token = "${jsondecode(file("credentials.json")).Token}" region = var.ec2_region } resource "aws_instance" "tpot" { ami = var.ec2_ami[var.ec2_region] instance_type = var.ec2_instance_type key_name = var.ec2_ssh_key_name subnet_id = var.ec2_subnet_id tags = { Name = "T-Pot Honeypot" } root_block_device { volume_type = "gp2" volume_size = 128 delete_on_termination = true } user_data = templatefile("../cloud-init.yaml", { timezone = var.timezone, password = var.linux_password, tpot_flavor = var.tpot_flavor, web_user = var.web_user, web_password = var.web_password }) vpc_security_group_ids = ["sg-ExistingSecurityGroup"] associate_public_ip_address = true } ``` - Please provide a screenshot of `glances` and `htop`. The installation has not been executed perfectly. - How much free disk space is available (`df -h`)? ``` [root@smoothbrake:~]# df -h Filesystem Size Used Avail Use% Mounted on udev 3.9G 0 3.9G 0% /dev tmpfs 786M 372K 786M 1% /run /dev/nvme0n1p1 126G 1.9G 119G 2% / tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/nvme0n1p15 124M 11M 114M 9% /boot/efi tmpfs 786M 0 786M 0% /run/user/1000 ``` - What is the current container status (`dps.sh`)? docker and docker-compose have not been installed. ``` [root@smoothbrake:~]# dps.sh grc: docker: No such file or directory [ ========| System |======== ] DATE: Thu Mar 16 18:36:18 JST 2023 UPTIME: 18:36:18 up 1:17, 1 user, load average: 0.00, 0.00, 0.00 T-POT: INACTIVE BLACKHOLE: DISABLED NAME STATUS PORTS adbhoney DOWN ciscoasa DOWN citrixhoneypot DOWN conpot_guardian_ast DOWN conpot_iec104 DOWN conpot_ipmi DOWN conpot_kamstrup_382 DOWN cowrie DOWN ddospot DOWN dicompot DOWN dionaea DOWN elasticpot DOWN elasticsearch DOWN ewsposter DOWN fatt DOWN heralding DOWN honeytrap DOWN ipphoney DOWN kibana DOWN logstash DOWN mailoney DOWN map_data DOWN map_redis DOWN map_web DOWN medpot DOWN nginx DOWN p0f DOWN redishoneypot DOWN sentrypeer DOWN snare DOWN spiderfoot DOWN suricata DOWN tanner DOWN tanner_api DOWN tanner_phpox DOWN tanner_redis DOWN ``` - What is the status of the T-Pot service (`systemctl status tpot`)? ``` [root@smoothbrake:~]# systemctl status tpot.service ● tpot.service - tpot Loaded: loaded (/etc/systemd/system/tpot.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Thu 2023-03-16 17:19:08 JST; 1h 18min ago Process: 576 ExecStartPre=/opt/tpot/bin/updateip.sh (code=exited, status=0/SUCCESS) Process: 643 ExecStartPre=/bin/bash -c /opt/tpot/bin/clean.sh on (code=exited, status=0/SUCCESS) Process: 693 ExecStartPre=/opt/tpot/bin/tpdclean.sh -y (code=exited, status=0/SUCCESS) Process: 712 ExecStartPre=/bin/bash -c /sbin/ethtool --offload $(/sbin/ip address | grep "^2: " | awk '{ print $2 }> Process: 718 ExecStartPre=/bin/bash -c /sbin/ethtool -K $(/sbin/ip address | grep "^2: " | awk '{ print $2 }' | tr > Process: 724 ExecStartPre=/bin/bash -c /sbin/ip link set $(/sbin/ip address | grep "^2: " | awk '{ print $2 }' | tr> Process: 730 ExecStartPre=/opt/tpot/bin/rules.sh /opt/tpot/etc/tpot.yml set (code=exited, status=0/SUCCESS) Process: 758 ExecStart=/usr/bin/docker-compose -f /opt/tpot/etc/tpot.yml up --no-color (code=exited, status=203/EXE> Process: 759 ExecStartPost=/bin/bash -c /usr/bin/sleep 30 && /usr/sbin/conntrack -D -p udp (code=exited, status=127) Process: 790 ExecStopPost=/opt/tpot/bin/rules.sh /opt/tpot/etc/tpot.yml unset (code=exited, status=0/SUCCESS) Main PID: 758 (code=exited, status=203/EXEC) CPU: 280ms Mar 16 17:19:03 smoothbrake systemd[1]: Failed to start tpot. Mar 16 17:19:08 smoothbrake systemd[1]: tpot.service: Failed to schedule restart job: Unit docker.service not found. Mar 16 17:19:08 smoothbrake systemd[1]: tpot.service: Failed with result 'exit-code'. ``` - What ports are being occupied? Stop T-Pot `systemctl stop tpot` and run `netstat -tulpen` ``` [root@smoothbrake:~]# netstat -tulpen Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name tcp 0 0 0.0.0.0:64295 0.0.0.0:* LISTEN 0 11562 600/sshd: /usr/sbin tcp6 0 0 :::64295 :::* LISTEN 0 11564 600/sshd: /usr/sbin udp 0 0 0.0.0.0:68 0.0.0.0:* 0 2043 412/dhclient udp 0 0 127.0.0.1:323 0.0.0.0:* 0 11533 587/chronyd udp6 0 0 ::1:323 :::* 0 11534 587/chronyd udp6 0 0 fe80::c64:abff:fe4f:546 :::* 0 11313 483/dhclient ``` - If a single container shows as `DOWN` you can run `docker logs ` for the latest log entries docker and docker-compose have not been installed.
tadashi-oya commented 1 year ago

And an adhoc workaround. >>>https://github.com/telekom-security/tpotce/discussions/1279#discussioncomment-5340753