search

telekom-security / tpotce

🍯 T-Pot - The All In One Multi Honeypot Platform 🐝
GNU General Public License v3.0
6.75k stars 1.07k forks source link

T-Pot on Fedora, Ubuntu, Debian, Suse, macOS & Windows #1344

Closed t3chn0m4g3 closed 7 months ago

t3chn0m4g3 commented 1 year ago

T-Pot - Technical Preview

T-Pot will be turning 10 years next year and this milestone will be celebrated when the time comes, which brings us today to the best time to reflect on how technology advanced, what this means for the project and how we can ensure T-Pot will meet the current and future requirements of the community.

TL;DR

  1. Download or use a running, supported distribution
  2. Install the ISO with as minimal packages / services as possible (SSH required!)
  3. Clone T-Pot: $ git clone https://github.com/telekom-security/tpotce
  4. Locate installer for your distribution: $ cd tpotce/preview/installer/<distro>
  5. Run installer as non-root: $ ./install.sh
    • Follow instructions, read messages, check for possible port conflicts and reboot
  6. Set username and password in config .env: vi preview/.env
  7. Start T-Pot for the first time: 
    $ cd tpotce/preview/
$ docker compose up

Table of Contents



Disclaimer

Last Time Departed

Jumping back to 2014 T-Pot was born as the direct ancestor of our Raspberry Pi images we used to offer for download (which probably by now only insiders will remember 😅). Docker was just the new kid on the block with the shiny new container engine everyone desperately unknowingly waited for and thus taking the dev-world by storm. At that point we wanted to ensure that T-Pot was something tangible, tethered to a physical device (Hello NUC my old friend 👋) while using latest technologies ensuring an easy transition should we ever leave hardware based installations (or VMs for that matter). And Oh-My-Zsh as you all know that day came faster than anticipated! (Special thanks @vorband, @shaderecker and @tmariuss for all of their contributions!)


Present Time

Flash Forward to today, T-Pot offers support for Debian, both as an ISO based installation or a post installation method (install your own Debian Server), support for OTC, AWS and other clouds through Ansible and Terraform Support. All of this in many different flavors and even a distributed installation. At the same time we are still relying on the same base concept we originally started with which does not seem fit for the foreseeable future.
In the last couple of years being independent of a certain platform was the one feature that stood out by far. The reason for this, until today, is the simple fact that T-Pot, although relying heavily on Docker, still relies on a fully controlled environment. This has its advantages but can not meet a demand where cloud based installations need different settings than we can provide (we can only run limited platform tests), companies follow different guidelines for allowed distributions or hosters simply offer Debian images slightly adjusted to their environments causing issues with the setting T-Pot relies on. Roll the dice or ask the Magic-8-Ball.


Destination Time

Back to the future of T-Pot. For a brief time we had the idea of T-Pot Light which should compensate for the missing platform support. A concept was whipped up to support all of T-Pot's dockered services on minimal installations of Debian, Fedora, OpenSuse and Ubuntu Server. And it worked! It worked so good that we have almost achieved feature parity for this Technical Preview and decided that this is the best candidate for the future of the development of T-Pot
We are thrilled to share this now, so you can test, provide us with feedback, open issues and discussions and give us the chance to make the next T-Pot the best T-Pot we have ever released!

Technical Preview

For the purpose of the Technical Preview T-Pot will still use the 22.04 images and for a great part rely on the 22.04 release. This will lay the groundwork though for the next T-Pot release by just relying on the latest Docker package repositories (yes, the distros mostly do not offer Docker's bleeding edge features), some tiny modifications on the host (installer and uninstaller provided!) and move all of T-Pot's core in its own Docker image with a simple, user adjustable, configuration.


Architecture

While the basic architecture still remains, the Technical Preview of T-Pot is mostly independent of the underlying OS with only some basic requirements:

  1. Underlying OS is available as supported distribution:
    • Only the bare minimum of services and packages are installed to avoid possible port conflicts with T-Pot's services
    • Debian, Fedora, OpenSuse and Ubuntu Server are currently supported, others might follow if the requirements will be met
  2. Latest Docker Engine from Docker's repositories is supported
    • Only the latest Docker Engine packages offer all the features needed for T-Pot
    • Docker Desktop does not offer host network capabilities and thus only a limited T-Pot experience (not available for the Technical Preview, but planned to even get started faster!)
  3. Changes to the host
    • Some changes to the host are necessary but will be kept as minimalistic as possible, just enough T-Pot will be able to run
    • There are uninstallers available this time 😁

System Requirements

The known T-Pot hardware (CPU, RAM, SSD) requirements and recommendations still apply.

Installation

Download one of the supported Linux distro images, git clone the T-Pot repository and run the installer specific to your system. Running T-Pot on top of a running and supported Linux system is possible, but a clean installation is recommended to avoid port conflicts with running services.

Choose your distro

Choose a supported distro of your choice. It is recommended to use the minimum / netiso installers linked below and only install a minimalistic set of packages. SSH is mandatory or you will not be able to connect to the machine remotely.

Distribution Name x64 arm64
Debian download download
Fedora download download
OpenSuse download download
Ubuntu download download



Get and install T-Pot

  1. Clone the GitHub repository: $ git clone https://github.com/telekom-security/tpotce
  2. Change into the tpotce/preview/installer folder: $ cd tpotce/preview/installer
  3. Locate your distribution, i.e. fedora: $ cd fedora
  4. Run the installer as non-root: $ ./install.sh:
    • ⚠️ Depending on your Linux distribution of choice the installer will:
      • Change the SSH port to tcp/64295
      • Disable the DNS Stub Listener to avoid port conflicts with honeypots
      • Set SELinux to Monitor Mode
      • Set the firewall target for the public zone to ACCEPT
      • Add Docker's repository and install Docker
      • Install recommended packages
      • Remove package known to cause issues
      • Add the current user to the docker group (allow docker interaction without sudo)
      • Add dps and dpsw aliases (grc docker ps -a, watch -c "grc --colour=on docker ps -a)
      • Display open ports on the host (compare with T-Pot required ports)
  5. Follow the installer instructions, you will have to enter your password at least once
  6. Check the installer messages for errors and open ports that might cause port conflicts
  7. Reboot: $ sudo reboot

T-Pot Config File

T-Pot offers a configuration file providing environment variables not only for the docker services (i.e. honeypots and tools) but also for the docker compose environment. The configuration file is hidden in the preview folder and is called .env. There is however an example file (env.example) which holds the default configuration.
Before the first start set the WEB_USER and WEB_PW. Once T-Pot was initialized it is recommended to remove the password and set WEB_PW=<changeme>. Other settings are available also, these however should only be changed if you are comfortable with possible errors 🫠 as some of the features are not fully integrated and tested yet.

# T-Pot config file. Do not remove.

# Set Web username and password here, only required for first run
#  Removing the password after first run is recommended
#  You can always add or remove users as you see fit using htpasswd:
#  htpasswd -b -c /<data_folder>/nginx/conf/nginxpasswd <username> <password>
WEB_USER=<changeme>
WEB_PW=<changeme>

# T-Pot Blackhole
#  ENABLED: T-Pot will download a db of known mass scanners and nullroute them
#           Be aware, this will put T-Pot off the map for stealth reasons and
#           you will get less traffic. Routes will active until reboot and will
#           be re-added with every T-Pot start until disabled.
#  DISABLED: This is the default and no stealth efforts are in place.
TPOT_BLACKHOLE=DISABLED

macOS & Windows

Sometimes it is just nice if you can spin up a T-Pot instance on macOS or Windows, i.e. for development, testing or just the fun of it. While Docker Desktop is rather limited not all honeypot types or T-Pot features are supported. Also remember, by default the macOS and Windows firewall are blocking access from remote, so testing is limited to the host. For production it is recommended to run T-Pot on Linux.
To get things up and running just follow these steps:

  1. Install Docker Desktop for macOS or Windows
  2. Clone the GitHub repository: $ git clone https://github.com/telekom-security/tpotce
  3. Change into the tpotce/preview/compose folder: $ cd tpotce/preview/compose
  4. Copy mac_win.yml to the tpotce/preview folder by overwriting docker-compose.yml: $ cp mac_win.yml ../docker-compose.yml
  5. Adjust the .env file by changing TPOT_OSTYPE to either mac or win: 
    # OSType (linux, mac, win)
#  Most docker features are available on linux
TPOT_OSTYPE=mac
  6. You have to ensure on your own there are no port conflicts keeping T-Pot from starting up. You can follow the README on how to Start T-Pot, however you may skip the crontab.

Start T-Pot

  1. Change into the tpotce/preview/ folder: $ cd tpotce/preview/
  2. Run: $ docker compose up (notice the missing dash, docker-compose no longer exists with the latest Docker installation)
    • You can also run $ docker compose -f /<path_to_tpot>/tpotce/preview/docker-compose.yml up directly if you want to avoid to change into the preview folder or add an alias of your choice.
  3. docker compose will now download all the necessary images to run the T-Pot Docker containers
  4. On the first run T-Pot (tpotinit) will initialize and create the data folder in the path specified (by default it is located in tpotce/preview/data/):
    • It takes about 2-3 minutes to bring all the containers up (should port conflicts arise docker compose will simply abort)
    • Once all containers have started successfully for the first time you can access T-Pot as described here or cancel with CTRL-C ...
  5. ... and run T-Pot in the background: $ docker compose up -d
    • Unless you run docker compose down -v T-Pot's Docker service will remain persistent and restart with a reboot
    • You can however add a crontab entry with crontab -e which will also add some container and image management. 
      @reboot docker compose -f /<path_to_tpot_>/tpotce/preview/docker-compose.yml down -v; \
docker container prune -f; \
docker image prune -f; \
docker compose -f /<path_to_tpot_>/tpotce/preview/docker-compose.yml up -d
  6. By default Docker will always check if the local and remote docker images match, if not, Docker will either revert to a fitting locally cached image or download the image from remote. This ensures T-Pot images will always be up-to-date

Stop T-Pot

  1. Change into the tpotce/preview/ folder: $ cd tpotce/preview/
  2. Run: $ docker compose down -v (notice the missing dash, docker-compose no longer exists with the latest docker installation)
  3. Docker will now stop all running T-Pot containers and disable reboot persistence (unless you made a crontab entry
    • You can also run $ docker compose -f /<path_to_tpot>/tpotce/preview/docker-compose.yml down -v directly if you want to avoid to change into the preview folder or add an alias of your choice.

Uninstall T-Pot

  1. Change into the tpotce/preview/uninstaller/ folder: $ cd tpotce/preview/uninstaller/
  2. Locate your distribution, i.e. fedora: $ cd fedora
  3. Run the installer as non-root: $ ./uninstall.sh:
    • The uninstaller will reverse the installation steps
  4. Follow the uninstaller instructions, you will have to enter your password at least once
  5. Check the uninstaller messages for errors
  6. Reboot: $ sudo reboot

Feedback

To ensure the next T-Pot release will be everything we and you - The T-Pot Community - have in mind please feel free to leave comments in the Technical Preview discussion pinned on our GitHub Discussions section. Please bear in mind that this Technical Preview is made public in the earliest stage of the T-Pot development process at your convenience for your valuable input.

Thank you for testing 💖

Special thanks to all the contributors and developers making this project possible!

t3chn0m4g3 commented 1 year ago

This issue is reserved for issues / feedback for the Technical Preview, anything else will be deleted, please use the Discussions instead or take the time to create an issue.

HachimanSec commented 1 year ago

Hey there @t3chn0m4g3,

I just installed the preview on Debian 10 and it worked flawless.

However when I started it for the first time, there appears to be a bind conflict: medpot | medpot | ▄▄ ▄▄ ▄▄▄▄▄▄▄ ▄▄▄▄▄▄ ▄▄▄▄▄▄▄ ▄▄▄▄▄▄▄ ▄▄▄▄▄▄▄ medpot | █ █▄█ █ █ ██ █ █ █ medpot | █ █ ▄▄▄█ ▄ █ ▄ █ ▄ █▄ ▄█ medpot | █ █ █▄▄▄█ █ █ █ █▄█ █ █ █ █ █ █ medpot | █ █ ▄▄▄█ █▄█ █ ▄▄▄█ █▄█ █ █ █ medpot | █ ██▄██ █ █▄▄▄█ █ █ █ █ █ █ medpot | █▄█ █▄█▄▄▄▄▄▄▄█▄▄▄▄▄▄██▄▄▄█ █▄▄▄▄▄▄▄█ █▄▄▄█ medpot | medpot | medpot | [*] Inform: V.1.2 medpot | [*] Inform: Starting Medpot at 30 Aug 23 18:17 UTC medpot | [*] Inform: Written by @schmalle, forked and updated by @s9rA16Bf4 medpot | [*] Inform: If you find any bugs, report them on 'github.com/s9rA16Bf4/medpot' or 'github.com/schmalle/medpot' medpot | [*] Inform: -------------------------------------------------------- medpot | [*] Inform: Log files will be located at '/var/log/medpot/medpot.log' medpot | [*] Inform: Will utilize port 2575 medpot | medpot | [*] Inform: Listening on host 0.0.0.0 on port 2575 dionaea | dionaea | Dionaea Version 0.11.0 dionaea | Compiled on Linux/x86_64 at May 30 2023 15:07:27 with gcc 11.3.0 dionaea | Started on 99eaedb84a4a running Linux/x86_64 release 4.19.0-24-cloud-amd64 dionaea | dionaea | Error response from daemon: driver failed programming external connectivity on endpoint ddospot (6caa23ce1450e78273f6dbf35afd02a1c13294a2ef628d1ed71c09359c3ef6f2): Error starting userland proxy: listen udp4 0.0.0.0:123: bind: address already in use

Any idea what this could be? It is a fresh and empty server. I havent installed in any bind or other DNS server components.

Thx!

PS: Is the Cockpit and the Kibana dashboard gone?

Cheers Tom

t3chn0m4g3 commented 1 year ago

@HachimanSec It seems a NTP daemon is running (according to the logs you provided). You need to uninstall the corresponding package.

Cockpit and Kibana will only be shown if the services are available and have been started. Reloading the page should at least reveal Kibana.

HachimanSec commented 1 year ago

Thanks for the info. Sorry for th late reply. I will try this.

HachimanSec commented 1 year ago

One more question. When I start the docker containers I receive many errors in regards to missing log files. Is this simply because of the first start? 

ewsposter            |     -> Send   1 COWRIE alert(s) to EWS Backend.
ewsposter            |  => Starting Elasticpot Honeypot Modul.
ewsposter            |  => Starting Mailoney Honeypot Modul.
ewsposter            |     -> Mission File! logfile = /data/mailoney/log/commands.log. Skip Honeypot.
ewsposter            |  => Starting Heralding Honeypot Modul.
ewsposter            |  => Starting Ciscoasa Honeypot Modul.
ewsposter            |  => Starting Tanner Honeypot Modul.
ewsposter            |     -> Mission File! logfile = /data/tanner/log/tanner_report.json. Skip Honeypot.
ewsposter            |  => Starting Glutton Honeypot Modul.
ewsposter            |     -> Mission File! logfile = /data/glutton/log/glutton.log. Skip Honeypot.
ewsposter            |  => Starting Adbhoney Honeypot Modul.
ewsposter            |     -> Mission File! logfile = /data/adbhoney/log/adbhoney.json. Skip Honeypot.
ewsposter            |  => Starting Ipphoney Honeypot Modul.
ewsposter            |  => Starting Dicompot Honeypot Modul.
ewsposter            |     -> Mission File! logfile = /data/dicompot/log/dicompot.log. Skip Honeypot.
ewsposter            |  => Starting Medpot Honeypot Modul.
ewsposter            |  => Starting Citrix Honeypot Modul.
ewsposter            |  => Starting Redishoneypot Honeypot Modul.
ewsposter            |  => Starting Endlessh Honeypot Modul.
ewsposter            |     -> Mission File! logfile = /data/endlessh/log/endlessh.log. Skip Honeypot.
ewsposter            |  => Starting Sentrypeer Honeypot Modul.
ewsposter            |     -> Mission File! logfile = /data/sentrypeer/log/sentrypeer.json. Skip Honeypot.
ewsposter            |  => Starting Log4Pot Honeypot Modul.
ewsposter            |     -> Mission File! logfile = /data/log4pot/log/log4pot.log. Skip Honeypot.
ewsposter            |  => Sleeping for 56 seconds ...
ls -lat /data/
total 160
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:48 spiderfoot
drwxrwxrwx  7 2000 2000  4096 Sep 19 12:41 dionaea
drwxrwxrwx  5 2000 2000  4096 Sep 19 12:41 honeytrap
drwxrwxrwx  4 2000 2000  4096 Sep 19 12:41 tanner
drwxrwxrwx  4 2000 2000  4096 Sep 19 12:41 adbhoney
drwxrwxrwx  6 2000 2000  4096 Sep 19 12:41 cowrie
drwxr-xr-x 19 root root  4096 Sep 19 12:41 ..
drwxrwxrwx  4 2000 2000  4096 Sep 19 12:40 elk
drwxrwxrwx 36 2000 2000  4096 Sep 19 12:40 .
-rwxrwxrwx  1 2000 2000    37 Sep 19 12:40 uuid
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 p0f
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 suricata
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 sentrypeer
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 rdpy
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 redishoneypot
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 medpot
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 mailoney
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 log4pot
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 ipphoney
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 honeypots
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 honeysap
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 hellpot
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 heralding
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 glutton
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 fatt
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 endlessh
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 elasticpot
drwxrwxrwx  4 2000 2000  4096 Sep 19 12:40 dicompot
drwxrwxrwx  5 2000 2000  4096 Sep 19 12:40 ddospot
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 conpot
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 citrixhoneypot
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 ciscoasa
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 tpot
drwxrwxrwx  3 2000 2000  4096 Sep 19 12:40 ews
drwxrwxrwx  5 2000 2000  4096 Sep 19 12:40 nginx
drwxrwxrwx  2 2000 2000  4096 Sep 19 12:40 blackhole

Content of tanner

ls -lat /data/tanner/log/
total 8
drwxrwxrwx 4 2000 2000 4096 Sep 19 12:41 ..
drwxrwxrwx 2 2000 2000 4096 Sep 19 12:40 .
t3chn0m4g3 commented 1 year ago

1404

HachimanSec commented 1 year ago

Thx, I assumed something like this. Works all very well!

t3chn0m4g3 commented 1 year ago

Thanks, great to hear it works for you.

digidomic commented 11 months ago

Cockpit and Kibana will only be shown if the services are available and have been started. Reloading the page should at least reveal Kibana.

Hi, I also installed this package today and have the problem, that the cockpit is not available (installed on Ubuntu). Is the Cockpit integraded so that it should work? If so, what can I check to make sure Cockpit will run? Thanks.

t3chn0m4g3 commented 7 months ago

As announced in #1487, T-Pot 24.x will arrive soon. T-Pot Technical Preview is closed.