Ghost Installation Troubleshooting Help

[faurkaurage]

Greetings

I hope all is well

I have recently learned about the GHOSTS Framework and have been trying without success to install the GHOST API Server in my home lab. I tried to install the Server on a Win 10 Box and had no success and now i'm trying to install on Ubuntu 20.04 with a bit more success but not quite there yet.

I followed the install directions from the below: [https://github.com/cmu-sei/GHOSTS/wiki/Installation-from-distribution-binaries] [https://cylab.be/blog/71/simulate-user-activity-with-the-ghosts-framework-introduction] [https://theoryit.com/cmu-ghosts-and-use-in-your-home-lab/]

Some Success: .My Ubuntu machine has DotNet 6.0.5 core installed .I was able to download/install docker & docker-compose correctly . My docker version is "Docker Version 20.10.12, build 20.10.12-0ubuntu2~20.04.1" .I downloaded the Github zip folder, extracted it, and placed it in my Documents folder 'GHOSTS' .I navigated to the folder with the docker-compose.yml file and ran docker-compose up -d

Problem: .After running 'docker-compose up -d' grafana comes up at localhost:3000, but localhost:5000/api/home does not come up, neither does the Postgres DB at: http://localhost:5432/ .Also do I need to add the Ghosts data source and import the dashboard in Grafana at this point? It seems like these things are already there by default. When I select 'upload JSON file' from the import page nothing happens.

Any help or thoughts would be greatly appreciated. I'm sure i'm doing something wrong or missing something small. If there are any YouTube videos with step by step guides for doing a quick install please point them in my direction

Looking Forward

Faurkaurage

[sei-dupdyke]

In the terminal run: docker ps --all

1. do you see all three containers running or not (the failed ones will show as "exited ago")?
2. if the API and postgres are exited, what we could do is look at the docker logs for each via docker logs ghosts-api (a -f flag will follow logs) and see if those have specific issues we can track down. i would start with postgres, because if that's not running, the API will fail (obviously).

post up results here please.

[faurkaurage]

Thanks again for all your help troubleshooting. I ran the docker command and things appear to be up (see screen below)

I am installing all of this on a Ubuntu VM, not sure if that would cause an issue. I've given my VM 3 CPUs, 8 GB RAM, and 100 GB HDD

'sudo docker ps --all'

'sudo docker logs ghosts-postgres'

There is a 'FATAL' log that states 'terminating connection due to administrator command'. I ran every command using 'sudo' was I not suppose to?

'sudo docker logs ghosts-api'

thanks again

Looking Forward

[faurkaurage]

Some progress :)

Instead of trying localhost:5000/api/home I tried 172.18.0.2:5000/api/home which is the docker IP instead of localhost (127.0.0.1) which is the IP of my VM.

[https://cylab.be/blog/71/simulate-user-activity-with-the-ghosts-framework-introduction]

Going to try adding a client and see if that works... MTF (more to follow)

(EDIT) UPDATE

I tried installing a client on a Windows 10 box and it did not work :(

I configured the application.json and added my docker IP address (assuming this is what I needed to do), and started ghosts.exe. A command prompt popped up with the Ghosts logo and then it opened a bunch of windows explorer on my PC and closed but I don't see anything reporting back to Grafana

I think there are a bunch of things that I'm confused on when reading the install directions. When installing a client, we simply need to update the IP address in application.json with the IP address of the docker container for ghosts-api correct?

[sei-dupdyke]

Awesome detail, thank you.

VM install is fine, GHOSTS users do it all the time. Specs look good. It is odd that your docker IP works but localhost does not. Maybe that is an ubuntu thing that I am not familiar with?

Regardless, you have it working over IP, so looking at that, it should be good to go. The client not being able to reach it, here is how I would troubleshoot:

From the client, can you hit http://172.18.0.2:5000/api/home in the browser as just a regular old GET request? If not, is there a firewall setting on the ubuntu box that is intercepting that request? Is UFW (the local ubuntu firewall) blocking it? See here: https://phoenixnap.com/kb/how-to-enable-disable-firewall-ubuntu

[faurkaurage]

From my client (WIN10) I could not hit http://172.18.0.2:5000/api/home via the browswer

I do think I am getting close to a fix. Ran NMAP from my GHOST API server and just realized each docker has it's own IP. (i'm a noob at docker). Thinking I need to add the 'right' IP address in the application.json file.

Application.json file should have the IP address to the GHOSTS-API container correct? 172.168.0.3:5000 in my case

Also the listener port should be l 8443?

I updated my Ubuntu Firewall via UFW to add rules to allow all those ports, turned off my firewall on my WIN10 client. Will retry and let you know how things go

It' WORKS!!

Think I had the wrong IP and PORT listed in the application.json.

[sei-dupdyke]

Good deal. Any lingering issues?

[faurkaurage]

Mr. Updyke

So far so good. I let my API server and client stay up all night and I noticed that there hasn't been any 'new activity' but i'm pretty sure that's because the client (Win10) does not have office suite installed, and I have not fully configured the json files to repeat actions. Now that I got the 'main' thing up, I can slow down and study how everything works in finer detail.

Questions:

1. When I downloaded the client the application.json port was listed as -1 by default. is that suppose to be 8443 by default?
2. After I make changes to any of the config files, do I simply stop 'ghosts' via the task manager and rerun the ghosts.exe to restart it using the new configs?
3. also, what is explorer \\nec-file\base?

My goal is to master how everything works then make a step-by-step installation guide that explains in fine detail how to install and use and troubleshoot if problems come up like the ones I faced (mainly because of not understanding docker and its networking). Then I will shoot this out to my unit and I think a lot of teams will show big interest in incorporating Ghosts into there training. After mastering Ghosts...I will move on to understanding Spectre

[sei-dupdyke]

Good!

When I downloaded the client the application.json port was listed as -1 by default. is that suppose to be 8443 by default?

This is for the client listener port, which you might use to send messages to a particular client. We've only used this in very specific cases, as it creates unrealistic traffic on the network and requires client host boxes to open connections to that port in order for traffic to go through (usually ala windows firewall/defender). Setting it to -1 turns it off.

Of course, there are other ways to get messages to each client, including via the API, and that's what I'd recommend in your case.

After I make changes to any of the config files, do I simply stop 'ghosts' via the task manager and rerun the ghosts.exe to restart it using the new configs?

Changing config/timeline.json will cause ghosts to automatically restart and run the new timeline (think of it as changing what that client "does all day" and so they reset their work priorities). Changing any other config does require a restart. We typically do not run as a service, but rather just start the exe as part of the user's login script (or via controls we've built with other range management tools that we use — stackstorm as an example.

also, what is explorer \nec-file\base?

This is just a left-over training artifact, it's a server and share. You can change/ignore at your convenience.

[faurkaurage]

Made one step forward then three steps back. lol

I added Microsoft Office Suite to my Win10 box and restarted and it was no longer connected to the server API. After a few hours of troubleshooting, I reverted everything to SNAPSHOTS (both API server, and client) and started over from scratch and could not get anything to work. I will take a step back and try again later and see if things work out.

Question: When you guys run Ghosts, what OS do you install the Server API on? You mentioned in the WIKI that you may have to chown something in grafana. What exactly do you have to do? I didn't quite understand what I needed to do for that part. That may be the one thing breaking everything for me.

Just wanted to say thanks so much for your help. You guys have a really good idea in this product that could benefit the cyber community in big ways, esp with a few small tweaks.

[sei-dupdyke]

Hmm, ok, here are some thoughts/tips. Sorry that this has been so painful to set up. The intent and hope are that everything just works out of the box!

1. If something is not working, start with logs first—for the client, they are stored in logs/app.log, and failed connections to the API should certainly be there. We can also crank the logging up if the problem is still not obvious—the logging level is controlled in the file nlog.config and is set to <logger name="*" minlevel="Info" writeTo="logfile"> (line 9). By changing minlevel from Info to Trace, we'll see a lot more logging data.

2. The integration with Office can be picky. If you install a 64-bit version of Office, then you need the 64-bit version of the GHOSTS client. If all you did was install Office and GHOSTS stopped working entirely, I think that should definitely show up in the logs. But handlers should be separated to the extent that if Office fails, it does not affect anything else.

When you guys run Ghosts, what OS do you install the Server API on?

We use Centos a lot, and also Ubuntu, but honestly, the OS should not matter in any way for the API because everything is contained within docker images (this is why I am a huge docker fan as an application developer).

You mentioned in the WIKI that you may have to chown something in grafana. What exactly do you have to do? I didn't quite understand what I needed to do for that part. That may be the one thing breaking everything for me.

This problem is localized only to Grafana. You'll know it is happening when the Grafana container just keeps endlessly restarting (when you run docker ps --all) because it has no rights to the volume on the host to store its data. This problem will not impact the API, as Grafana only reads data from the API's data in the Postgres container—you just won't have reports.