FreeTAKTeam / FreeTAKServer-User-Docs

FreeTAKServer documentation for end users
Eclipse Public License 2.0
49 stars 43 forks source link

AWS Installation instructions #25

Closed WillRoGe closed 1 year ago

WillRoGe commented 1 year ago

Followed AWS instructions but get this error (see image) when running the .FTS

From fresh install, multiple times.

Have tried adding IPs to config files.

image

bwebb94 commented 1 year ago

Hey! - could you confirm that both config files have the IPs setup as per the documentation? I can do a walk through and make sure I'm not hitting the same errors - I wrote the doc after going through an AWS install. May also help to reboot the server from the cli and then attempt starting the fts service. note that there's the fts service and fts-ui service

WillRoGe commented 1 year ago

Thanks for getting back to me!

IPs are definitely as per the documentation.

One thing might be the the issue:

"sudo ip addr add $publicip dev ens33"

I only have either lo or eth0, so have put eth0 in the place of ens33.

Otherwise, I am not sure!

WillRoGe commented 1 year ago

This is everything I did within the console:

I only did the steps detailed on the AWS setup doc prior to this.

`# FTS Server

Set up instance

sudo apt-get update && sudo apt-get upgrade -y

#https://medium.com/codex/set-up-an-ubuntu-based-ec2-instance-with-a-virtual-python-environment-ae7c434fccc6

## install the venv and make a new venv

sudo apt install python3-venv
#python3 -m venv <your-env>
python3 -m venv FTS
#source <your-env>/bin/activate
source FTS/bin/activate

#check python version is 3.8.X
python

#Find ethernet adapter: https://aws.amazon.com/premiumsupport/knowledge-center/ec2-ubuntu-secondary-network-interface/
ip a | grep ^[[:digit:]]

sudo ip addr add <AmazonIP> dev <ethernet adapter>

#install FTS

pip install wget

wget -qO - bit.ly/ftszerotouch | sudo bash

For anyone else installing on AWS in the UI [config.py](http://config.py/) the IP needs to be the AWS Public Address and the APPIP is the AWS Private Address to get it working.

cd /usr/local/lib/python3.8/dist-packages
sudo nano FreeTAKServer/controllers/configuration/MainConfig.py
## do below
sudo nano FreeTAKServer-UI/config.py
## do below

config.py values

FTSConfig.yaml values

WillRoGe commented 1 year ago

In addition to above, checked FTSConfig.yaml

bwebb94 commented 1 year ago

Ah ok, I've never done the venv setup steps with python, I would wipe the instance or start over - run the update and upgrade, then the ethernet adapter step, then run zero touch, then update the config files

WillRoGe commented 1 year ago

I had another go - fresh install, followed as above without any venv.

A very similar error shows up.

What network security settings did you use on AWS instance setup?

image

bwebb94 commented 1 year ago

I didn't make any changes to the network within the ec2 instance - all my security settings are done within the SG - should be FTS Core Services - 8087 - raw TCP client port 8089 - TLS client port 19023 - REST API

FTU UI - 5000 - Web UI 1880 - Web Map + Others

RTSP-simple-server - 9997 - REST API 9998 - Metrics Listener

and then you can use the SG setup to determine if you want the ports open to 0.0.0.0/0 or your own /32 home IP

WillRoGe commented 1 year ago

Those are the same as mine but the error is as above still so it must not be the ports

bwebb94 commented 1 year ago

Question - do you reboot your instance internally before running the zero touch installer?

bwebb94 commented 1 year ago

I'm going to do a step by step and see if I encounter the same error you do - have you verified the UI works?

WillRoGe commented 1 year ago

I have run another instance to check that I actually did do reboots. I have carefully written everything I did. I commented out anything I removed.

The same error that I most recently sent shows up.

Thank you so much for your help - I am really grateful

`Set up instance

sudo apt-get update && sudo apt-get upgrade -y

sudo reboot

#https://medium.com/codex/set-up-an-ubuntu-based-ec2-instance-with-a-virtual-python-environment-ae7c434fccc6

## install the venv and make a new venv

#sudo apt install python3-venv
#python3 -m venv <your-env>
#python3 -m venv FTS
#source <your-env>/bin/activate
#source FTS/bin/activate

#check python version is 3.8.X
python

#Find ethernet adapter: https://aws.amazon.com/premiumsupport/knowledge-center/ec2-ubuntu-secondary-network-interface/
ip a | grep ^[[:digit:]]

sudo ip addr add <AmazonIP> dev <ethernet adapter>

##in this case <ethernet adapter> is eth0 (also tried lo)

sudo reboot

#install FTS

sudo apt install python3-pip

pip install wget

sudo reboot

wget -qO - bit.ly/ftszerotouch | sudo bash

For anyone else installing on AWS in the UI the IP needs to be the AWS Public Address and the APPIP is the AWS Private Address to get it working.

cd /usr/local/lib/python3.8/dist-packages
#sudo nano FreeTAKServer/controllers/configuration/MainConfig.py
## do below
sudo nano FreeTAKServer-UI/config.py
# - edit to be as below in config.py and FTSConfig.yaml
## do below
check FTSConfig.yaml

sudo python3 -m FreeTAKServer.controllers.services.FTS

cd /usr/local/lib/python3.8/dist-packages/FreeTAKServer-UI

sudo python3 run.py

config.py values

FTSConfig.yaml values

WillRoGe commented 1 year ago

When I run the UI, it gets stuck on the:

(1077) wsgi starting up on http://AWS_Private_IP:5000

Have left it for a while but no change

bwebb94 commented 1 year ago

Ah okay I know why your install is failing.

Find ethernet adapter: https://aws.amazon.com/premiumsupport/knowledge-center/ec2-ubuntu-secondary-network-interface/

ip a | grep ^[[:digit:]]

sudo ip addr add dev

in this case is eth0 (also tried lo)

sudo reboot

Don't reboot after adding the adapter - the OS will drop that change and then the installer will fail

bwebb94 commented 1 year ago

Add the adapter run zti then you can reboot and start services

WillRoGe commented 1 year ago

okay, thank you - let me give that a go

WillRoGe commented 1 year ago

I tried following the steps without a reboot after the add ip step. That still results in the same problem!

My steps:


sudo apt-get update && sudo apt-get upgrade -y

sudo apt install python3-pip

pip install wget

sudo reboot

sudo ip addr add <Amazon_public_IP> dev eth0

wget -qO - bit.ly/ftszerotouch | sudo bash

```bash
cd /usr/local/lib/python3.8/dist-packages

sudo nano FreeTAKServer-UI/config.py
# - edit to be as below in config.py and FTSConfig.yaml
## do below
check FTSConfig.yaml

sudo python3 -m FreeTAKServer.controllers.services.FTS

cd /usr/local/lib/python3.8/dist-packages/FreeTAKServer-UI

sudo python3 run.py
``
bwebb94 commented 1 year ago

What size instance are you running this on as well? I'll do a side by side on a t2 micro and see what results I get

WillRoGe commented 1 year ago

I tried Ubuntu 20.04 LTS (HVM) ami-08c40ec9ead489470 but it had python3.10 and I struggled with that.

Then tried Ubuntu 20.04 LTS (HVM) ami-0149b2daceec4bb0 as it has python3.8 preinstalled rather than 3.10. which worked for install.

x86 architecture.

t2.micro (1vCPU, 1 GiB)

Keypair added

Networking: Allows SSH from anywhere

Allow HTTPS and HTTP

Storage: 8 GiB gp2

WillRoGe commented 1 year ago

@bwebb94 I've not had any luck improving the situation, have you?

bwebb94 commented 1 year ago

so I redid the install this weekend and I found that there were a couple changes that needed to get made to our install doc, I've gone ahead and update those, but tl;dr for your steps

connect to instance with SG setup per doc sudo apt-get update && sudo apt-get upgrade -y reboot reconnect add the public IP to the ethernet adapter run zti edit ftsconfig.yaml and config.py per documentation sudo systemctl restart FTS sudo systemctl restart FTS-UI

Web UI on http:5000 should be running for you at that point - those were the steps I retried yesterday

WillRoGe commented 1 year ago

Hello once again!

Tried the above but there is definitely something going wrong.

When I try the "sudo systemctl restart" commands, it just says that it isn't found. It clearly isn't starting FTS when the installation finishes.

Where do you get "ens33" from? I only ever get "eth0" so I am wondering why that difference exists.

I believe the installation is failing for some reason. It is skipping 15 steps.

I have followed the guide as close as I can but there must be a step missing or I am doing something wrong on repeat. I have tried it with all the TCP ports input and then also all the ports entirely open.

brothercorvo commented 1 year ago

@WillRoGe can you post here the first error in the ZT installer?

bwebb94 commented 1 year ago

the ENS33 came from the adapter on the instance at the time I made the document. One thing different from my process I've also never run the run.py after any installation - maybe try skipping that step?

WillRoGe commented 1 year ago

@brothercorvo

There is no error message on the install. Just that 15 steps are skipped.

I have also tried this on Azure.

I would be super grateful if someone could run a full, fresh install - maybe a video of it?

As I am getting the same error every time and have no idea what I am doing wrong.

running ap 0.0.0.0 running wsgi 0.0.0.0 starting SSL now 0.0.0.0 wsgi starting

there has been an exception in the start function of TCPCoTService [Errno 98] address already in use. ssl cot service starting [errno 98] address already in use.

The zti is not running FTS on exit - restarting the services just comes up saying that no service is found.

brothercorvo commented 1 year ago

We discussed your issue in our team call yesterday. We are able to install FTS following the exact instructions provided. I will probably do another test this week and let you know.

On Mon, Oct 31, 2022 at 12:33 PM WillRoGe @.***> wrote:

@brothercorvo https://github.com/brothercorvo

There is no error message on the install. Just that 15 steps are skipped.

I have also tried this on Azure.

I would be super grateful if someone could run a full, fresh install - maybe a video of it?

As I am getting the same error every time and have no idea what I am doing wrong.

running ap 0.0.0.0 running wsgi 0.0.0.0 starting SSL now 0.0.0.0 wsgi starting

there has been an exception in the start function of TCPCoTService [Errno 98] address already in use. ssl cot service starting [errno 98] address already in use.

The zti is not running FTS on exit - restarting the services just comes up saying that no service is found.

— Reply to this email directly, view it on GitHub https://github.com/FreeTAKTeam/FreeTAKServer-User-Docs/issues/25#issuecomment-1297269706, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOPIAPPMBRTZLCPAK6KERKTWF7RDBANCNFSM6AAAAAARKHRKTU . You are receiving this because you were mentioned.Message ID: @.***>

-- Thanks Giu Platania Director Door Business Consulting Cell: +1 902 748 0992 LinkedIn https://www.linkedin.com/in/giuseppe-platania-304b8233/

WillRoGe commented 1 year ago

Thanks for getting back to me!

The instructions provided miss steps - installing pip3, installing wget etc that I've worked out myself. So I wonder if there are other steps required that I am missing?

WillRoGe commented 1 year ago

Additionally the instance that exists on Amazon now has eth0 as the adapter. It'd be good to see a setup following that instance as I can't find one that has the ens33 adapter

WillRoGe commented 1 year ago

@bwebb94 if I skip the run.py step then nothing is running because the zti doesn't start the services for me :)

bwebb94 commented 1 year ago

@WillRoGe you could also use systemctl service start to accomplish the same thing -- I never ran the run.py in any installation I've done and I've never encountered the same errors you have. The address already in use sounds like the service is started and you're invoking an additional instance of it which is why you're getting the "Address already in use" -- for the adapter it is noted in the installation document that your adapter type may change via instance, and ENS33 is not authoritative, just the example I had.

WillRoGe commented 1 year ago

I had a go at sudo systemctl start FTS and it says unit FTS.service not found

brothercorvo commented 1 year ago

Thanks for getting back to me!

The instructions provided miss steps - installing pip3, installing wget etc that I've worked out myself. So I wonder if there are other steps required that I am missing?

you must follow the instructions that @bwebb94 provided in this document https://freetakteam.github.io/FreeTAKServer-User-Docs/Installation/Cloud/AWS/

brothercorvo commented 1 year ago

we are assuming that you will:

  1. configure AWS
  2. run the ZeroTouch installer

so you do not neet to - installing pip3, installing wget and so on

WillRoGe commented 1 year ago

Okay - let me go back again and try it step by step exactly as per the instructions - maybe installing pip3 etc is the issue

WillRoGe commented 1 year ago

Progress - the webUI is running :)

However it is saying "sorry, there seems to be an issue with your connection"

I have edited config.py

But FTSConfig.yaml doesn't exist

the sudo systemctl commands all respond with "not found"

Sorry, I realise I am being a massive pain. Thank you both so much for your help

bwebb94 commented 1 year ago

no worries at all, we want to make sure everyone can get to a working state following the instructions. Was this with a new ec2 instance?

WillRoGe commented 1 year ago

This is a new ec2 instance yep! (I think I am on my 12th now!)

So I think all I need to work out now is how to restart the services. The server doesn't disappear when I do sudo reboot

WillRoGe commented 1 year ago

Okay I worked that out - it was due to the caps. It needs to be systemctl fts / fts-ui

However the problem of no connection still persists. I will try looking at the ports again

WillRoGe commented 1 year ago

Checked and double checked all the ports to be exactly as yours are. Have also double checked the settings you mention for the .yaml and config.py

WillRoGe commented 1 year ago

I have a feeling that the web-ui isn't restarting (and hence taking the update to config.py)

It stays available on ip:5000 even when I have stopped both services

bwebb94 commented 1 year ago

The server won't disappear when you do a sudo reboot, but you'll lose the public IP assignment on the ethernet adapter, so if you rebooted make sure you add it, then try restarting the services

WillRoGe commented 1 year ago

It all works - thank you SO MUCH

I really appreciate the help you have given me over the last two weeks.

Is there any way that I can give anything back?

brothercorvo commented 1 year ago

We appreciate donations that are used to pay for the public servers. Check out our docs for info. You can aldo give a star to the project in github