Setup problems

[kbarlee]

Hi

What prerequisites/ further steps are required to run this software? I have tried using the apt install 20.04 package, and also manually compiling and installing. Things do not seem to work out of the box - haulage fails to start and the colte-web services are missing. My debugging workflow is shown below.

COLTE installer

echo "deb [signed-by=/usr/share/keyrings/colte-archive-keyring.gpg] http://colte.cs.washington.edu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/colte.list
sudo wget -O /usr/share/keyrings/colte-archive-keyring.gpg http://colte.cs.washington.edu/colte-archive-keyring.gpg
sudo apt install software-properties-common
sudo apt update
sudo apt install colte

Haulage

sudo haulage
INFO[0000] Starting haulage                             
INFO[0000] Parsed parameters                             Parameters="{20m0s 1m0s wlp1s0 192.168.151.0/24 [192.168.151.1] {5s haulage_db haulage_db haulage_db}}"
FATA[0000] wlp1s0: No such device exists (SIOCGIFHWADDR: No such device)

Edit the haulage config to match Open5GS out of the box setup...

sudo nano /etc/haulage/config.yml

flowLogInterval: "20m"
userLogInterval: "1m"
interface: "ogstun"
userSubnet: "10.45.0.0/16"
ignoredUserAddresses: ["10.45.0.1"]

custom:
  reenablePollInterval: "5s"
  dbLocation: "haulage_db"
  dbUser: "haulage_db"
  dbPass: "haulage_db"

sudo haulage
INFO[0000] Starting haulage                             
INFO[0000] Parsed parameters                             Parameters="{20m0s 1m0s ogstun 10.45.0.0/16 [10.45.0.1] {5s haulage_db haulage_db haulage_db}}"
INFO[0000] Initializing context                         
ERRO[0000] Unable to query initial bridged state         error="Error 1146: Table 'haulage_db.customers' doesn't exist"
panic: runtime error: invalid memory address or nil pointer dereference
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4ed9f6]

goroutine 1 [running]:
database/sql.(*Rows).close(0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/database/sql/sql.go:3155 +0x76
database/sql.(*Rows).Close(0x0, 0x0, 0x0)
    /usr/local/go/src/database/sql/sql.go:3151 +0x33
panic(0x744540, 0x9ddd30)
    /usr/local/go/src/runtime/panic.go:969 +0x1b9
database/sql.(*Rows).Next(0x0, 0x0)
    /usr/local/go/src/database/sql/sql.go:2835 +0x30
github.com/uw-ictd/haulage/internal/storage.QueryGlobalBridgedState(0xc000023380, 0x0, 0x0, 0x0)
    /haulage/internal/storage/dbface.go:190 +0x212
main.synchronizeFiltersToDb(0xc000023380)
    /haulage/custom.go:180 +0x45
main.OnStart(0xa07740, 0xc000024b30, 0xa, 0xc000024b40, 0xa, 0xc000024b50, 0xa, 0x1176592e000, 0xdf8475800, 0x12a05f200)
    /haulage/custom.go:57 +0x2b3
main.main()
    /haulage/main.go:326 +0x45a

Checking mysql...

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| haulage_db         |
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.01 sec)

mysql> USE haulage_db;
Database changed

mysql> SHOW TABLES;
Empty set (0.00 sec)

need to manually add the tables...

wget https://raw.githubusercontent.com/uw-ictd/haulage/master/haulage.sql
sudo mysql  -p haulage_db < haulage.sql

mysql> SHOW TABLES;
+----------------------+
| Tables_in_haulage_db |
+----------------------+
| answers              |
| customers            |
| dnsResponses         |
| flowlogs             |
| static_ips           |
+----------------------+
5 rows in set (0.00 sec)

and finally it runs!

INFO[0000] Starting haulage                             
INFO[0000] Parsed parameters                             Parameters="{20m0s 1m0s ogstun 10.45.0.0/16 [10.45.0.1] {5s haulage_db haulage_db haulage_db}}"
INFO[0000] Initializing context                         
INFO[0000] ----------Beginning state synchronization---------- 
INFO[0000] ----------State synchronization ended---------- 
INFO[0000] Context initialization complete              
INFO[0000] Context initialization complete              
INFO[0000] Beginning packet by packet operation

Toggle a phone off flight mode

ERRO[0005] Unable to lookup imsi's static ip address     error="sql: no rows in result set" ip=10.45.0.4
WARN[0005] Failed to init user                           user=10.45.0.4

ok - need to add the user to haulage database..

wget https://raw.githubusercontent.com/uw-ictd/haulage/master/init/haulagedb.py
sudo python3 haulagedb.py add 235047364000067 07364000067 10.45.0.4

sudo haulage
INFO[0000] Starting haulage                             
INFO[0000] Parsed parameters                             Parameters="{20m0s 1m0s ogstun 10.45.0.0/16 [10.45.0.1] {5s haulage_db haulage_db haulage_db}}"
INFO[0000] Initializing context                         
INFO[0000] ----------Beginning state synchronization---------- 
INFO[0000] Setting user bridging                         Bridged:=true User=10.45.0.4
INFO[0000] iptables generated output                     error="exit status 1"
INFO[0000] ----------State synchronization ended---------- 
INFO[0000] Context initialization complete              
INFO[0000] Context initialization complete              
INFO[0000] Beginning packet by packet operation         

I'm assuming this is doing what it is meant to now:

mysql> SELECT * FROM customers;
+-----------------+----------+----------+--------+--------------+---------+---------+---------+-------+-------------+
| imsi            | username | raw_down | raw_up | data_balance | balance | bridged | enabled | admin | msisdn      |
+-----------------+----------+----------+--------+--------------+---------+---------+---------+-------+-------------+
| 235047364000067 | NULL     |     9346 |   4855 |      9985799 |  0.0000 |       1 |       1 |     0 | 07364000067 |
+-----------------+----------+----------+--------+--------------+---------+---------+---------+-------+-------------+
1 row in set (0.00 sec)

<phone> run some speedtests....

INFO[0010] No balance remaining                          User=10.45.0.4
INFO[0010] iptables generated output                     error="exit status 1"

<phone> phone data connection not working....

sudo iptables  -L -v -n
Chain FORWARD (policy ACCEPT 27 packets, 23532 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  526 34014 REJECT     all  --  *      *       10.45.0.4            0.0.0.0/0            reject-with icmp-port-unreachable

sudo python3 haulagedb.py topup 235047364000067 1000000000
haulagedb: Haulage Database Configuration Tool
haulagedb: topup user 235047364000067 add 1000000000 bytes to current balance of 0 bytes to create new data_balance of 1000000000? [Y/n] y
haulagedb: updating user 235047364000067 setting new data_balance to 1000000000

INFO[0065] Re-enabling user traffic                      User=10.45.0.4

<phone> phone data connection working again!

mysql> SELECT * FROM customers;
+-----------------+----------+-----------+-----------+--------------+---------+---------+---------+-------+-------------+
| imsi            | username | raw_down  | raw_up    | data_balance | balance | bridged | enabled | admin | msisdn      |
+-----------------+----------+-----------+-----------+--------------+---------+---------+---------+-------+-------------+
| 235047364000067 | NULL     | 136592335 | 205255816 |    707411560 |  0.0000 |       1 |       1 |     0 | 07364000067 |
+-----------------+----------+-----------+-----------+--------------+---------+---------+---------+-------+-------------+
1 row in set (0.00 sec)

GUIs

readme line 23 says _After installation, the admin tool will be running and listening on http://localhost:7998, and the user webgui will be running and listening on http://localhost:7999. You can start or stop these services with systemctl {start | stop} {colte_webgui | colte_webadmin}, respectively. Haulage can be started with sudo haulage or sudo systemctl start haulage. To start Open5Gs, refer to the docs here._

However there are no services called colte_webgui or colte_webadmin installed // nor colte-webgui or colte-webadmin as described further down in the readme.

The config files I was expecting ( https://github.com/uw-ictd/colte/tree/main/conf ) were not there,

ls /etc/colte/
config.yml

Checking the services at https://github.com/uw-ictd/colte/tree/main/package it suggests the executable path for the GUIs is

/usr/bin/colte-webgui/bin/www
/usr/bin/colte-webadmin/bin/www

but there is nothing in these folders

ls /usr/bin/colte-webgui/bin
ls: cannot access '/usr/bin/colte-webgui/bin': No such file or directory

What do I need to do to get the GUIs running?

Thanks Kenny

[matt9j]

Hi Kenny, thanks for reaching out and for the detailed report! The install process is rough right now and is something I am hoping to improve over the summer, so I appreciate the input!

What do I need to do to get the GUIs running?

It looks like something is very broken with your install that I don't expect. Ubuntu 20.04 is not one of our main testing platforms, although we do build the packages for it, so it's definitely possible the issue is with colte. You mention above you attempted to install via both source and apt package-- do you currently have the colte apt package installed? What is the output from sudo apt policy colte? The binary and config directories being empty does not make sense if you have the apt package installed, and I am curious if there is something else going on. I can spin up an ubuntu 20.04 VM and see if there's maybe something broken in the automated package build for 20.04. I'll reply here with what I find on a fresh VM deployment.

Seeing the manual edit to the haulage config above, it looks like you need to edit the top-level colte config file and run colteconf. I apologize the readme was unclear about this (it makes is look like you don't need to configure anything before running, but as currently implemented you do). I'll edit it now!

[matt9j]

Also thanks for your patience with a slow reply-- I was out on vacation last week and am just now back on the grid!

[matt9j]

I just tested installing from apt on a clean ubuntu 20.04 vm and the webservices appear to be working fine immediately after install. Could you try reinstalling from apt and let me know if apt throws any errors?

[matt9j]

Closing this due to inactivity, and it seems like install on ubuntu 20.04 are now working smoothy (that is the target distribution for the Seattle community network).