search

benisai / netify-sqlite

GNU General Public License v3.0
1 stars 0 forks source link

Error on the docker last command: up #1

Open davidngrc opened 1 month ago

davidngrc commented 1 month ago

I am on Proxmox 8.2 , newly installed ubuntu server 22.04 as LXC container. I then ssh into this VM as root

apt update
apt install docker-compose

git clone https://github.com/benisai/netify-sqlite.git
cd netify-sqlite/server/files/
gzip -cd GeoLite2-City.mmdb.cpgz | cpio -idmv 
cd ../..
docker build -t netify-sqlite .

it build successfully

Step 7/7 : ENTRYPOINT ["./entrypoint.sh"]
 ---> Running in 3307dca95a2a
Removing intermediate container 3307dca95a2a
 ---> 97dc7fb64404
Successfully built 97dc7fb64404
Successfully tagged netify-sqlite:latest

change the router ip

nano docker-compose.yml 
docker-compose up -d

then on the last step, it got error.

Creating network "netify-sqlite_default" with driver "bridge"
Creating netify-sqlite ... error

ERROR: for netify-sqlite  Cannot start service netify-sqlite: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "./entrypoint.sh": permission denied: unknown

ERROR: for netify-sqlite  Cannot start service netify-sqlite: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "./entrypoint.sh": permission denied: unknown
ERROR: Encountered errors while bringing up the project.

Please help.

davidngrc commented 1 month ago

I then re-install LXC ubuntu,
and try the official docker also fail with same error. it is from https://docs.docker.com/engine/install/ubuntu/

I then re-install LXC ubuntu, and try apt install docker-compose-v2 also fail with same error.

if possible, please provide instruction installation without docker? thank you.

benisai commented 1 month ago

Let me see if I have time to try on my proxmox with LXC tonight. I did this on a full Ubuntu install.

Docker and LXC Ubuntu's don't really mix from my understanding. Try a full VM install.

davidngrc commented 1 month ago

I found the fix here https://github.com/AbdBarho/stable-diffusion-webui-docker/issues/424#issuecomment-1526025247

and I use the v1 docker from apt

apt install docker-compose

chmod +x ./entrypoint.sh
docker build -t netify-sqlite .
docker compose up -d

now it is up however, I found nothing is running, so I

docker exec -it netify-sqlite bash

root@netify-sqlite:/app# ps
    PID TTY          TIME CMD
     12 pts/0    00:00:00 bash
     18 pts/0    00:00:00 ps

in just 2 second, the docker exists. I try connect to the docker again

docker exec -it netify-sqlite bash

Error response from daemon: Container 8c297d13781f6514b22d9fe60435889b0618474d33bb419ae1341606d5679f59 is restarting, wait until the container is running

please help

davidngrc commented 1 month ago

I am on commit https://github.com/benisai/netify-sqlite/commit/1d1d397f1f455d7dd698cdac624108dddc3f737f

finally get it to display

1st change the sleep time, then no longer auto exit.

nano entrypoint.sh
# change to 30 sec
sleep 30  # Adjust the sleep duration as needed

I bash into the docker, and found the node.js was not running. and did some finding, and found it was sqlite3 was installed by default https://stackoverflow.com/questions/56933231/sqlite3-isnt-getting-installed

docker exec -it netify-sqlite bash
root@netify-sqlite:/app# cd /app/web
root@netify-sqlite:/app/web# node app.js 
node:internal/modules/cjs/loader:1222
  throw err;
  ^

Error: Cannot find module 'sqlite3'
Require stack:
- /app/web/js/netify/sqlite-netify.js
- /app/web/app.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1219:15)
    at Module._load (node:internal/modules/cjs/loader:1045:27)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:215:24)
    at Module.require (node:internal/modules/cjs/loader:1304:12)
    at require (node:internal/modules/helpers:123:16)
    at Object.<anonymous> (/app/web/js/netify/sqlite-netify.js:3:17)
    at Module._compile (node:internal/modules/cjs/loader:1504:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1588:10)
    at Module.load (node:internal/modules/cjs/loader:1282:32) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/app/web/js/netify/sqlite-netify.js', '/app/web/app.js' ]
}

Node.js v22.5.1

add the sqlite3

nano Dockerfile
RUN cd web && npm install
RUN npm install sqlite3  # add this line

and modify the ip to country error

nano ./server/main.py
# change    
# except geoip2.errors.AddressNotFoundError:
# to
except Exception as e:

after run, find out this will get error, so change to condition never meet

nano ./server/main.py
# change 
# if "cpu_cores" in line:
# to
if "cpu_cores xxxxx" in line:

after run, use "ps aux" find out two "python3 get-dhcp.py" is running, comment out one in

nano entrypoint.sh
# cd /app/server && python3 get-dhcp.py &

finally rebuild and visit

docker build -t netify-sqlite .
docker compose up -d
ufw allow 3222

http://x.x.x.x:3222

davidngrc commented 1 month ago

right now, it only capture the br-lan and wan. but I want to capture the wifi interface. there is two way to correct this on the OpenWrt end.

on the OpenWrt end, 1st method is 

vi /etc/config/netifyd
# add additional the interface like
list internal_if 'phy0-ap1 phy1-ap0 phy2-ap1'

it will add the the original br-lan interface as shown in below

/etc/init.d/netifyd restart
/etc/init.d/netifyd status
/etc/init.d/netifyd info
{
    "netifyd": {
        "instances": {
            "instance1": {
                "running": true,
                "pid": 9883,
                "command": [
                    "/usr/sbin/netifyd",
                    "-R",
                    "-I",
                    "br-lan",
                    "-E",
                    "wan",
                    "-I",
                    "phy0-ap1",
                    "phy1-ap0",
                    "phy2-ap1"
                ],
                "term_timeout": 20,
                "respawn": {
                    "threshold": 3600,
                    "timeout": 5,
                    "retry": 0
                }
            }
        }
    }
}

on the OpenWrt end, 2nd method is

vi /etc/init.d/netifyd
# new the bottom, there is this line
        if [ "$autoconfig" -gt 0 ]; then                      
                NETIFYD_AUTODETECT=yes                         
                options="$(auto_detect_options)"                              
                [ ! -z "$options" ] && procd_append_param command $options
# change the last line to
                [ ! -z "$options" ] && procd_append_param command -I phy0-ap1 phy1-ap0 phy2-ap1 -E wan

it will remove the br-lan interface, and only use the interface I wanted, shown in below

/etc/init.d/netifyd restart
/etc/init.d/netifyd status
/etc/init.d/netifyd info
{
    "netifyd": {
        "instances": {
            "instance1": {
                "running": true,
                "pid": 10749,
                "command": [
                    "/usr/sbin/netifyd",
                    "-R",
                    "-I",
                    "phy0-ap1",
                    "phy1-ap0",
                    "phy2-ap1"
                ],
                "term_timeout": 20,
                "respawn": {
                    "threshold": 3600,
                    "timeout": 5,
                    "retry": 0
                }
            }
        }
    }
}

on the LXC docker container side,

  1. no idea why, but many times the netifyDB.db file is created, but there is no table inside, I guess it was caused by add row to db script. so run the create db by commenting out some if else line, as it use IF EXISTS inside create-sqlite.py. 
    
nano server/main.py

Check if the file netifyDB.db exists

if not os.path.exists(SQL_DB_FILE):

print("netifyDB.db file missing. Running create-sqlite.py...")

subprocess.run(['python3', 'create-sqlite.py'])
time.sleep(3)

else:

print("netifyDB.db file found. No need to run create-sqlite.py.")


2. add more tools inside the docker

nano netify-sqlite/Dockerfile

Install Node.js dependencies

RUN cd web && npm install RUN apt update && apt install -y sqlite3 nano RUN npm install sqlite3


there was many unknown record in the database, so I change

in the big loop, add below code inside the if block

if "established" in line or "flow" in line and "local_ip" in line:

.....

digest = flow_data.get("digest", "Unknown")

add below code

        if local_ip == "Unknown":
            continue

        # I only want some IP subnet to be monitored.
        if "192.168.3." not in local_ip   and   "192.168.3." not in dest_ip  : 
            continue

        # and want to host name of the dest ip
        dest_type2 = dest_type
        try:
            host=dest_type
            if "192.168.3." in local_ip : 
                host, _, _ = socket.gethostbyaddr(dest_ip)
            else:
                host, _, _ = socket.gethostbyaddr(local_ip)
            dest_type = host
            #print(f'{ip_address} - {host}')
        #except socket.herror as e:
        except Exception as e:
            dest_type = dest_type2

run below command inside docker to query the data inside the database

docker build -t netify-sqlite . docker compose up -d

run command inside the docker container

docker exec -it netify-sqlite bash

remove the db if needed

rm /app/database/netifyDB.sqlite

cd server

python3 main.py

sqlite3 /app/database/netifyDB.sqlite 'select from netify_purge' sqlite3 /app/database/netifyDB.sqlite 'select from netify_data' sqlite3 /app/database/netifyDB.sqlite 'SELECT name FROM sqlite_temp_master WHERE type="table"'


finally, if you re-run

/app/server python3 main.py


please also do
get the "node app.js PID", and kill it

ps aux | grep node kill 9


and then re-run

/app/web node app.js