search

ITRS-Group / cordial

Cordial comprises a set of interrelated programs, integrations, libraries and packages for Geneos, principally written in Go.
Other
7 stars 0 forks source link

geneos ps reports incorrect port for Web Servers #65

Closed rcanillas-itrs closed 1 year ago

rcanillas-itrs commented 1 year ago 
[geneosuser@cen7gen01 ws-6.1.1-cordial]$ geneos --version
geneos version v1.4.2

I added and started a Web Server instance. 

[geneosuser@cen7gen01 ~]$ geneos add webserver ws-6.1.1-cordial
imported "config.xml.min.tmpl" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/config.xml
imported "log4j.properties" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/log4j.properties
imported "log4j2.properties" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/log4j2.properties
imported "logging.properties" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/logging.properties
imported "login.conf" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/login.conf
imported "security.properties" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/security.properties
imported "security.xml" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/security.xml
imported "sso.properties" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/sso.properties
imported "users.properties" to localhost:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/users.properties
webserver "ws-6.1.1-cordial" added, port 8080

The port that got assigned was 8080, and this is reflected in geneos show:

...
            "logfile": "webdashboard.log",
            "maxmem": "1024m",
            "name": "ws-6.1.1-cordial",
            "port": 8080,
            "program": "/home/geneosuser/geneos/packages/webserver/active_prod/JRE/bin/java",
...

But geneos ps reports a different port:

Type       Name              Host       PID   Ports   User        Group       Starttime                  Version            Home
webserver  ws-6.1.1-cordial  localhost  5442  [7042]  geneosuser  geneosuser  2023-01-17T16:10:52+08:00  active_prod:6.1.1  /home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial

Could cause confusion for when users try to access the Web Dashboard on a web browser. Unless I am misunderstanding which port exactly is being shown through geneos ps

pgalbavy-itrs commented 1 year ago

Should not happen. There may be a command line issue somewhere.

the geneos ps command reports the actual ports open for LISTEN as per the OS and not from the config.

pgalbavy-itrs commented 1 year ago

What's the full command line please?

geneos command ws-6.1.1-cordial output will do, or ps axww output

rcanillas-itrs commented 1 year ago

As requested:

[geneosuser@cen7gen01 ws-6.1.1-cordial]$ geneos command ws-6.1.1-cordial
=== webserver "ws-6.1.1-cordial" ===
command line:
         /home/geneosuser/geneos/packages/webserver/active_prod/JRE/bin/java -XX:+UseConcMarkSweepGC -Xmx1024m -server -Djava.io.tmpdir=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/webapps -Djava.awt.headless=true -DsecurityConfig=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/security.xml -Dcom.itrsgroup.configuration.file=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/config.xml -Dcom.itrsgroup.dashboard.resources.dir=/home/geneosuser/geneos/packages/webserver/active_prod/resources -Djava.library.path=/home/geneosuser/geneos/packages/webserver/active_prod/JRE/lib:/home/geneosuser/geneos/packages/webserver/active_prod/lib64 -Dlog4j2.configurationFile=file:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/log4j2.properties -Dworking.directory=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial -Dcom.itrsgroup.legacy.database.maxconnections=100 -Dcom.itrsgroup.sso.config.file=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/sso.properties -Djava.security.auth.login.config=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/login.conf -Djava.security.krb5.conf=/etc/krb5.conf -Dcom.itrsgroup.bdosync=DataView,BDOSyncType_Level,DV1_SyncLevel_RedAmberCells -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp -jar /home/geneosuser/geneos/packages/webserver/active_prod/geneos-web-server.jar -dir /home/geneosuser/geneos/packages/webserver/active_prod/webapps -port 8080 -maxThreads 254

working directory:
         /home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial

environment:
         LD_LIBRARY_PATH=/home/geneosuser/geneos/packages/webserver/active_prod/JRE/lib:/home/geneosuser/geneos/packages/webserver/active_prod/lib64
[geneosuser@cen7gen01 ws-6.1.1-cordial]$ ps axww | grep webserver
 6978 ?        Ssl    0:10 /home/geneosuser/geneos/packages/webserver/active_prod/JRE/bin/java -XX:+UseConcMarkSweepGC -Xmx1024m -server -Djava.io.tmpdir=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/webapps -Djava.awt.headless=true -DsecurityConfig=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/security.xml -Dcom.itrsgroup.configuration.file=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/config.xml -Dcom.itrsgroup.dashboard.resources.dir=/home/geneosuser/geneos/packages/webserver/active_prod/resources -Djava.library.path=/home/geneosuser/geneos/packages/webserver/active_prod/JRE/lib:/home/geneosuser/geneos/packages/webserver/active_prod/lib64 -Dlog4j2.configurationFile=file:/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/log4j2.properties -Dworking.directory=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial -Dcom.itrsgroup.legacy.database.maxconnections=100 -Dcom.itrsgroup.sso.config.file=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/sso.properties -Djava.security.auth.login.config=/home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial/config/login.conf -Djava.security.krb5.conf=/etc/krb5.conf -Dcom.itrsgroup.bdosync=DataView,BDOSyncType_Level,DV1_SyncLevel_RedAmberCells -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp -jar /home/geneosuser/geneos/packages/webserver/active_prod/geneos-web-server.jar -dir /home/geneosuser/geneos/packages/webserver/active_prod/webapps -port 8080 -maxThreads 254
pgalbavy-itrs commented 1 year ago

In local testing I see both 7042 and 8080 open:

Type       Name  Host       PID  Ports        User    Group   Starttime             Version            Home
webserver  demo  localhost  21   [7042 8080]  geneos  geneos  2023-01-17T09:28:47Z  active_prod:6.1.1  /home/geneos/webserver/webservers/demo

I suspect 7042 is the ORB port or similar. Can you check the Web Server logs in case it was unable to LISTEN on port 8080 (because of some other process having it open) and then logging and error but continuing?

rcanillas-itrs commented 1 year ago

My web server's PID is 6978 and it shows that the web server indeed has both 7042 and 8080

[geneosuser@cen7gen01 ws-6.1.1-cordial]$ netstat -panut | grep 6978
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:7042            0.0.0.0:*               LISTEN      6978/java
tcp6       0      0 :::8080                 :::*                    LISTEN      6978/java

Web server logs show 8080 is fine:

2023-01-17 17:30:13.837 WebAppServer [INFO] -------------------------------------------
2023-01-17 17:30:13.837 WebAppServer [INFO] JRE information
2023-01-17 17:30:13.837 WebAppServer [INFO] -------------------------------------------
2023-01-17 17:30:13.837 WebAppServer [INFO] Java Version:11.0.17
2023-01-17 17:30:13.837 WebAppServer [INFO] Runtime name:OpenJDK Runtime Environment
2023-01-17 17:30:13.838 WebAppServer [INFO] Vendor:Eclipse Adoptium
2023-01-17 17:30:13.838 WebAppServer [INFO] VM Name:OpenJDK 64-Bit Server VM
2023-01-17 17:30:13.838 WebAppServer [INFO] OS:Linux
2023-01-17 17:30:13.838 WebAppServer [INFO] OS Version:3.10.0-1160.81.1.el7.x86_64
2023-01-17 17:30:13.838 WebAppServer [INFO] -------------------------------------------
2023-01-17 17:30:13.838 WebAppServer [INFO] Java Version information
2023-01-17 17:30:13.838 WebAppServer [INFO] -------------------------------------------
2023-01-17 17:30:13.842 WebAppServer [INFO] GA6.1.1-221223                geneos-web-server
2023-01-17 17:30:13.843 WebAppServer [INFO] -------------------------------------------
2023-01-17 17:30:13.843 WebAppServer [INFO] Native Version information
2023-01-17 17:30:13.843 WebAppServer [INFO] -------------------------------------------
2023-01-17 17:30:13.861 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:52:28    User    EMF2 ActiveConsole
2023-01-17 17:30:13.862 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:50:59    User    EMF2 Comms Library
2023-01-17 17:30:13.862 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:51:33    User    EMF2 Dataviews Library
2023-01-17 17:30:13.862 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:51:17    User    EMF2 Directory Library
2023-01-17 17:30:13.862 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:51:12    User    EMF2 Distributed Event List Library
2023-01-17 17:30:13.862 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:51:47    User    EMF2 HTTP Library
2023-01-17 17:30:13.862 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:52:26    User    EMF2 JNO Library
2023-01-17 17:30:13.863 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:51:40    User    EMF2 Naming Library
2023-01-17 17:30:13.863 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:51:52    User    EMF2 Netobjects Library
2023-01-17 17:30:13.863 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:51:09    User    EMF2 Primitives Library
2023-01-17 17:30:13.863 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:51:51    User    EMF2 Xml Library
2023-01-17 17:30:13.863 WebAppServer [INFO] GA6.1.1-221223      Dec 27 2022 02:50:45    User    l_lib Library
2023-01-17 17:30:13.865 WebAppServer [INFO] Starting server on port 8080...
2023-01-17 17:30:13.890 WebAppServer [INFO] Starting handler...
2023-01-17 17:30:14.781 GeneosWebAppContextListener [INFO] Found parameter com.itrsgroup.resources.dir='/home/geneosuser/geneos/packages/webserver/active_prod/resources'
2023-01-17 17:30:15.649 GeneosSystem [INFO] Setting maximum database connections to 100
2023-01-17 17:30:16.350 NativeLoggerBridge [INFO] SSLContext: TLS version 1.0 and upwards accepted for secure connections.
2023-01-17 17:30:16.984 WebAppServer [INFO] Handler started!
2023-01-17 17:30:17.295 GeneosSystem [INFO] Initialised!
2023-01-17 17:30:17.297 GeneosWebAppContextListener [INFO] BDO sync = DataView,BDOSyncType_Level,DV1_SyncLevel_RedAmberCells

2023-01-17 17:30:17.297 NativeLoggerBridge [INFO] CONMANAGER: sleepInTryHeartbeat is true
2023-01-17 17:30:17.298 NativeLoggerBridge [INFO] CONMANAGER: tryHeartbeatIterations is set to 10
2023-01-17 17:30:17.298 NativeLoggerBridge [INFO] CONMANAGER: heartbeatInterval is set to 70
2023-01-17 17:30:17.298 NativeLoggerBridge [INFO] CONMANAGER: connectWait time is set to 15
2023-01-17 17:30:17.389 GeneosSystem [INFO] Started ActiveDashboard GeneosSystem
2023-01-17 17:30:17.393 GeneosSystem [INFO] GatewayMonitor created and registered!
2023-01-17 17:30:17.397 GeneosSystem [INFO] DashboardMonitor created and registered!
2023-01-17 17:30:17.489 GatewaySectionHandler [INFO] Setting diagnostics port to '7042'.
2023-01-17 17:30:18.031 WebAppServer [INFO] http://cen7gen01:8080
2023-01-17 17:30:18.032 WebAppServer [INFO] http://192.168.36.194:8080
2023-01-17 17:30:18.032 WebAppServer [INFO] Server is ready!
pgalbavy-itrs commented 1 year ago

The look-up is done here:

https://github.com/ITRS-Group/cordial/blob/main/tools/geneos/internal/instance/instance_linux.go#L124

It uses /proc/[PID]/fd/ to get the open sockets and then matches these to /proc/net/tcp - unless there are weird permission issues this should work as expected.

What do you see if you:

ls -l /proc/6978/fd | grep 'socket:'
cat /proc/net/tcp

and visually match the "socket" value to the rows in /proc/net/tcp where the st column is 0A (LISTEN) ?

Peter

rcanillas-itrs commented 1 year ago

Nothing for ls -l /proc/6978/fd | grep 'socket;':

[geneosuser@cen7gen01 ws-6.1.1-cordial]$ ls -l /proc/6978/fd | grep 'socket;'
[geneosuser@cen7gen01 ws-6.1.1-cordial]$ sudo ls -l /proc/6978/fd | grep 'socket;'
[sudo] password for geneosuser:
[geneosuser@cen7gen01 ws-6.1.1-cordial]$

Output for ls -l /proc/6978/fd | grep 'socket' (ommitted semicolon):

[geneosuser@cen7gen01 ws-6.1.1-cordial]$ ls -l /proc/6978/fd | grep 'socket'
lrwx------ 1 geneosuser geneosuser 64 Jan 17 17:32 122 -> socket:[86751]
lrwx------ 1 geneosuser geneosuser 64 Jan 17 17:32 123 -> socket:[86750]
lrwx------ 1 geneosuser geneosuser 64 Jan 17 17:32 127 -> socket:[87121]
lrwx------ 1 geneosuser geneosuser 64 Jan 17 17:32 80 -> socket:[86706]

Output for cat /proc/net/tcp (unfortunately no idea what any of these mean, sorry):

[geneosuser@cen7gen01 ws-6.1.1-cordial]$ cat /proc/net/tcp
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   0: 00000000:1B7C 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72764 1 0000000000000000 100 0 0 10 0
   1: 00000000:1B7F 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72476 1 0000000000000000 100 0 0 10 0
   2: 00000000:1B81 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72747 1 0000000000000000 100 0 0 10 0
   3: 00000000:0801 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 35155 1 0000000000000000 100 0 0 10 0
   4: 00000000:1B82 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 87121 1 0000000000000000 100 0 0 10 0
   5: 00000000:9523 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 35170 1 0000000000000000 100 0 0 10 0
   6: 00000000:0CEA 00000000:0000 0A 00000000:00000000 00:00000000 00000000   987        0 37195 1 0000000000000000 100 0 0 10 0
   7: 00000000:1EAD 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72746 1 0000000000000000 100 0 0 10 0
   8: 00000000:006F 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 23494 1 0000000000000000 100 0 0 10 0
   9: 00000000:4E50 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 34925 1 0000000000000000 100 0 0 10 0
  10: 00000000:23B1 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72486 1 0000000000000000 100 0 0 10 0
  11: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 34903 1 0000000000000000 100 0 0 10 0
  12: 00000000:C398 00000000:0000 0A 00000000:00000000 00:00000000 00000000    29        0 34938 1 0000000000000000 100 0 0 10 0
  13: 0100007F:0019 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 37058 1 0000000000000000 100 0 0 10 0
  14: C224A8C0:0016 0124A8C0:FAE6 01 00000000:00000000 02:0009609F 00000000     0        0 89039 2 0000000000000000 20 4 17 10 -1
  15: C224A8C0:0016 0124A8C0:DA2C 01 00000000:00000000 02:00094705 00000000     0        0 37915 4 0000000000000000 20 4 31 10 33
  16: C224A8C0:1B7C C224A8C0:E510 01 00000000:00000000 00:00000000 00000000  1001        0 72768 1 0000000000000000 20 4 28 10 -1
  17: 0100007F:23B1 0100007F:D2F0 01 00000000:00000000 00:00000000 00000000  1001        0 72573 1 0000000000000000 20 4 0 10 -1
  18: 0100007F:1B81 0100007F:E6F8 01 00000000:00000000 00:00000000 00000000  1001        0 72767 1 0000000000000000 20 4 30 10 -1
  19: C224A8C0:0016 0124A8C0:D9E0 01 00000000:00000000 02:0000209F 00000000     0        0 70721 2 0000000000000000 20 20 21 10 20
  20: C224A8C0:1B7F 0124A8C0:E2C1 01 00000000:00000000 00:00000000 00000000  1001        0 72483 1 0000000000000000 23 4 26 10 -1
  21: 0100007F:E6F8 0100007F:1B81 01 00000000:00000000 00:00000000 00000000  1001        0 72478 1 0000000000000000 20 4 31 10 -1
  22: C224A8C0:E510 C224A8C0:1B7C 01 00000000:00000000 00:00000000 00000000  1001        0 72479 1 0000000000000000 20 4 30 10 -1

Linux kernel and OS version, in case any of these are relevant:

[geneosuser@cen7gen01 ws-6.1.1-cordial]$ uname -r
3.10.0-1160.81.1.el7.x86_64
[geneosuser@cen7gen01 ws-6.1.1-cordial]$ cat /etc/*release*
CentOS Linux release 7.9.2009 (Core)
Derived from Red Hat Enterprise Linux 7.9 (Source)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.9.2009 (Core)
CentOS Linux release 7.9.2009 (Core)
cpe:/o:centos:centos:7
pgalbavy-itrs commented 1 year ago

Running with sudo will make permissions odd. You would have to run sudo geneos ps to make it see file descriptors for processes that are run as another user.

Note: I edited my messge to change the semi-colon typo to a colon :)

pgalbavy-itrs commented 1 year ago

There is partial support for running intelligently sudo but this needs much improvement.

pgalbavy-itrs commented 1 year ago

Do you get more line from /proc/net/tcp if you run as root or geneosuser ? The lines at the top, e.g.

 sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   0: 00000000:1B7C 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72764 1 0000000000000000 100 0 0 10 0
   1: 00000000:1B7F 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72476 1 0000000000000000 100 0 0 10 0
   2: 00000000:1B81 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72747 1 0000000000000000 100 0 0 10 0

are the ones that show LISTENing sockets - the 4th column is '0A' to show this and the 12th (?) column 'inode' should match the value inside the brackets for socket:[INODE] in the first listing.

rcanillas-itrs commented 1 year ago

As geneosuser:

[geneosuser@cen7gen01 ws-6.1.1-cordial]$ cat /proc/net/tcp
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   0: 00000000:1B7C 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72764 1 0000000000000000 100 0 0 10 0
   1: 00000000:1B7F 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72476 1 0000000000000000 100 0 0 10 0
   2: 00000000:1B81 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72747 1 0000000000000000 100 0 0 10 0
   3: 00000000:0801 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 35155 1 0000000000000000 100 0 0 10 0
   4: 00000000:1B82 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 87121 1 0000000000000000 100 0 0 10 0
   5: 00000000:9523 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 35170 1 0000000000000000 100 0 0 10 0
   6: 00000000:0CEA 00000000:0000 0A 00000000:00000000 00:00000000 00000000   987        0 37195 1 0000000000000000 100 0 0 10 0
   7: 00000000:1EAD 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72746 1 0000000000000000 100 0 0 10 0
   8: 00000000:006F 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 23494 1 0000000000000000 100 0 0 10 0
   9: 00000000:4E50 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 34925 1 0000000000000000 100 0 0 10 0
  10: 00000000:23B1 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1002        0 96534 1 0000000000000000 100 0 0 10 0
  11: 00000000:7555 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1002        0 96494 1 0000000000000000 100 0 0 10 0
  12: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 34903 1 0000000000000000 100 0 0 10 0
  13: 00000000:C398 00000000:0000 0A 00000000:00000000 00:00000000 00000000    29        0 34938 1 0000000000000000 100 0 0 10 0
  14: 0100007F:0019 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 37058 1 0000000000000000 100 0 0 10 0
  15: C224A8C0:0016 0124A8C0:FAE6 01 00000000:00000000 02:0005D02C 00000000     0        0 89039 2 0000000000000000 20 4 17 10 -1
  16: C224A8C0:0016 0124A8C0:DA2C 01 00000000:00000000 02:0005B692 00000000     0        0 37915 4 0000000000000000 20 4 31 10 33
  17: C224A8C0:1B7C C224A8C0:E510 01 00000000:00000000 00:00000000 00000000  1001        0 72768 1 0000000000000000 20 4 28 10 -1
  18: 0100007F:1B81 0100007F:E6F8 01 00000000:00000000 00:00000000 00000000  1001        0 72767 1 0000000000000000 20 4 30 10 -1
  19: C224A8C0:0016 0124A8C0:D9E0 01 00000000:00000000 02:0007102C 00000000     0        0 70721 2 0000000000000000 20 4 30 10 20
  20: C224A8C0:0016 0124A8C0:D9E8 01 00000000:00000000 02:000AF692 00000000     0        0 98288 2 0000000000000000 21 7 9 10 -1
  21: C224A8C0:1B7F 0124A8C0:E2C1 01 00000000:00000000 00:00000000 00000000  1001        0 72483 1 0000000000000000 23 4 28 10 20
  22: C224A8C0:7555 C224A8C0:9326 01 00000000:00000000 00:00000000 00000000  1002        0 96532 1 0000000000000000 20 4 30 10 -1
  23: C224A8C0:9326 C224A8C0:7555 01 00000000:00000000 00:00000000 00000000  1001        0 96530 1 0000000000000000 20 4 28 10 -1
  24: 0100007F:E6F8 0100007F:1B81 01 00000000:00000000 00:00000000 00000000  1001        0 72478 1 0000000000000000 20 4 31 10 -1
  25: C224A8C0:E510 C224A8C0:1B7C 01 00000000:00000000 00:00000000 00000000  1001        0 72479 1 0000000000000000 20 4 30 10 -1

As root:

[root@cen7gen01 ~]# cat /proc/net/tcp
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   0: 00000000:1B7C 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72764 1 ffff97a13aa13e00 100 0 0 10 0
   1: 00000000:1B7F 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72476 1 ffff97a2b567e4c0 100 0 0 10 0
   2: 00000000:1B81 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72747 1 ffff97a13aa12e80 100 0 0 10 0
   3: 00000000:0801 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 35155 1 ffff97a2b5679f00 100 0 0 10 0
   4: 00000000:1B82 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 87121 1 ffff97a13aa11f00 100 0 0 10 0
   5: 00000000:9523 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 35170 1 ffff97a2b567a6c0 100 0 0 10 0
   6: 00000000:0CEA 00000000:0000 0A 00000000:00000000 00:00000000 00000000   987        0 37195 1 ffff97a2b567b640 100 0 0 10 0
   7: 00000000:1EAD 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 72746 1 ffff97a13aa126c0 100 0 0 10 0
   8: 00000000:006F 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 23494 1 ffff97a2b5678000 100 0 0 10 0
   9: 00000000:4E50 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 34925 1 ffff97a2b5678f80 100 0 0 10 0
  10: 00000000:23B1 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1002        0 96534 1 ffff97a13aa17440 100 0 0 10 0
  11: 00000000:7555 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1002        0 96494 1 ffff97a13aa13640 100 0 0 10 0
  12: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 34903 1 ffff97a2b56787c0 100 0 0 10 0
  13: 00000000:C398 00000000:0000 0A 00000000:00000000 00:00000000 00000000    29        0 34938 1 ffff97a2b5679740 100 0 0 10 0
  14: 0100007F:0019 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 37058 1 ffff97a2b567ae80 100 0 0 10 0
  15: C224A8C0:0016 0124A8C0:FAE6 01 00000000:00000000 02:0005B75A 00000000     0        0 89039 2 ffff97a13aa14d80 20 4 17 10 -1
  16: C224A8C0:0016 0124A8C0:DA2C 01 00000000:00000000 02:00059DC1 00000000     0        0 37915 2 ffff97a13aa107c0 20 4 23 10 33
  17: C224A8C0:1B7C C224A8C0:E510 01 00000000:00000000 00:00000000 00000000  1001        0 72768 1 ffff97a2b567be00 20 4 26 10 -1
  18: 0100007F:1B81 0100007F:E6F8 01 00000000:00000000 00:00000000 00000000  1001        0 72767 1 ffff97a2b567f440 20 4 30 10 -1
  19: C224A8C0:0016 0124A8C0:D9E0 01 00000000:00000000 02:0006F75A 00000000     0        0 70721 2 ffff97a13aa164c0 20 4 30 10 20
  20: C224A8C0:0016 0124A8C0:D9E8 01 00000000:00000000 02:000ADDC1 00000000     0        0 98288 4 ffff97a13aa15d00 20 4 25 10 -1
  21: C224A8C0:1B7F 0124A8C0:E2C1 01 00000000:00000000 00:00000000 00000000  1001        0 72483 1 ffff97a13aa145c0 23 4 28 10 20
  22: C224A8C0:7555 C224A8C0:9326 01 00000000:00000000 00:00000000 00000000  1002        0 96532 1 ffff97a13aa11740 20 4 28 10 -1
  23: C224A8C0:9326 C224A8C0:7555 01 00000000:00000000 00:00000000 00000000  1001        0 96530 1 ffff97a13aa10f80 20 4 30 10 -1
  24: 0100007F:E6F8 0100007F:1B81 01 00000000:00000000 00:00000000 00000000  1001        0 72478 1 ffff97a2b567dd00 20 4 31 10 -1
  25: C224A8C0:E510 C224A8C0:1B7C 01 00000000:00000000 00:00000000 00000000  1001        0 72479 1 ffff97a2b567ec80 20 4 30 10 -1

As both users, I only see this socket item:

lrwx------ 1 geneosuser geneosuser 64 Jan 17 17:32 127 -> socket:[87121]

in /proc/net/tcp:

  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   4: 00000000:1B82 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1001        0 87121 1 0000000000000000 100 0 0 10 0

Do you think it's an OS or user issue or something? I could try on a different VM if you want me to.

pgalbavy-itrs commented 1 year ago

I think it's because the process has a IPv6 TCP socket open, but I don't know where those might be - meybe they are different, will read more docs:

tcp6       0      0 :::8080                 :::*                    LISTEN      6978/java

My testing has been on IPv4 only systems. Another things to add to the list TODO.

Please leave this with me for the moment.

pgalbavy-itrs commented 1 year ago

I have just checked another system I have and that has a /proc/net/tcp6 with identical entries but with longer addresses. I will merge the two together in the code and produce the list of ports from that. Should have something in the codebase soon.

rcanillas-itrs commented 1 year ago

Thanks Peter! Let me know if there's anything I can do to help. Cheers!

pgalbavy-itrs commented 1 year ago

Can you try this dev build as I do not have a live IPv6 system to test on?

geneos.gz

Unzip and make executable, obviously.

rcanillas-itrs commented 1 year ago 
[geneosuser@cen7gen01 ~]$ geneos -v
geneos version v1.4.3-dev

[geneosuser@cen7gen01 ~]$ geneos ps ws-6.1.1-cordial
Type       Name              Host       PID   Ports     User        Group       Starttime                  Version            Home
webserver  ws-6.1.1-cordial  localhost  2172  [7042 0]  geneosuser  geneosuser  2023-01-18T14:55:07+08:00  active_prod:6.1.1  /home/geneosuser/geneos/webserver/webservers/ws-6.1.1-cordial

I will try on a different VM.

rcanillas-itrs commented 1 year ago

Environment:

[geneos@mon8-latest ~]$ uname -r
3.10.0-1160.76.1.el7.x86_64
[geneos@mon8-latest ~]$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Started webserver. From log:

2023-01-18 15:11:06.366 WebAppServer [INFO] Handler started!
2023-01-18 15:11:07.517 NativeLoggerBridge [INFO] SSLContext: TLS version 1.0 and upwards accepted for secure connections.
2023-01-18 15:11:07.636 WebAppServer [INFO] http://mon8-latest:8080
2023-01-18 15:11:07.636 WebAppServer [INFO] http://192.168.36.128:8080
2023-01-18 15:11:07.636 WebAppServer [INFO] Server is ready!
2023-01-18 15:11:08.633 GeneosSystem [INFO] Initialised!

On production geneos:

[geneos@mon8-latest ~]$ geneos -v
geneos version v1.4.2

[geneos@mon8-latest ~]$ geneos ps
Type       Name          Host       PID    Ports  User    Group   Starttime                  Version            Home
webserver  op5-ws-6.1.1  localhost  26606  []     geneos  geneos  2023-01-18T15:10:58+08:00  active_prod:6.1.1  /home/geneos/webserver/webservers/op5-ws-6.1.1

On dev build geneos:

[geneos@mon8-latest ~]$ geneos -v
geneos version v1.4.2

[geneos@mon8-latest ~]$ geneos ps
Type       Name          Host       PID    Ports  User    Group   Starttime                  Version            Home
webserver  op5-ws-6.1.1  localhost  26606  []     geneos  geneos  2023-01-18T15:10:58+08:00  active_prod:6.1.1  /home/geneos/webserver/webservers/op5-ws-6.1.1

Checking fd:

[geneos@mon8-latest ~]$ ls -l /proc/26606/fd | grep 'socket:'
lrwx------ 1 geneos geneos 64 Jan 18 15:11 121 -> socket:[196690]
lrwx------ 1 geneos geneos 64 Jan 18 15:11 122 -> socket:[196636]
lrwx------ 1 geneos geneos 64 Jan 18 15:11 123 -> socket:[196637]
lrwx------ 1 geneos geneos 64 Jan 18 15:11 80 -> socket:[194515]

tcp:

[geneos@mon8-latest ~]$ cat /proc/net/tcp
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   0: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 20822 1 0000000000000000 100 0 0 10 0
   1: 0100007F:1538 00000000:0000 0A 00000000:00000000 00:00000000 00000000    26        0 22874 1 0000000000000000 100 0 0 10 0
   2: 0100007F:0019 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 23990 1 0000000000000000 100 0 0 10 0
   3: 0100007F:1F40 00000000:0000 0A 00000000:00000000 00:00000000 00000000   299        0 26095 1 0000000000000000 100 0 0 10 0
   4: 00000000:1B82 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 26217 1 0000000000000000 100 0 0 10 0
   5: 00000000:0202 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 21329 1 0000000000000000 100 0 0 10 0
   6: 00000000:1622 00000000:0000 0A 00000000:00000000 00:00000000 00000000   996        0 22037 1 0000000000000000 100 0 0 10 0
   7: 0100007F:00C7 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 23241 1 0000000000000000 100 0 0 10 0
   8: 0100007F:1F48 00000000:0000 0A 00000000:00000000 00:00000000 00000000   297        0 26249 1 0000000000000000 100 0 0 10 0
   9: 00000000:6448 00000000:0000 0A 00000000:00000000 00:00000000 00000000   298        0 24344 1 0000000000000000 100 0 0 10 0
  10: 0100007F:1F49 00000000:0000 0A 00000000:00000000 00:00000000 00000000   299        0 26215 1 0000000000000000 100 0 0 10 0
  11: 00000000:0CEA 00000000:0000 0A 00000000:00000000 00:00000000 00000000    27        0 24517 1 0000000000000000 100 0 0 10 0
  12: 0100007F:18EB 00000000:0000 0A 00000000:00000000 00:00000000 00000000   296        0 20829 1 0000000000000000 100 0 0 10 0
  13: 00000000:006F 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 16659 1 0000000000000000 100 0 0 10 0
  14: 00000000:1111 00000000:0000 0A 00000000:00000000 00:00000000 00000000   298        0 21096 1 0000000000000000 100 0 0 10 0
  15: 0100007F:1111 0100007F:AB2A 01 00000000:00000000 00:00000000 00000000   298        0 24346 1 0000000000000000 20 4 29 10 -1
  16: 8024A8C0:0016 0124A8C0:DB08 01 00000000:00000000 02:00035169 00000000     0        0 111732 4 0000000000000000 20 4 23 10 18
  17: 0100007F:1F48 0100007F:A22E 06 00000000:00000000 03:000016CF 00000000     0        0 0 3 0000000000000000
  18: 0100007F:9A70 0100007F:1628 01 00000000:00000000 02:00000CB6 00000000   297        0 39969 2 0000000000000000 20 4 28 10 -1
  19: 0100007F:AB2A 0100007F:1111 01 00000000:00000000 00:00000000 00000000   298        0 24930 1 0000000000000000 20 4 30 10 -1
  20: 0100007F:1538 0100007F:9AD0 01 00000000:00000000 02:00073E36 00000000    26        0 23254 2 0000000000000000 20 4 19 10 -1
  21: 0100007F:9A6E 0100007F:1628 01 00000000:00000000 02:00000C36 00000000   297        0 39937 2 0000000000000000 20 4 30 10 -1
  22: 0100007F:997E 0100007F:1628 01 00000000:00000000 02:00000669 00000000   297        0 26672 2 0000000000000000 20 4 30 10 -1
  23: 0100007F:9A72 0100007F:1628 01 00000000:00000000 02:00000D50 00000000   297        0 39981 2 0000000000000000 20 4 28 10 -1
  24: 0100007F:9A88 0100007F:1628 01 00000000:00000000 02:00000B1D 00000000   297        0 40256 2 0000000000000000 20 4 30 10 -1
  25: 0100007F:9A8A 0100007F:1628 01 00000000:00000000 02:00000B83 00000000   297        0 40294 2 0000000000000000 20 4 30 10 -1
  26: 0100007F:9AD0 0100007F:1538 01 00000000:00000000 02:00073E36 00000000     0        0 21359 2 0000000000000000 20 4 30 10 -1

tcp6:

[geneos@mon8-latest ~]$ cat /proc/net/tcp6
  sl  local_address                         remote_address                        st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   0: 00000000000000000000000000000000:0016 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 20824 1 0000000000000000 100 0 0 10 0
   1: 00000000000000000000000001000000:1538 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000    26        0 22873 1 0000000000000000 100 0 0 10 0
   2: 00000000000000000000000001000000:0019 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 23991 1 0000000000000000 100 0 0 10 0
   3: 00000000000000000000000000000000:01BB 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 23588 1 0000000000000000 100 0 0 10 0
   4: 00000000000000000000000000000000:0202 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 21331 1 0000000000000000 100 0 0 10 0
   5: 00000000000000000000000000000000:1622 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000   996        0 22124 1 0000000000000000 100 0 0 10 0
   6: 00000000000000000000000000000000:22A5 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 25557 1 0000000000000000 100 0 0 10 0
   7: 00000000000000000000000000000000:1628 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000   298        0 25219 1 0000000000000000 100 0 0 10 0
   8: 00000000000000000000000000000000:006F 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 16662 1 0000000000000000 100 0 0 10 0
   9: 00000000000000000000000000000000:1F90 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000  1002        0 196636 1 0000000000000000 100 0 0 10 0 *****
  10: 00000000000000000000000000000000:0050 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 21347 1 0000000000000000 100 0 0 10 0
  11: 00000000000000000000000000000000:1111 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000   298        0 21097 1 0000000000000000 100 0 0 10 0
  12: 0000000000000000FFFF00000100007F:1628 0000000000000000FFFF00000100007F:9A70 01 00000000:00000000 00:00000000 00000000   298        0 39456 1 0000000000000000 20 4 30 10 -1
  13: 0000000000000000FFFF00000100007F:1628 0000000000000000FFFF00000100007F:9A8A 01 00000000:00000000 00:00000000 00000000   298        0 39852 1 0000000000000000 20 4 30 10 -1
  14: 0000000000000000FFFF00000100007F:1628 0000000000000000FFFF00000100007F:9A6E 01 00000000:00000000 00:00000000 00000000   298        0 39938 1 0000000000000000 20 4 28 10 -1
  15: 0000000000000000FFFF00000100007F:1628 0000000000000000FFFF00000100007F:997E 01 00000000:00000000 00:00000000 00000000   298        0 26673 1 0000000000000000 20 4 30 10 -1
  16: 0000000000000000FFFF00000100007F:1628 0000000000000000FFFF00000100007F:9A88 01 00000000:00000000 00:00000000 00000000   298        0 40257 1 0000000000000000 20 4 28 10 -1
  17: 0000000000000000FFFF00000100007F:1628 0000000000000000FFFF00000100007F:9A72 01 00000000:00000000 00:00000000 00000000   298        0 39480 1 0000000000000000 20 4 30 10 -1
  18: 0000000000000000FFFF00008024A8C0:01BB 0000000000000000FFFF00000124A8C0:CBB1 01 00000000:00000000 02:000B00EF 00000000    48        0 197854 2 0000000000000000 21 4 26 10 52

Port 8080 looked like it appeared at /proc/net/tcp6 (line 9; marked with *****).

pgalbavy-itrs commented 1 year ago

I did not change the scanner to take account of the longer IPv6 addresses. I was rushing. I will fix and prepare a new test binary. I will also be trying to ensure I test on an IPv6 enabled container/VM in the future.

pgalbavy-itrs commented 1 year ago

I am somewhat jetlagged but looking to stay awake, so please try the attached binary - it should now just grab to port after the ';' regardless of the length of the hex IP address.

geneos.gz

rcanillas-itrs commented 1 year ago

Config of instances on my VM has changed, but can confirm that port is properly listed:

[geneosuser@cen7gen01 ~]$ grep port /home/geneosuser/geneos/webserver/webservers/cordial-ws/webserver.json
  "port": 8100,
  "statsd_port": "30095",
[geneosuser@cen7gen01 ~]$ geneos -v
geneos version v1.4.3-dev

[geneosuser@cen7gen01 ~]$ geneos ps cordial-ws
Type       Name        Host       PID   Ports        User        Group       Starttime                  Version            Home
webserver  cordial-ws  localhost  1602  [7042 8100]  geneosuser  geneosuser  2023-01-20T15:17:41+08:00  active_prod:6.1.1  /home/geneosuser/geneos/webserver/webservers/cordial-ws
pgalbavy-itrs commented 1 year ago

Thanks, I will clean-up and commit the fix shortly.

rcanillas-itrs commented 1 year ago

Good stuff. Thanks Peter!