eu-nebulous / resource-manager

Mozilla Public License 2.0
0 stars 0 forks source link

Failed to register edge #28

Open robert-sanfeliu opened 4 days ago

robert-sanfeliu commented 4 days ago

On PROD environment, registering a edge device failed:

  ██████╗ ███████╗███████╗ ██████╗ ██╗   ██╗██████╗  ██████╗███████╗    ██████╗ ██╗███████╗ ██████╗ ██████╗ ██╗   ██╗███████╗██████╗ ██╗   ██╗
  ██╔══██╗██╔════╝██╔════╝██╔═══██╗██║   ██║██╔══██╗██╔════╝██╔════╝    ██╔══██╗██║██╔════╝██╔════╝██╔═══██╗██║   ██║██╔════╝██╔══██╗╚██╗ ██╔╝
  ██████╔╝█████╗  ███████╗██║   ██║██║   ██║██████╔╝██║     █████╗      ██║  ██║██║███████╗██║     ██║   ██║██║   ██║█████╗  ██████╔╝ ╚████╔╝
  ██╔══██╗██╔══╝  ╚════██║██║   ██║██║   ██║██╔══██╗██║     ██╔══╝      ██║  ██║██║╚════██║██║     ██║   ██║╚██╗ ██╔╝██╔══╝  ██╔══██╗  ╚██╔╝
  ██║  ██║███████╗███████║╚██████╔╝╚██████╔╝██║  ██║╚██████╗███████╗    ██████╔╝██║███████║╚██████╗╚██████╔╝ ╚████╔╝ ███████╗██║  ██║   ██║
  ╚═╝  ╚═╝╚══════╝╚══════╝ ╚═════╝  ╚═════╝ ╚═╝  ╚═╝ ╚═════╝╚══════╝    ╚═════╝ ╚═╝╚══════╝ ╚═════╝ ╚═════╝   ╚═══╝  ╚══════╝╚═╝  ╚═╝   ╚═╝

 :: App version ::                        (1.0.2-SNAPSHOT)
 :: Spring Boot ::                        (v3.2.10)
 :: Java (TM)   ::                        (21.0.1)

Nov 18, 2024 3:43:51 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service [Tomcat]
Nov 18, 2024 3:43:52 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet engine: [Apache Tomcat/10.1.30]
Nov 18, 2024 3:43:52 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring embedded WebApplicationContext
2024-11-18 15:44:01:125 +0000 [main] WARN GroovyTemplateAutoConfiguration - Cannot find template location: classpath:/templates/ (please add some templates, check your Groovy configuration, or set spring.groovy.template.check-template-location=false)
Nov 18, 2024 3:44:27 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring DispatcherServlet 'dispatcherServlet'
2024-11-18 15:47:12:612 +0000 [task-1] WARN SALRegistrationService - SALRegistrationService: processQueue(): Will register device: Device(id=d8764b84-77b7-47df-83e1-097f7457baba, ref=application_id|all-applications|92850c0a-a8f0-4f6f-8574-f52ef3214220, os=LINUX, name=eut_rob2, owner=admin, ipAddress=54.83.127.200, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}, requestId=35a189d3-65b4-49e1-a612-f3e0a80f5cbb, creationDate=2024-11-18T15:47:12.390000496Z, lastUpdateDate=null, archiveDate=null, status=NEW_DEVICE, nodeReference=9884710e-1951-4ce9-9613-e4743330893c, messages=[], statusUpdate=null, metrics=null, suspectTimestamp=null, retries=0, registeredToSAL=false)
2024-11-18 15:47:12:622 +0000 [task-1] WARN SALRegistrationService - Using default IP address as the environmental variable was not set or found
2024-11-18 15:47:12:622 +0000 [task-1] WARN SALRegistrationService - SALRegistrationService: processQueue(): EXCEPTION caught. Will not register device to SAL: Device(id=d8764b84-77b7-47df-83e1-097f7457baba, ref=application_id|all-applications|92850c0a-a8f0-4f6f-8574-f52ef3214220, os=LINUX, name=eut_rob2, owner=admin, ipAddress=54.83.127.200, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}, requestId=35a189d3-65b4-49e1-a612-f3e0a80f5cbb, creationDate=2024-11-18T15:47:12.390000496Z, lastUpdateDate=null, archiveDate=null, status=NEW_DEVICE, nodeReference=9884710e-1951-4ce9-9613-e4743330893c, messages=[], statusUpdate=null, metrics=null, suspectTimestamp=null, retries=0, registeredToSAL=false)
java.lang.NumberFormatException: Cannot parse null string
        at java.base/java.lang.Integer.parseInt(Unknown Source)
        at java.base/java.lang.Integer.parseInt(Unknown Source)
        at eu.nebulous.resource.discovery.registration.service.SALRegistrationService.register(SALRegistrationService.java:73)
        at eu.nebulous.resource.discovery.registration.service.SALRegistrationService.processQueue(SALRegistrationService.java:198)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

The error seems to appear when parsing the CPU. Line 73 of:

https://github.com/eu-nebulous/resource-manager/blob/r1/resource-discovery/src/main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java

EMS shows:

2024-11-15T13:32:56.265Z  INFO 30 --- [   scheduling-4] e.n.e.s.ExternalBrokerListenerService    : ExternalBrokerListenerService: New Variable Values: {spec_components_1_traits_0_properties_replicas=5.0}
2024-11-15T13:32:56.265Z  INFO 30 --- [   scheduling-4] eu.nebulous.ems.service.MvvService       : MvvService.translateAndSetValues: New Variable Values: {spec_components_1_traits_0_properties_replicas=5.0}
2024-11-15T13:32:56.265Z  INFO 30 --- [   scheduling-4] eu.nebulous.ems.service.MvvService       : MvvService.translateAndSetValues: New Constant values: {}
2024-11-18T15:35:48.705Z  INFO 30 --- [ Session Task-1] .b.c.i.ClientInstallationRequestListener : InstallationEventListener: New node DIAGNOSTICS request with Id: 6eeff1bc-2427-4bf1-b5a1-0483d81fd612
2024-11-18T15:35:48.722Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Executing Client installation Task #0: task-id=6eeff1bc-2427-4bf1-b5a1-0483d81fd612, node-id=2220ba26-aa5f-42d3-8d91-c5fd76243806, name=eut_rob, type=DIAGNOSTICS, address=54.83.127.200
2024-11-18T15:35:48.723Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Using SshClientInstaller (default) for task #0
2024-11-18T15:35:49.263Z  WARN 30 --- [inaProcessor-20] o.a.s.c.k.AcceptAllServerKeyVerifier     : Server at /54.83.127.200:22 presented unverified EC key: SHA256:Undx5zjhav07UIVtT9oF7nK4H5OJhVnd33YsIDSRy04
2024-11-18T15:35:49.270Z  INFO 30 --- [inaProcessor-20] .s.c.k.e.p.HostBoundPubkeyAuthentication : Server announced support for publickey-hostbound@openssh.com version 0
2024-11-18T15:35:49.854Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Session will be recorded in file: /opt/ems-server/logs/54.83.127.200-22-2024.11.18.15.35.49.851-0.txt
2024-11-18T15:35:49.858Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Connected to remote host: task #0: host: 54.83.127.200:22
2024-11-18T15:35:49.858Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       :
  ----------------------------------------------------------------------
  Task #0 :  Instruction Set: Detect node features (OS, architecture, cores, RAM, disk etc)
2024-11-18T15:35:49.865Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Executing installation instructions set: Detect node features (OS, architecture, cores, RAM, disk etc)
2024-11-18T15:35:49.866Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Executing instruction 1/6: Detecting target node type...
2024-11-18T15:35:49.866Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: LOG: Detecting target node type...
2024-11-18T15:35:49.867Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Executing instruction 2/6: Copying detection script to node...
2024-11-18T15:35:49.867Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: UPLOAD: bin/detect.sh -> /tmp/detect.sh
2024-11-18T15:35:49.867Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Uploading file: task #0: local: bin/detect.sh -> remote: /tmp/detect.sh
2024-11-18T15:35:51.347Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: File upload completed in 1480ms: task #0: local: bin/detect.sh -> remote: /tmp/detect.sh
2024-11-18T15:35:51.347Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Executing instruction 3/6: Make detection script executable
2024-11-18T15:35:51.347Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: EXEC: chmod +x /tmp/detect.sh
2024-11-18T15:35:51.709Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: EXEC: exit-status=0
2024-11-18T15:35:51.710Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Executing instruction 4/6: Run detection script
2024-11-18T15:35:51.710Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: EXEC: if [ ! -e /tmp/detect.txt ]; then /tmp/detect.sh &> /tmp/detect.txt; fi
2024-11-18T15:35:52.125Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: EXEC: exit-status=0
2024-11-18T15:35:52.126Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Executing instruction 5/6: Copying detection results back to EMS server...
2024-11-18T15:35:52.126Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: DOWNLOAD: /tmp/detect.txt -> logs/detect.${NODE_ADDRESS}--${TIMESTAMP-FILE}.txt
2024-11-18T15:35:52.126Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Downloading file: task #0: remote: /tmp/detect.txt -> local: logs/detect.${NODE_ADDRESS}--${TIMESTAMP-FILE}.txt
2024-11-18T15:35:52.919Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: File download completed: task #0: remote: /tmp/detect.txt -> local: logs/detect.${NODE_ADDRESS}--${TIMESTAMP-FILE}.txt
2024-11-18T15:35:52.921Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'CPU_SOCKETS' to: 1
2024-11-18T15:35:52.921Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'CPU_CORES' to: 2
2024-11-18T15:35:52.921Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'CPU_PROCESSORS' to: 2
2024-11-18T15:35:52.921Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_TOTAL_KB' to: 3924288
2024-11-18T15:35:52.921Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_AVAILABLE_KB' to: 2600188
2024-11-18T15:35:52.921Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_FREE_KB' to: 139428
2024-11-18T15:35:52.921Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_USED_KB' to: 3784860
2024-11-18T15:35:52.922Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_UTILIZATION' to: 96.447
2024-11-18T15:35:52.922Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'DISK_TOTAL_KB' to: 20141632
2024-11-18T15:35:52.922Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'DISK_FREE_KB' to: 13523304
2024-11-18T15:35:52.922Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'DISK_USED_KB' to: 6618328
2024-11-18T15:35:52.922Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'DISK_UTILIZATION' to: 32.8589
2024-11-18T15:35:52.922Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'OS_ARCHITECTURE' to: aarch64
2024-11-18T15:35:52.922Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'OS_KERNEL' to: Linux
2024-11-18T15:35:52.922Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'OS_KERNEL_RELEASE' to: 6.8.0-1015-aws
2024-11-18T15:35:52.923Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Executing instruction 6/6: Detection results...
2024-11-18T15:35:52.923Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: LOG: Detection results:
    CPU_SOCKETS=1
    CPU_CORES=2
    CPU_PROCESSORS=2
    RAM_TOTAL_KB=3924288
    RAM_AVAILABLE_KB=2600188
    RAM_FREE_KB=139428
    RAM_USED_KB=3784860
    RAM_UTILIZATION=96.447
    DISK_TOTAL_KB=20141632
    DISK_FREE_KB=13523304
    DISK_USED_KB=6618328
    DISK_UTILIZATION=32.8589
    OS_ARCHITECTURE=aarch64
    OS_KERNEL=Linux
    OS_KERNEL_RELEASE=6.8.0-1015-aws
2024-11-18T15:35:52.923Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #0: Installation Instructions set succeeded: Detect node features (OS, architecture, cores, RAM, disk etc)
2024-11-18T15:35:52.923Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       :
  -------------------------------------------------------------------------
  Task #0 :  Instruction sets processed: successful=1, failed=0, exit-result=SUCCESS
2024-11-18T15:35:52.923Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task completed successfully #0
2024-11-18T15:35:52.926Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Disconnected from remote host: task #0: host: 54.83.127.200:22
2024-11-18T15:35:52.927Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Task execution result #0: success=true
2024-11-18T15:35:52.927Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Client installation Task #0: result=SUCCESS, duration=4204ms
2024-11-18T15:35:52.929Z  INFO 30 --- [pool-4-thread-1] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Sending SUCCESS execution report for Task #0: destination=ems.client.installation.reports, report={reference=15bcda77-040a-402e-927b-088522fd233a, requestId=6eeff1bc-2427-4bf1-b5a1-0483d81fd612, requestType=DIAGNOSTICS, deviceIpAddress=54.83.127.200, timestamp=1731944152929, status=SUCCESS, deviceId=2220ba26-aa5f-42d3-8d91-c5fd76243806, nodeInfo={CPU_SOCKETS=1, CPU_CORES=2, CPU_PROCESSORS=2, RAM_TOTAL_KB=3924288, RAM_AVAILABLE_KB=2600188, RAM_FREE_KB=139428, RAM_USED_KB=3784860, RAM_UTILIZATION=96.447, DISK_TOTAL_KB=20141632, DISK_FREE_KB=13523304, DISK_USED_KB=6618328, DISK_UTILIZATION=32.8589, OS_ARCHITECTURE=aarch64, OS_KERNEL=Linux, OS_KERNEL_RELEASE=6.8.0-1015-aws}}
2024-11-18T15:46:09.616Z  INFO 30 --- [ Session Task-2] .b.c.i.ClientInstallationRequestListener : InstallationEventListener: New node DIAGNOSTICS request with Id: 35a189d3-65b4-49e1-a612-f3e0a80f5cbb
2024-11-18T15:46:09.617Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Executing Client installation Task #1: task-id=35a189d3-65b4-49e1-a612-f3e0a80f5cbb, node-id=92850c0a-a8f0-4f6f-8574-f52ef3214220, name=eut_rob2, type=DIAGNOSTICS, address=54.83.127.200
2024-11-18T15:46:09.617Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Using SshClientInstaller (default) for task #1
2024-11-18T15:46:10.074Z  WARN 30 --- [inaProcessor-30] o.a.s.c.k.AcceptAllServerKeyVerifier     : Server at /54.83.127.200:22 presented unverified EC key: SHA256:Undx5zjhav07UIVtT9oF7nK4H5OJhVnd33YsIDSRy04
2024-11-18T15:46:10.075Z  INFO 30 --- [inaProcessor-30] .s.c.k.e.p.HostBoundPubkeyAuthentication : Server announced support for publickey-hostbound@openssh.com version 0
2024-11-18T15:46:10.655Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Session will be recorded in file: /opt/ems-server/logs/54.83.127.200-22-2024.11.18.15.46.10.655-1.txt
2024-11-18T15:46:10.655Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Connected to remote host: task #1: host: 54.83.127.200:22
2024-11-18T15:46:10.655Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       :
  ----------------------------------------------------------------------
  Task #1 :  Instruction Set: Detect node features (OS, architecture, cores, RAM, disk etc)
2024-11-18T15:46:10.656Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Executing installation instructions set: Detect node features (OS, architecture, cores, RAM, disk etc)
2024-11-18T15:46:10.656Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Executing instruction 1/6: Detecting target node type...
2024-11-18T15:46:10.656Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: LOG: Detecting target node type...
2024-11-18T15:46:10.656Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Executing instruction 2/6: Copying detection script to node...
2024-11-18T15:46:10.656Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: UPLOAD: bin/detect.sh -> /tmp/detect.sh
2024-11-18T15:46:10.656Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Uploading file: task #1: local: bin/detect.sh -> remote: /tmp/detect.sh
2024-11-18T15:46:11.767Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: File upload completed in 1111ms: task #1: local: bin/detect.sh -> remote: /tmp/detect.sh
2024-11-18T15:46:11.767Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Executing instruction 3/6: Make detection script executable
2024-11-18T15:46:11.767Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: EXEC: chmod +x /tmp/detect.sh
2024-11-18T15:46:12.133Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: EXEC: exit-status=0
2024-11-18T15:46:12.133Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Executing instruction 4/6: Run detection script
2024-11-18T15:46:12.133Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: EXEC: if [ ! -e /tmp/detect.txt ]; then /tmp/detect.sh &> /tmp/detect.txt; fi
2024-11-18T15:46:12.498Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: EXEC: exit-status=0
2024-11-18T15:46:12.499Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Executing instruction 5/6: Copying detection results back to EMS server...
2024-11-18T15:46:12.499Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: DOWNLOAD: /tmp/detect.txt -> logs/detect.${NODE_ADDRESS}--${TIMESTAMP-FILE}.txt
2024-11-18T15:46:12.499Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Downloading file: task #1: remote: /tmp/detect.txt -> local: logs/detect.${NODE_ADDRESS}--${TIMESTAMP-FILE}.txt
2024-11-18T15:46:13.313Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: File download completed: task #1: remote: /tmp/detect.txt -> local: logs/detect.${NODE_ADDRESS}--${TIMESTAMP-FILE}.txt
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'CPU_SOCKETS' to: 1
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'CPU_CORES' to: 2
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'CPU_PROCESSORS' to: 2
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_TOTAL_KB' to: 3924288
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_AVAILABLE_KB' to: 2600188
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_FREE_KB' to: 139428
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_USED_KB' to: 3784860
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'RAM_UTILIZATION' to: 96.447
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'DISK_TOTAL_KB' to: 20141632
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'DISK_FREE_KB' to: 13523304
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'DISK_USED_KB' to: 6618328
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'DISK_UTILIZATION' to: 32.8589
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'OS_ARCHITECTURE' to: aarch64
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'OS_KERNEL' to: Linux
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'OS_KERNEL_RELEASE' to: 6.8.0-1015-aws
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Executing instruction 6/6: Detection results...
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: LOG: Detection results:
    CPU_SOCKETS=1
    CPU_CORES=2
    CPU_PROCESSORS=2
    RAM_TOTAL_KB=3924288
    RAM_AVAILABLE_KB=2600188
    RAM_FREE_KB=139428
    RAM_USED_KB=3784860
    RAM_UTILIZATION=96.447
    DISK_TOTAL_KB=20141632
    DISK_FREE_KB=13523304
    DISK_USED_KB=6618328
    DISK_UTILIZATION=32.8589
    OS_ARCHITECTURE=aarch64
    OS_KERNEL=Linux
    OS_KERNEL_RELEASE=6.8.0-1015-aws
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #1: Installation Instructions set succeeded: Detect node features (OS, architecture, cores, RAM, disk etc)
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       :
  -------------------------------------------------------------------------
  Task #1 :  Instruction sets processed: successful=1, failed=0, exit-result=SUCCESS
2024-11-18T15:46:13.314Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task completed successfully #1
2024-11-18T15:46:13.316Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Disconnected from remote host: task #1: host: 54.83.127.200:22
2024-11-18T15:46:13.316Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Task execution result #1: success=true
2024-11-18T15:46:13.316Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Client installation Task #1: result=SUCCESS, duration=3699ms
2024-11-18T15:46:13.316Z  INFO 30 --- [pool-4-thread-2] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Sending SUCCESS execution report for Task #1: destination=ems.client.installation.reports, report={reference=1a3e469f-7bd4-4de4-a4b4-bde6f6a94654, requestId=35a189d3-65b4-49e1-a612-f3e0a80f5cbb, requestType=DIAGNOSTICS, deviceIpAddress=54.83.127.200, timestamp=1731944773316, status=SUCCESS, deviceId=92850c0a-a8f0-4f6f-8574-f52ef3214220, nodeInfo={CPU_SOCKETS=1, CPU_CORES=2, CPU_PROCESSORS=2, RAM_TOTAL_KB=3924288, RAM_AVAILABLE_KB=2600188, RAM_FREE_KB=139428, RAM_USED_KB=3784860, RAM_UTILIZATION=96.447, DISK_TOTAL_KB=20141632, DISK_FREE_KB=13523304, DISK_USED_KB=6618328, DISK_UTILIZATION=32.8589, OS_ARCHITECTURE=aarch64, OS_KERNEL=Linux, OS_KERNEL_RELEASE=6.8.0-1015-aws}}
2024-11-18T15:47:09.626Z  INFO 30 --- [ Session Task-3] .b.c.i.ClientInstallationRequestListener : InstallationEventListener: New node ONBOARDING request with Id: 35a189d3-65b4-49e1-a612-f3e0a80f5cbb
2024-11-18T15:47:09.658Z  INFO 30 --- [ Session Task-3] g.i.i.ems.baguette.server.NodeRegistry   : NodeRegistry.addNode(): Provided-Address=54.83.127.200,  Resolved-IP-Address=54.83.127.200
2024-11-18T15:47:09.659Z  INFO 30 --- [ Session Task-3] g.i.i.e.c.c.NodeRegistrationCoordinator  : NodeRegistrationCoordinator.registerNode(): ExecutionWare: PROACTIVE
2024-11-18T15:47:09.682Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Executing Client installation Task #2: task-id=VM-LINUX-92850c0a-a8f0-4f6f-8574-f52ef3214220-eut_rob2-DEFAULT-54.83.127.200-_, node-id=92850c0a-a8f0-4f6f-8574-f52ef3214220, name=eut_rob2, type=VM, address=54.83.127.200
2024-11-18T15:47:09.710Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Using SshClientInstaller (default) for task #2
2024-11-18T15:47:10.158Z  WARN 30 --- [inaProcessor-40] o.a.s.c.k.AcceptAllServerKeyVerifier     : Server at /54.83.127.200:22 presented unverified EC key: SHA256:Undx5zjhav07UIVtT9oF7nK4H5OJhVnd33YsIDSRy04
2024-11-18T15:47:10.159Z  INFO 30 --- [inaProcessor-40] .s.c.k.e.p.HostBoundPubkeyAuthentication : Server announced support for publickey-hostbound@openssh.com version 0
2024-11-18T15:47:10.739Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Session will be recorded in file: /opt/ems-server/logs/54.83.127.200-22-2024.11.18.15.47.10.738-2.txt
2024-11-18T15:47:10.739Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Connected to remote host: task #2: host: 54.83.127.200:22
2024-11-18T15:47:10.739Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       :
  ----------------------------------------------------------------------
  Task #2 :  Instruction Set: Application identifications from cluster node labels
2024-11-18T15:47:10.739Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Executing installation instructions set: Application identifications from cluster node labels
2024-11-18T15:47:10.739Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Executing instruction 1/4: Identifying application from cluster nodes labels...
2024-11-18T15:47:10.739Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: LOG: Identifying application from cluster nodes labels...
2024-11-18T15:47:10.739Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Executing instruction 2/4: Run 'kubectl' command to get 'app' label
2024-11-18T15:47:10.740Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: EXEC: echo $( KUBECONFIG=~/.kube/config  kubectl get nodes -o=jsonpath='{range .items[*]}{.metadata.labels.app}{"\n"}{end}' | sort -u -i |awk NF ) | grep -v ' ' | tee /tmp/.APP_ID
  Task #2 ERR> W1118 15:47:11.651586  718600 loader.go:222] Config not found: /home/ubuntu/.kube/config
  Task #2 ERR> E1118 15:47:11.654908  718600 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
  Task #2 ERR> E1118 15:47:11.655257  718600 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
  Task #2 ERR> E1118 15:47:11.656750  718600 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
  Task #2 ERR> E1118 15:47:11.657042  718600 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
  Task #2 ERR> E1118 15:47:11.658447  718600 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
  Task #2 ERR> The connection to the server localhost:8080 was refused - did you specify the right host or port?
  Task #2 OUT>
2024-11-18T15:47:11.563Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: EXEC: exit-status=0
2024-11-18T15:47:11.564Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Executing instruction 3/4: Copying result back to EMS server...
2024-11-18T15:47:11.564Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: DOWNLOAD: /tmp/.APP_ID -> logs/app-identification-result.54.83.127.200--2024.11.18.15.47.09.681.txt
2024-11-18T15:47:11.564Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Downloading file: task #2: remote: /tmp/.APP_ID -> local: logs/app-identification-result.54.83.127.200--2024.11.18.15.47.09.681.txt
2024-11-18T15:47:12.370Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: File download completed: task #2: remote: /tmp/.APP_ID -> local: logs/app-identification-result.54.83.127.200--2024.11.18.15.47.09.681.txt
2024-11-18T15:47:12.371Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: processPatterns: Setting variable 'APPLICATION_ID' to:
2024-11-18T15:47:12.371Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Executing instruction 4/4: Application identification results...
2024-11-18T15:47:12.371Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: LOG: Application Id:
2024-11-18T15:47:12.371Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Installation Instructions set succeeded: Application identifications from cluster node labels
2024-11-18T15:47:12.371Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       :
  ----------------------------------------------------------------------
  Task #2 :  Instruction Set: EPM deployment at application cluster
2024-11-18T15:47:12.372Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Installation Instructions set is skipped due to failed condition: EPM deployment at application cluster
2024-11-18T15:47:12.372Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       :
  ----------------------------------------------------------------------
  Task #2 :  Instruction Set: Netdata deployment at application cluster
2024-11-18T15:47:12.372Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #2: Installation Instructions set is skipped due to failed condition: Netdata deployment at application cluster
2024-11-18T15:47:12.372Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       :
  -------------------------------------------------------------------------
  Task #2 :  Instruction sets processed: successful=1, failed=0, exit-result=SUCCESS
2024-11-18T15:47:12.372Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task completed successfully #2
2024-11-18T15:47:12.374Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Disconnected from remote host: task #2: host: 54.83.127.200:22
2024-11-18T15:47:12.374Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Task execution result #2: success=true
2024-11-18T15:47:12.376Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Client installation Task #2: result=SUCCESS, duration=2694ms
2024-11-18T15:47:12.376Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Sending SUCCESS execution report for Task #2: destination=ems.client.installation.reports, report={reference=9884710e-1951-4ce9-9613-e4743330893c, requestId=35a189d3-65b4-49e1-a612-f3e0a80f5cbb, requestType=INSTALL, deviceIpAddress=54.83.127.200, timestamp=1731944832376, status=SUCCESS, deviceId=92850c0a-a8f0-4f6f-8574-f52ef3214220, nodeInfo={}}
robert-sanfeliu commented 3 days ago

I tried again and got the same results.

This is what EMS says:

    CPU_SOCKETS=1
    CPU_CORES=2
    CPU_PROCESSORS=2
    RAM_TOTAL_KB=424700
    RAM_AVAILABLE_KB=226660
    RAM_FREE_KB=41800
    RAM_USED_KB=382900
    RAM_UTILIZATION=90.1578
    DISK_TOTAL_KB=20141632
    DISK_FREE_KB=18451092
    DISK_USED_KB=1690540
    DISK_UTILIZATION=8.39326
    OS_ARCHITECTURE=aarch64
    OS_KERNEL=Linux
    OS_KERNEL_RELEASE=6.8.0-1015-aws
2024-11-19T16:02:34.102Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task #7: Installation Instructions set succeeded: Detect node features (OS, architecture, cores, RAM, disk etc)
2024-11-19T16:02:34.102Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       :
  -------------------------------------------------------------------------
  Task #7 :  Instruction sets processed: successful=1, failed=0, exit-result=SUCCESS
2024-11-19T16:02:34.102Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Task completed successfully #7
2024-11-19T16:02:34.103Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.i.i.SshClientInstaller       : SshClientInstaller: Disconnected from remote host: task #7: host: 52.90.39.105:22
2024-11-19T16:02:34.103Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Task execution result #7: success=true
2024-11-19T16:02:34.103Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Client installation Task #7: result=SUCCESS, duration=4807ms
2024-11-19T16:02:34.103Z  INFO 30 --- [pool-4-thread-3] g.i.i.e.b.c.install.ClientInstaller      : ClientInstaller: Sending SUCCESS execution report for Task #7: destination=ems.client.installation.reports, report={reference=52340b89-3277-4cba-a54b-822826cc3819, requestId=815f7bc8-4df8-4c44-b38f-d506c2b30488, requestType=DIAGNOSTICS, deviceIpAddress=52.90.39.105, timestamp=1732032154103, status=SUCCESS, deviceId=8514ae55-842f-4943-9ab3-00a02852126c, nodeInfo={CPU_SOCKETS=1, CPU_CORES=2, CPU_PROCESSORS=2, RAM_TOTAL_KB=424700, RAM_AVAILABLE_KB=226660, RAM_FREE_KB=41800, RAM_USED_KB=382900, RAM_UTILIZATION=90.1578, DISK_TOTAL_KB=20141632, DISK_FREE_KB=18451092, DISK_USED_KB=1690540, DISK_UTILIZATION=8.39326, OS_ARCHITECTURE=aarch64, OS_KERNEL=Linux, OS_KERNEL_RELEASE=6.8.0-1015-aws}}

Resource manager:

2024-11-19 16:03:26:288 +0000 [scheduling-1] DEBUG Manager - Registering publisher eu.nebulouscloud.exn.core.Publisher@72aa16be topic://eu.nebulouscloud.monitoring.device_lost
2024-11-19 16:03:29:289 +0000 [scheduling-1] DEBUG Publisher - topic://eu.nebulouscloud.monitoring.device_lost Sending {"device_name":"eut_rob5","timestamp":1732032200}-> null
2024-11-19 16:03:29:291 +0000 [scheduling-1] WARN DeviceProcessor - processFailedDevices: Marked as FAILED device with Id: bb9d1821-df42-43c3-b50d-b3e012bdcac1
2024-11-19 16:03:31:627 +0000 [task-1] WARN SALRegistrationService - SALRegistrationService: processQueue(): Will register device: Device(id=adb52e7c-85d0-48c4-b1ad-800519f460c6, ref=application_id|all-applications|8514ae55-842f-4943-9ab3-00a02852126c, os=LINUX, name=eut_rob8, owner=admin, ipAddress=52.90.39.105, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}, requestId=815f7bc8-4df8-4c44-b38f-d506c2b30488, creationDate=2024-11-19T16:03:31.626875528Z, lastUpdateDate=null, archiveDate=null, status=NEW_DEVICE, nodeReference=5a9ad43a-e686-4591-9ad2-196dfa0a8ec6, messages=[], statusUpdate=null, metrics=null, suspectTimestamp=null, retries=0, registeredToSAL=false)
2024-11-19 16:03:31:627 +0000 [task-1] WARN SALRegistrationService - Using default IP address as the environmental variable was not set or found
2024-11-19 16:03:31:628 +0000 [task-1] WARN SALRegistrationService - SALRegistrationService: processQueue(): EXCEPTION caught. Will not register device to SAL: Device(id=adb52e7c-85d0-48c4-b1ad-800519f460c6, ref=application_id|all-applications|8514ae55-842f-4943-9ab3-00a02852126c, os=LINUX, name=eut_rob8, owner=admin, ipAddress=52.90.39.105, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}, requestId=815f7bc8-4df8-4c44-b38f-d506c2b30488, creationDate=2024-11-19T16:03:31.626875528Z, lastUpdateDate=null, archiveDate=null, status=NEW_DEVICE, nodeReference=5a9ad43a-e686-4591-9ad2-196dfa0a8ec6, messages=[], statusUpdate=null, metrics=null, suspectTimestamp=null, retries=0, registeredToSAL=false)
java.lang.NumberFormatException: Cannot parse null string
        at java.base/java.lang.Integer.parseInt(Unknown Source)
        at java.base/java.lang.Integer.parseInt(Unknown Source)
        at eu.nebulous.resource.discovery.registration.service.SALRegistrationService.register(SALRegistrationService.java:73)
        at eu.nebulous.resource.discovery.registration.service.SALRegistrationService.processQueue(SALRegistrationService.java:198)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
atsag commented 3 days ago

@robert-sanfeliu The DEV environment that you mention, is different from cd? If yes, then perhaps the code being used there is older and we should reproduce the error in cd

robert-sanfeliu commented 3 days ago

I mixed environments, I really ment PROD environment. @atsag : your question is still valid: Yes, code from PROD environment is what is found on r1 branch for the different components. For EMS, that is involved in this bug report, we don't have a clear vision on what is there, as Radek mentions

robert-sanfeliu commented 1 day ago

On PROD environment, after applying latest changes from main, the problem still persists:

  ██████╗ ███████╗███████╗ ██████╗ ██╗   ██╗██████╗  ██████╗███████╗    ██████╗ ██╗███████╗ ██████╗ ██████╗ ██╗   ██╗███████╗██████╗ ██╗   ██╗
  ██╔══██╗██╔════╝██╔════╝██╔═══██╗██║   ██║██╔══██╗██╔════╝██╔════╝    ██╔══██╗██║██╔════╝██╔════╝██╔═══██╗██║   ██║██╔════╝██╔══██╗╚██╗ ██╔╝
  ██████╔╝█████╗  ███████╗██║   ██║██║   ██║██████╔╝██║     █████╗      ██║  ██║██║███████╗██║     ██║   ██║██║   ██║█████╗  ██████╔╝ ╚████╔╝
  ██╔══██╗██╔══╝  ╚════██║██║   ██║██║   ██║██╔══██╗██║     ██╔══╝      ██║  ██║██║╚════██║██║     ██║   ██║╚██╗ ██╔╝██╔══╝  ██╔══██╗  ╚██╔╝
  ██║  ██║███████╗███████║╚██████╔╝╚██████╔╝██║  ██║╚██████╗███████╗    ██████╔╝██║███████║╚██████╗╚██████╔╝ ╚████╔╝ ███████╗██║  ██║   ██║
  ╚═╝  ╚═╝╚══════╝╚══════╝ ╚═════╝  ╚═════╝ ╚═╝  ╚═╝ ╚═════╝╚══════╝    ╚═════╝ ╚═╝╚══════╝ ╚═════╝ ╚═════╝   ╚═══╝  ╚══════╝╚═╝  ╚═╝   ╚═╝

 :: App version ::                        (1.0.2-SNAPSHOT)
 :: Spring Boot ::                        (v3.2.10)
 :: Java (TM)   ::                        (21.0.1)

Nov 22, 2024 8:55:54 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service [Tomcat]
Nov 22, 2024 8:55:54 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet engine: [Apache Tomcat/10.1.30]
Nov 22, 2024 8:55:55 AM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring embedded WebApplicationContext
2024-11-22 08:55:56:161 +0000 [main] WARN GroovyTemplateAutoConfiguration - Cannot find template location: classpath:/templates/ (please add some templates, check your Groovy configuration, or set spring.groovy.template.check-template-location=false)
Nov 22, 2024 9:03:58 AM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring DispatcherServlet 'dispatcherServlet'
2024-11-22 09:07:09:616 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: request: RegistrationRequest(id=49b75a74-a24d-4d68-90eb-bcbb0ee56e30, device=Device(id=9ad16a04-c051-40c1-a7b3-3922f3a126a1, ref=application_id|all-applications|9ad16a04-c051-40c1-a7b3-3922f3a126a1, price=0.0, os=LINUX, name=eut_rob_202411221005, owner=admin, provider=idk, ipAddress=34.203.239.174, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}), requester=admin, requestDate=2024-11-22T09:06:06.210Z, lastUpdateDate=2024-11-22T09:07:05.718Z, archiveDate=null, status=DATA_COLLECTION_REQUESTED, history=[], nodeReference=null, messages=[])
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: nodeInfo: java.util.LinkedHashMap {CPU_SOCKETS=1, CPU_CORES=2, CPU_PROCESSORS=2, RAM_TOTAL_KB=3924284, RAM_AVAILABLE_KB=3570120, RAM_FREE_KB=3519840, RAM_USED_KB=404444, RAM_UTILIZATION=10.3062, DISK_TOTAL_KB=20141632, DISK_FREE_KB=18561000, DISK_USED_KB=1580632, DISK_UTILIZATION=7.84759, OS_ARCHITECTURE=aarch64, OS_KERNEL=Linux, OS_KERNEL_RELEASE=6.8.0-1015-aws}
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: CPU_SOCKETS = 1
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: CPU_SOCKETS = 1
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: CPU_CORES = 2
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: CPU_CORES = 2
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: CPU_PROCESSORS = 2
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: CPU_PROCESSORS = 2
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: RAM_TOTAL_KB = 3924284
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: RAM_TOTAL_KB = 3924284
2024-11-22 09:07:09:626 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: RAM_AVAILABLE_KB = 3570120
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: RAM_AVAILABLE_KB = 3570120
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: RAM_FREE_KB = 3519840
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: RAM_FREE_KB = 3519840
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: RAM_USED_KB = 404444
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: RAM_USED_KB = 404444
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: RAM_UTILIZATION = 10.3062
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: RAM_UTILIZATION = 10.3062
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: DISK_TOTAL_KB = 20141632
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: DISK_TOTAL_KB = 20141632
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: DISK_FREE_KB = 18561000
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: DISK_FREE_KB = 18561000
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: DISK_USED_KB = 1580632
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: DISK_USED_KB = 1580632
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: DISK_UTILIZATION = 7.84759
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: DISK_UTILIZATION = 7.84759
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: OS_ARCHITECTURE = aarch64
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: OS_ARCHITECTURE = aarch64
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: OS_KERNEL = Linux
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: OS_KERNEL = Linux
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse:       Dev-info pair: OS_KERNEL_RELEASE = 6.8.0-1015-aws
2024-11-22 09:07:09:627 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: Dev-info pair ADDED: OS_KERNEL_RELEASE = 6.8.0-1015-aws
2024-11-22 09:08:08:018 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: request: RegistrationRequest(id=49b75a74-a24d-4d68-90eb-bcbb0ee56e30, device=Device(id=9ad16a04-c051-40c1-a7b3-3922f3a126a1, ref=application_id|all-applications|9ad16a04-c051-40c1-a7b3-3922f3a126a1, price=0.0, os=LINUX, name=eut_rob_202411221005, owner=admin, provider=idk, ipAddress=34.203.239.174, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={CPU_SOCKETS=1, CPU_CORES=2, CPU_PROCESSORS=2, RAM_TOTAL_KB=3924284, RAM_AVAILABLE_KB=3570120, RAM_FREE_KB=3519840, RAM_USED_KB=404444, RAM_UTILIZATION=10.3062, DISK_TOTAL_KB=20141632, DISK_FREE_KB=18561000, DISK_USED_KB=1580632, DISK_UTILIZATION=7.84759, OS_ARCHITECTURE=aarch64, OS_KERNEL=Linux, OS_KERNEL_RELEASE=6.8.0-1015-aws}), requester=admin, requestDate=2024-11-22T09:06:06.210Z, lastUpdateDate=2024-11-22T09:08:05.713Z, archiveDate=null, status=ONBOARDING_REQUESTED, history=[], nodeReference=null, messages=[])
2024-11-22 09:08:08:019 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: processResponse: nodeInfo: java.util.LinkedHashMap {}
2024-11-22 09:08:08:019 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: copyDeviceToMonitoring: BEGIN: request: RegistrationRequest(id=49b75a74-a24d-4d68-90eb-bcbb0ee56e30, device=Device(id=9ad16a04-c051-40c1-a7b3-3922f3a126a1, ref=application_id|all-applications|9ad16a04-c051-40c1-a7b3-3922f3a126a1, price=0.0, os=LINUX, name=eut_rob_202411221005, owner=admin, provider=idk, ipAddress=34.203.239.174, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}), requester=admin, requestDate=2024-11-22T09:06:06.210Z, lastUpdateDate=2024-11-22T09:08:08.014Z, archiveDate=null, status=SUCCESS, history=[], nodeReference=2ddeb8a3-81c1-4713-9b0a-a8c120ac00fa, messages=[])
2024-11-22 09:08:08:019 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: copyDeviceToMonitoring: request-DEVICE: Device(id=9ad16a04-c051-40c1-a7b3-3922f3a126a1, ref=application_id|all-applications|9ad16a04-c051-40c1-a7b3-3922f3a126a1, price=0.0, os=LINUX, name=eut_rob_202411221005, owner=admin, provider=idk, ipAddress=34.203.239.174, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={})
2024-11-22 09:08:08:056 +0000 [ActiveMQ Session Task-3] WARN RegistrationRequestProcessor - RegistrationRequestProcessor: copyDeviceToMonitoring: COPIED-DEVICE: Device(id=40f1f102-f147-494f-9f2d-10a6f9724281, sal_id=null, ref=application_id|all-applications|9ad16a04-c051-40c1-a7b3-3922f3a126a1, price=0.0, os=LINUX, name=eut_rob_202411221005, owner=admin, provider=idk, ipAddress=34.203.239.174, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}, requestId=49b75a74-a24d-4d68-90eb-bcbb0ee56e30, creationDate=2024-11-22T09:08:08.022420961Z, lastUpdateDate=null, archiveDate=null, status=NEW_DEVICE, nodeReference=2ddeb8a3-81c1-4713-9b0a-a8c120ac00fa, messages=[], statusUpdate=null, metrics=null, suspectTimestamp=null, retries=0, registeredToSAL=false)
2024-11-22 09:08:08:064 +0000 [task-3] WARN SALRegistrationService - SALRegistrationService: processQueue(): Will register device: Device(id=40f1f102-f147-494f-9f2d-10a6f9724281, sal_id=null, ref=application_id|all-applications|9ad16a04-c051-40c1-a7b3-3922f3a126a1, price=0.0, os=LINUX, name=eut_rob_202411221005, owner=admin, provider=idk, ipAddress=34.203.239.174, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}, requestId=49b75a74-a24d-4d68-90eb-bcbb0ee56e30, creationDate=2024-11-22T09:08:08.022420961Z, lastUpdateDate=null, archiveDate=null, status=NEW_DEVICE, nodeReference=2ddeb8a3-81c1-4713-9b0a-a8c120ac00fa, messages=[], statusUpdate=null, metrics=null, suspectTimestamp=null, retries=0, registeredToSAL=false)
2024-11-22 09:08:08:065 +0000 [task-3] WARN SALRegistrationService - SALRegistrationService: register: BEGIN: device: Device(id=40f1f102-f147-494f-9f2d-10a6f9724281, sal_id=null, ref=application_id|all-applications|9ad16a04-c051-40c1-a7b3-3922f3a126a1, price=0.0, os=LINUX, name=eut_rob_202411221005, owner=admin, provider=idk, ipAddress=34.203.239.174, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}, requestId=49b75a74-a24d-4d68-90eb-bcbb0ee56e30, creationDate=2024-11-22T09:08:08.022420961Z, lastUpdateDate=null, archiveDate=null, status=NEW_DEVICE, nodeReference=2ddeb8a3-81c1-4713-9b0a-a8c120ac00fa, messages=[], statusUpdate=null, metrics=null, suspectTimestamp=null, retries=0, registeredToSAL=false)
2024-11-22 09:08:08:065 +0000 [task-3] WARN SALRegistrationService - Using default IP address (158.39.201.36) to fetch Proactive client jar files from, as the environmental variable was not set or found
2024-11-22 09:08:08:066 +0000 [task-3] WARN SALRegistrationService - SALRegistrationService: register: DEVICE-INFO: {}
2024-11-22 09:08:08:066 +0000 [task-3] WARN SALRegistrationService - SALRegistrationService: processQueue(): EXCEPTION caught. Will not register device to SAL: Device(id=40f1f102-f147-494f-9f2d-10a6f9724281, sal_id=null, ref=application_id|all-applications|9ad16a04-c051-40c1-a7b3-3922f3a126a1, price=0.0, os=LINUX, name=eut_rob_202411221005, owner=admin, provider=idk, ipAddress=34.203.239.174, port=22, location=DeviceLocation(id=null, name=nowhere, continent=null, continentCode=null, country=null, countryCode=null, state=null, stateCode=null, city=null, zipcode=null, address=null, extra=null, latitude=0.0, longitude=0.0), username=ubuntu, deviceInfo={}, requestId=49b75a74-a24d-4d68-90eb-bcbb0ee56e30, creationDate=2024-11-22T09:08:08.022420961Z, lastUpdateDate=null, archiveDate=null, status=NEW_DEVICE, nodeReference=2ddeb8a3-81c1-4713-9b0a-a8c120ac00fa, messages=[], statusUpdate=null, metrics=null, suspectTimestamp=null, retries=0, registeredToSAL=false)
java.lang.NumberFormatException: Cannot parse null string
        at java.base/java.lang.Integer.parseInt(Unknown Source)
        at java.base/java.lang.Integer.parseInt(Unknown Source)
        at eu.nebulous.resource.discovery.registration.service.SALRegistrationService.register(SALRegistrationService.java:78)
        at eu.nebulous.resource.discovery.registration.service.SALRegistrationService.processQueue(SALRegistrationService.java:214)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)