search

seafloor-geodesy / gnatss

Community Seafloor Global Navigation Satellite Systems - Acoustic (GNSS-A) Transponder Surveying Software
https://gnatss.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
10 stars 14 forks source link

fix: Research into core dumped error when in remote server and fix it #207

Closed lsetiawan closed 8 months ago

lsetiawan commented 9 months ago 
          I found another bug. If I run gnatss on a remote server that I have logged into with the ssh -Y command I get the following error:
Saving the distance from center file to /wd4/GNATSS_tests/NDP1_2022/output/dist_center.csv
Saving the latest residuals to /wd4/GNATSS_tests/NDP1_2022/output/residuals.csv
Saving the process results dataset to /wd4/GNATSS_tests/NDP1_2022/output/process_dataset.nc
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, minimal, minimalegl, offscreen, vnc, webgl, xcb.

Aborted (core dumped)

Logging in with ssh -X fixes this.

Originally posted by @johnbdesanto in https://github.com/seafloor-geodesy/gnatss/issues/175#issuecomment-1821909994

lsetiawan commented 9 months ago

@johnbdesanto when you're back in the office and have access to the server, please confirm and identify the following:

  1. That the output files mentioned in the console log are there.
  2. Please list out the spec of the server that is being used, mainly the operating system used and it's version.
madhavmk commented 9 months ago

I was unable to replicate this bug on my VirtualBox Ubuntu 22.04.3 LTS instance running locally:

ssh command

ssh -Y -p 2222 madhavmk@127.0.0.1

Commands run inside VirtualBox Ubuntu instance

madhavmk@Ubuntu-22:~/Desktop/gnatss$ gnatss run --qc --config-yaml tests/data/config.yaml
Loading configuration ...
/home/madhavmk/.local/lib/python3.10/site-packages/gnatss/configs/main.py:54: UserWarning: Configuration file `config.yaml` not found. Will attempt to retrieve configuration from environment variables.
  warnings.warn(
/home/madhavmk/.local/lib/python3.10/site-packages/gnatss/loaders.py:23: UserWarning: Loading attempt failed, trying to load configuration from file path given.
  warnings.warn(
Configuration loaded.
Gathering sound_speed at ./tests/data/2022/NCL1/ctd/CTD_NCL1_Ch_Mi
Gathering travel_times at ./tests/data/2022/NCL1/**/WG_*/pxp_tt
Gathering gps_solution at ./tests/data/2022/NCL1/**/posfilter/POS_FREED_TRANS_TWTT
Load sound speed profile data...
Computing harmonic mean...
lat=45.302064471 lon=-124.978181346 height=-1176.5866 internal_delay=0.2 sv_mean=1481.551 pxp_id='NCL1-1' azimuth=-178.48 elevation=39.99
lat=45.295207747 lon=-124.958752845 height=-1146.5881 internal_delay=0.32 sv_mean=1481.521 pxp_id='NCL1-2' azimuth=-55.73 elevation=39.75
lat=45.309643593 lon=-124.959348875 height=-1133.7305 internal_delay=0.44 sv_mean=1481.509 pxp_id='NCL1-3' azimuth=59.01 elevation=40.02
Finished computing harmonic mean
Load deletions data...
Load quality controls data...
Load travel times...
Cleaning travel times data...
Load GPS data...
Cross referencing transmit, reply, and gps solutions...
Filtering out data outside of distance limit...
Preparing data inputs...
Perform solve...
--- 8006 epochs, 24018 measurements ---
After iteration: 1, rms residual = 390.09 cm, error factor = 71.597
NCL1-1
D_x = 5.254253e-01 m, Sigma(x) = 2.513651e-04 m
D_y = -9.612451e-01 m, Sigma(y) = 2.587962e-04 m
D_z = 4.799863e-01 m, Sigma(z) = 2.617288e-04 m
NCL1-2
D_x = 5.254253e-01 m, Sigma(x) = 2.513651e-04 m
D_y = -9.612451e-01 m, Sigma(y) = 2.587962e-04 m
D_z = 4.799863e-01 m, Sigma(z) = 2.617288e-04 m
NCL1-3
D_x = 5.254253e-01 m, Sigma(x) = 2.513651e-04 m
D_y = -9.612451e-01 m, Sigma(y) = 2.587962e-04 m
D_z = 4.799863e-01 m, Sigma(z) = 2.617288e-04 m

After iteration: 2, rms residual = 355.36 cm, error factor = 65.189
NCL1-1
D_x = -5.768502e-04 m, Sigma(x) = 2.513408e-04 m
D_y = 1.855479e-05 m, Sigma(y) = 2.587953e-04 m
D_z = 2.859777e-04 m, Sigma(z) = 2.617279e-04 m
NCL1-2
D_x = -5.768502e-04 m, Sigma(x) = 2.513408e-04 m
D_y = 1.855479e-05 m, Sigma(y) = 2.587953e-04 m
D_z = 2.859777e-04 m, Sigma(z) = 2.617279e-04 m
NCL1-3
D_x = -5.768502e-04 m, Sigma(x) = 2.513408e-04 m
D_y = 1.855479e-05 m, Sigma(y) = 2.587953e-04 m
D_z = 2.859777e-04 m, Sigma(z) = 2.617279e-04 m

After iteration: 3, rms residual = 355.36 cm, error factor = 65.188
NCL1-1
D_x = 2.359752e-08 m, Sigma(x) = 2.513408e-04 m
D_y = -9.835729e-09 m, Sigma(y) = 2.587952e-04 m
D_z = 2.45057e-08 m, Sigma(z) = 2.617279e-04 m
NCL1-2
D_x = 2.359753e-08 m, Sigma(x) = 2.513408e-04 m
D_y = -9.83572e-09 m, Sigma(y) = 2.587952e-04 m
D_z = 2.450569e-08 m, Sigma(z) = 2.617279e-04 m
NCL1-3
D_x = 2.359753e-08 m, Sigma(x) = 2.513408e-04 m
D_y = -9.835728e-09 m, Sigma(y) = 2.587952e-04 m
D_z = 2.45057e-08 m, Sigma(z) = 2.617279e-04 m

---- FINAL SOLUTION ----
NCL1-1
x = -2575644.9546 +/- 2.513408e-04 m del_e = 0.9811 +/- 2.564246e-04 m
y = -3681386.8627 +/- 2.587952e-04 m del_n = -0.0082 +/- 2.64908e-04 m
z = 4510168.4296 +/- 2.617279e-04 m del_u = 0.6837 +/- 2.504375e-04 m
Lat. = 45.3020643975038 deg, Long. = -124.9781688348579, Hgt.msl = -1175.9028640831987 m
NCL1-2
x = -2574718.8687 +/- 2.513408e-04 m del_e = 0.9809 +/- 2.564215e-04 m
y = -3682721.1541 +/- 2.587952e-04 m del_n = -0.0083 +/- 2.649103e-04 m
z = 4509653.8242 +/- 2.617279e-04 m del_u = 0.684 +/- 2.504381e-04 m
Lat. = 45.2952076721127 deg, Long. = -124.95874033752962, Hgt.msl = -1145.9041291070164 m
NCL1-3
x = -2574109.0192 +/- 2.513408e-04 m del_e = 0.9809 +/- 2.564216e-04 m
y = -3681767.3029 +/- 2.587952e-04 m del_n = -0.0085 +/- 2.649083e-04 m
z = 4510791.2117 +/- 2.617279e-04 m del_u = 0.684 +/- 2.504401e-04 m
Lat. = 45.309643516627474 deg, Long. = -124.95933636431563, Hgt.msl = -1133.0465384024333 m
------------------------

There are 0 outliers found during this run.

Saving the latest residuals to /home/madhavmk/Desktop/gnatss/tests/data/output/residuals.csv

madhavmk@Ubuntu-22:~/Desktop/gnatss/tests/data/output$ ls
deletions.csv  residuals.csv  residuals_enu_components.png  residuals.png
lsetiawan commented 9 months ago

@madhavmk Thanks for looking into this. Could you possibly try a CentOS machine in virtual box?

johnbdesanto commented 9 months ago

I was able to replicate this issue on our machine after updating gnatss to the current 0.1.0 version.

The output to the terminal window is:

Saving the distance from center file to /wd4/GNATSS_tests/NCL1_2023/output/dist_center.csv
Saving the latest residuals to /wd4/GNATSS_tests/NCL1_2023/output/residuals.csv
Saving the latest residual outliers to /wd4/GNATSS_tests/NCL1_2023/output/outliers.csv
Saving the process results dataset to /wd4/GNATSS_tests/NCL1_2023/output/process_dataset.nc
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, minimal, minimalegl, offscreen, vnc, webgl, xcb.

Aborted (core dumped)

The created files are:

(gipsyx) [jdesanto@geod-proc01-prd-a056 NCL1_2023]$ ls output/
dist_center.csv  outliers.csv  process_dataset.nc  residuals.csv

As you can see, the plots are not being generated but the other files are and I have verified they have not been corrupted.

The version of linux our machine is running is an equivalent to CentOS 8.8, see below:

(gipsyx) [jdesanto@geod-proc01-prd-a056 NCL1_2023]$ cat /etc/os-release 
NAME="Rocky Linux"
VERSION="8.8 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.8 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.8"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.8"
madhavmk commented 9 months ago

I was unable to reproduce this bug on a CentOS 8 machine with gnatss version 0.1.0, python version 3.10.13 I have attached the logs below:

PS C:\Users\Madhav> ssh -Y -p 4422 madhav@localhost
madhav@localhost's password:
Activate the web console with: systemctl enable --now cockpit.socket

Last login: Sun Jan  7 14:16:09 2024 from 10.0.2.2
(base) [madhav@localhost ~]$ cat /etc/os-release
NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Stream 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
(base) [madhav@localhost ~]$ pip show gnatss
WARNING: Package(s) not found: gnatss
(base) [madhav@localhost ~]$ conda activate gnatss
(gnatss) [madhav@localhost ~]$ python --version
Python 3.10.13
(gnatss) [madhav@localhost ~]$ pip show gnatss
Name: gnatss
Version: 0.1.0
Summary: Community Seafloor Global Navigation Satellite Systems - Acoustic (GNSS-A) Transponder Surveying Software
Home-page:
Author:
Author-email:
License:
Location: /home/madhav/anaconda3/envs/gnatss/lib/python3.10/site-packages
Requires: astropy, cftime, fsspec, matplotlib, nptyping, numba, numpy, pandas, pluggy, pydantic, pydantic-settings, pymap3d, pyproj, pyyaml, scipy, typer, xarray
Required-by:
(gnatss) [madhav@localhost ~]$ cd Desktop/
(gnatss) [madhav@localhost Desktop]$ cd gnatss/
(gnatss) [madhav@localhost gnatss]$ gnatss run --qc --config-yaml tests/data/config.yaml
Loading configuration ...
/home/madhav/anaconda3/envs/gnatss/lib/python3.10/site-packages/gnatss/configs/main.py:54: UserWarning: Configuration file `config.yaml` not found. Will attempt to retrieve configuration from environment variables.
  warnings.warn(
/home/madhav/anaconda3/envs/gnatss/lib/python3.10/site-packages/gnatss/loaders.py:23: UserWarning: Loading attempt failed, trying to load configuration from file path given.
  warnings.warn(
Configuration loaded.
Gathering sound_speed at ./tests/data/2022/NCL1/ctd/CTD_NCL1_Ch_Mi
Gathering travel_times at ./tests/data/2022/NCL1/**/WG_*/pxp_tt
Gathering gps_solution at ./tests/data/2022/NCL1/**/posfilter/POS_FREED_TRANS_TWTT
Load sound speed profile data...
Computing harmonic mean...
lat=45.302064471 lon=-124.978181346 height=-1176.5866 internal_delay=0.2 sv_mean=1481.551 pxp_id='NCL1-1' azimuth=-178.48 elevation=39.99
lat=45.295207747 lon=-124.958752845 height=-1146.5881 internal_delay=0.32 sv_mean=1481.521 pxp_id='NCL1-2' azimuth=-55.73 elevation=39.75
lat=45.309643593 lon=-124.959348875 height=-1133.7305 internal_delay=0.44 sv_mean=1481.509 pxp_id='NCL1-3' azimuth=59.01 elevation=40.02
Finished computing harmonic mean
Load deletions data...
Load quality controls data...
Load travel times...
Cleaning travel times data...
Load GPS data...
Cross referencing transmit, reply, and gps solutions...
Filtering out data outside of distance limit...
Preparing data inputs...
Perform solve...
--- 8007 epochs, 24021 measurements ---
After iteration: 1, rms residual = 1736.56 cm, error factor = 391.275
NCL1-1
D_x = 6.392306e-01 m, Sigma(x) = 2.513480e-04 m
D_y = -1.109283e+00 m, Sigma(y) = 2.587786e-04 m
D_z = 5.417462e-01 m, Sigma(z) = 2.617112e-04 m
NCL1-2
D_x = 6.392306e-01 m, Sigma(x) = 2.513480e-04 m
D_y = -1.109283e+00 m, Sigma(y) = 2.587786e-04 m
D_z = 5.417462e-01 m, Sigma(z) = 2.617112e-04 m
NCL1-3
D_x = 6.392306e-01 m, Sigma(x) = 2.513480e-04 m
D_y = -1.109283e+00 m, Sigma(y) = 2.587786e-04 m
D_z = 5.417462e-01 m, Sigma(z) = 2.617112e-04 m

After iteration: 2, rms residual = 1727.21 cm, error factor = 389.798
NCL1-1
D_x = -7.702894e-04 m, Sigma(x) = 2.513201e-04 m
D_y = 1.641293e-05 m, Sigma(y) = 2.587779e-04 m
D_z = 3.868512e-04 m, Sigma(z) = 2.617106e-04 m
NCL1-2
D_x = -7.702894e-04 m, Sigma(x) = 2.513201e-04 m
D_y = 1.641293e-05 m, Sigma(y) = 2.587779e-04 m
D_z = 3.868512e-04 m, Sigma(z) = 2.617106e-04 m
NCL1-3
D_x = -7.702894e-04 m, Sigma(x) = 2.513201e-04 m
D_y = 1.641293e-05 m, Sigma(y) = 2.587779e-04 m
D_z = 3.868512e-04 m, Sigma(z) = 2.617106e-04 m

After iteration: 3, rms residual = 1727.21 cm, error factor = 389.798
NCL1-1
D_x = 2.984966e-08 m, Sigma(x) = 2.513201e-04 m
D_y = -1.232617e-08 m, Sigma(y) = 2.587779e-04 m
D_z = 3.374132e-08 m, Sigma(z) = 2.617106e-04 m
NCL1-2
D_x = 2.984966e-08 m, Sigma(x) = 2.513201e-04 m
D_y = -1.232618e-08 m, Sigma(y) = 2.587779e-04 m
D_z = 3.374133e-08 m, Sigma(z) = 2.617106e-04 m
NCL1-3
D_x = 2.984966e-08 m, Sigma(x) = 2.513201e-04 m
D_y = -1.232617e-08 m, Sigma(y) = 2.587779e-04 m
D_z = 3.374133e-08 m, Sigma(z) = 2.617106e-04 m

---- FINAL SOLUTION ----
NCL1-1
x = -2575644.841 +/- 2.513201e-04 m del_e = 1.159 +/- 2.564037e-04 m
y = -3681387.0107 +/- 2.587779e-04 m del_n = -0.0046 +/- 2.6489e-04 m
z = 4510168.4914 +/- 2.617106e-04 m del_u = 0.7672 +/- 2.504211e-04 m
Lat. = 45.302064429755646 deg, Long. = -124.97816656548731, Hgt.msl = -1175.8193834619506 m
NCL1-2
x = -2574718.7551 +/- 2.513201e-04 m del_e = 1.1589 +/- 2.564007e-04 m
y = -3682721.3021 +/- 2.587779e-04 m del_n = -0.0048 +/- 2.648923e-04 m
z = 4509653.8861 +/- 2.617106e-04 m del_u = 0.7675 +/- 2.504217e-04 m
Lat. = 45.295207704068396 deg, Long. = -124.9587380686862, Hgt.msl = -1145.8206064692506 m
NCL1-3
x = -2574108.9056 +/- 2.513201e-04 m del_e = 1.1589 +/- 2.564007e-04 m
y = -3681767.4509 +/- 2.587779e-04 m del_n = -0.005 +/- 2.648903e-04 m
z = 4510791.2735 +/- 2.617106e-04 m del_u = 0.7675 +/- 2.504237e-04 m
Lat. = 45.30964354840547 deg, Long. = -124.95933409489345, Hgt.msl = -1132.9630161752702 m
------------------------

There are 1 outliers found during this run.
This is 0.01% of the total number of data points.
Please re-run the program again to remove these outliers.

Saving the latest residuals to /home/madhav/Desktop/gnatss/tests/data/output/residuals.csv
Saving the latest residual outliers to /home/madhav/Desktop/gnatss/tests/data/output/outliers.csv
(gnatss) [madhav@localhost gnatss]$
madhavmk commented 9 months ago

Hello @lsetiawan , The bug does not seem to be reproducible on a CentOS 8 machine (see above logs). Please suggest further actions.

lsetiawan commented 8 months ago

I think since this bug is not reproducible, we'll close this for now and if it comes up again, we'll worry about it then.