canfar / arcade

ALMA Retrieval with the CANFAR Data System
GNU General Public License v3.0
1 stars 2 forks source link

CARTA exiting unexpectedly #35

Open brianmajor opened 3 years ago

brianmajor commented 3 years ago

For some users their CARTA session will unexpectedly exit. There isn't much in the logs to help explain the reason for the exit. Here are two examples of the end of the log file:

Session 1 (Tue Sep  1 20:11:46 2020): Client 1 [::ffff:10.233.73.13] Connected. Num sessions: 1
2020-09-01 20:12:57 INFO    FITSCoordinateUtil::fromFITSHeader  passing empty or nonexistant spectral Coordinate axis
2020-09-01 20:13:11 INFO    RegionTextParser::_determineVersion RegionTextParser::_determineVersion: Found spec version 0
2020-09-01 20:13:11 INFO    AsciiRegionFileParser::_parse   Combined 1 image regions (which excludes any annotation regions)
2020-09-01 20:13:11 INFO    AsciiRegionFileParser::_parse   The specified region will select all pixels that are included in the region. Full pixels will be included even when they are only partially covered by the region(s).
2020-09-01 20:13:17 INFO    RegionTextParser::_determineVersion RegionTextParser::_determineVersion: Found spec version 0
2020-09-01 20:13:17 INFO    AsciiRegionFileParser::_parse   Combined 1 image regions (which excludes any annotation regions)
2020-09-01 20:13:17 INFO    AsciiRegionFileParser::_parse   The specified region will select all pixels that are included in the region. Full pixels will be included even when they are only partially covered by the region(s).
Tue Sep  1 20:13:34 UTC 2020 arcade-carta DONE
Smoothed with smoothing factor of 4 and kernel size of 7 in 1.854 ms at 123.539 MPix/s
Contoured 499x459 image in 1.128 ms at 203.051 MPix/s. Found 2500 vertices in 2 segments across 1 levels
Session 2 (Tue Sep  1 12:58:50 2020): Client 2 [::ffff:10.233.73.13] Disconnected. Remaining sessions: 1
No remaining sessions.
Session 3 (Tue Sep  1 14:45:17 2020): Client 3 [::ffff:10.233.73.13] Connected. Num sessions: 1
2020-09-01 14:45:53 INFO    FITSCoordinateUtil::fromFITSHeader  passing empty or nonexistant spectral Coordinate axis
2020-09-01 14:46:04 INFO    FITSCoordinateUtil::fromFITSHeader  passing empty or nonexistant spectral Coordinate axis
2020-09-01 14:46:19 INFO    FITSCoordinateUtil::fromFITSHeader  passing empty or nonexistant spectral Coordinate axis
Tue Sep  1 16:49:55 UTC 2020 arcade-carta DONE

The CARTA process is executed and monitored by a wrapper script, so it's possible the script is exiting prematurely and thus completing (terminating) the CARTA kubernetes job.

https://github.com/opencadc/arcade/blob/master/session-containers/arcade-carta/src/arcade-carta