Determine actual SeedLink packet sizes at run time instead of hardcoding SeedLink packet sizes to the SLRECSIZE const at compile time

[yensoon2]

Good morning @chad-iris,

I’m a user of some of your SeedLink software – thank you very much for writing them.

I noticed that slinktool v4.3 (i.e. currently the latest version of slinktool) fails to work with “low-latency” SeedLink servers that serve 256-byte SeedLink packets, because slinktool v4.3 has been hardcoded at compile time to only work with 512-byte SeedLink packets.

This pull request allows slinktool to determine SeedLink packet sizes at run time, so that slinktool can work well with both “normal” SeedLink servers that serve 512-byte SeedLink packets and “low-latency” SeedLink servers that serve 256-byte SeedLink packets, without requiring users to recompile slinktool.

$ slinktool_v4.3 -vv -p -S "DG_TEST" normal-server slinktool version: 4.3 Parsed 1 streams from stream list [normal-server] network socket opened [normal-server] sending: HELLO [normal-server] connected to: SeedLink v3.0 (Normal SeedLink server) [normal-server] capabilities: SLPROTO:3.0 [normal-server] organization: DG.TEST [DG_TEST] sending: STATION TEST DG [DG_TEST] station is OK [DG_TEST] requesting next available data [DG_TEST] DATA/FETCH/TIME command is OK [normal-server] 1 station(s) accepted [normal-server] sending: END 2019.336.15:33:09.0, seq 319, Received Data blockette DG_TEST_00_HHN, 719 samples, 100 Hz, 2019,336,15:32:59.630000 (latency ~2.2 sec) 2019.336.15:33:09.0, seq 320, Received Data blockette DG_TEST_00_HHE, 719 samples, 100 Hz, 2019,336,15:32:59.930000 (latency ~1.9 sec) 2019.336.15:33:09.0, seq 321, Received Data blockette DG_TEST_00_HHZ, 721 samples, 100 Hz, 2019,336,15:32:59.990000 (latency ~1.9 sec) 2019.336.15:33:15.1, seq 322, Received Data blockette DG_TEST_00_HHN, 717 samples, 100 Hz, 2019,336,15:33:06.820000 (latency ~1.9 sec) 2019.336.15:33:15.1, seq 323, Received Data blockette DG_TEST_00_HHE, 718 samples, 100 Hz, 2019,336,15:33:07.120000 (latency ~1.5 sec) 2019.336.15:33:15.1, seq 324, Received Data blockette DG_TEST_00_HHZ, 720 samples, 100 Hz, 2019,336,15:33:07.200000 (latency ~1.5 sec) 2019.336.15:33:22.1, seq 325, Received Data blockette DG_TEST_00_HHN, 720 samples, 100 Hz, 2019,336,15:33:13.990000 (latency ~1.5 sec) 2019.336.15:33:22.1, seq 326, Received Data blockette DG_TEST_00_HHE, 719 samples, 100 Hz, 2019,336,15:33:14.300000 (latency ~1.2 sec) 2019.336.15:33:23.0, seq 327, Received Data blockette DG_TEST_00_HHZ, 719 samples, 100 Hz, 2019,336,15:33:14.400000 (latency ~1.8 sec) ^CTerminating connection [normal-server] network socket closed $ slinktool_v4.3 -vv -p -S "DG_TEST" lowlat-server slinktool version: 4.3 Parsed 1 streams from stream list [lowlat-server] network socket opened [lowlat-server] sending: HELLO [lowlat-server] connected to: SeedLink v3.0 (Low-latency SeedLink server) [lowlat-server] capabilities: SLPROTO:3.0 [lowlat-server] organization: DG.TEST [DG_TEST] sending: STATION TEST DG [DG_TEST] station is OK [DG_TEST] requesting next available data [DG_TEST] DATA/FETCH/TIME command is OK [lowlat-server] 1 station(s) accepted [lowlat-server] sending: END 2019.336.15:35:38.0, seq 365, Received Data blockette DG_TEST_00_HHZ, 70 samples, 100 Hz, 2019,336,15:35:36.900000 (latency ~0.8 sec) error: update_stream(): could not determine sequence number error: update_stream(): could not determine sequence number error: update_stream(): could not determine sequence number error: update_stream(): could not determine sequence number error: update_stream(): could not determine sequence number error: update_stream(): could not determine sequence number error: update_stream(): could not determine sequence number error: update_stream(): could not determine sequence number ^CTerminating connection [lowlat-server] network socket closed $ slinktool_patched -vv -p -S "DG_TEST" normal-server slinktool version: 4.3 Parsed 1 streams from stream list [normal-server] network socket opened [normal-server] sending: HELLO [normal-server] connected to: SeedLink v3.0 (Normal SeedLink server) [normal-server] capabilities: SLPROTO:3.0 [normal-server] organization: DG.TEST [DG_TEST] sending: STATION TEST DG [DG_TEST] station is OK [DG_TEST] requesting next available data [DG_TEST] DATA/FETCH/TIME command is OK [normal-server] 1 station(s) accepted [normal-server] sending: END 2019.336.15:38:18.0, seq 454, Received a 512-byte Data record DG_TEST_00_HHN, 717 samples, 100 Hz, 2019,336,15:38:09.120000 (latency ~1.9 sec) 2019.336.15:38:18.0, seq 455, Received a 512-byte Data record DG_TEST_00_HHZ, 715 samples, 100 Hz, 2019,336,15:38:09.470000 (latency ~1.6 sec) 2019.336.15:38:18.0, seq 456, Received a 512-byte Data record DG_TEST_00_HHE, 719 samples, 100 Hz, 2019,336,15:38:09.490000 (latency ~1.5 sec) 2019.336.15:38:24.1, seq 457, Received a 512-byte Data record DG_TEST_00_HHN, 720 samples, 100 Hz, 2019,336,15:38:16.290000 (latency ~1.5 sec) 2019.336.15:38:24.1, seq 458, Received a 512-byte Data record DG_TEST_00_HHZ, 721 samples, 100 Hz, 2019,336,15:38:16.620000 (latency ~1.2 sec) 2019.336.15:38:25.0, seq 459, Received a 512-byte Data record DG_TEST_00_HHE, 720 samples, 100 Hz, 2019,336,15:38:16.680000 (latency ~1.1 sec) 2019.336.15:38:32.1, seq 460, Received a 512-byte Data record DG_TEST_00_HHN, 720 samples, 100 Hz, 2019,336,15:38:23.490000 (latency ~1.9 sec) 2019.336.15:38:32.1, seq 461, Received a 512-byte Data record DG_TEST_00_HHZ, 721 samples, 100 Hz, 2019,336,15:38:23.830000 (latency ~1.5 sec) 2019.336.15:38:32.1, seq 462, Received a 512-byte Data record DG_TEST_00_HHE, 721 samples, 100 Hz, 2019,336,15:38:23.880000 (latency ~1.5 sec) ^CTerminating connection [normal-server] network socket closed $ slinktool_patched -vv -p -S "DG_TEST" lowlat-server slinktool version: 4.3 Parsed 1 streams from stream list [lowlat-server] network socket opened [lowlat-server] sending: HELLO [lowlat-server] connected to: SeedLink v3.0 (Low-latency SeedLink server) [lowlat-server] capabilities: SLPROTO:3.0 [lowlat-server] organization: DG.TEST [DG_TEST] sending: STATION TEST DG [DG_TEST] station is OK [DG_TEST] requesting next available data [DG_TEST] DATA/FETCH/TIME command is OK [lowlat-server] 1 station(s) accepted [lowlat-server] sending: END 2019.336.15:42:31.0, seq 630, Received a 256-byte Data record DG_TEST_00_HHZ, 70 samples, 100 Hz, 2019,336,15:42:29.900000 (latency ~0.6 sec) 2019.336.15:42:31.0, seq 631, Received a 256-byte Data record DG_TEST_00_HHN, 70 samples, 100 Hz, 2019,336,15:42:29.900000 (latency ~0.6 sec) 2019.336.15:42:31.0, seq 632, Received a 256-byte Data record DG_TEST_00_HHE, 70 samples, 100 Hz, 2019,336,15:42:29.900000 (latency ~0.6 sec) 2019.336.15:42:31.0, seq 633, Received a 256-byte Data record DG_TEST_00_HHZ, 70 samples, 100 Hz, 2019,336,15:42:30.600000 (latency ~0.1 sec) 2019.336.15:42:31.0, seq 634, Received a 256-byte Data record DG_TEST_00_HHN, 70 samples, 100 Hz, 2019,336,15:42:30.600000 (latency ~0.1 sec) 2019.336.15:42:31.0, seq 635, Received a 256-byte Data record DG_TEST_00_HHE, 70 samples, 100 Hz, 2019,336,15:42:30.600000 (latency ~0.1 sec) 2019.336.15:42:32.0, seq 636, Received a 256-byte Data record DG_TEST_00_HHZ, 70 samples, 100 Hz, 2019,336,15:42:31.300000 (latency ~0.1 sec) 2019.336.15:42:32.0, seq 637, Received a 256-byte Data record DG_TEST_00_HHN, 70 samples, 100 Hz, 2019,336,15:42:31.300000 (latency ~0.1 sec) 2019.336.15:42:32.0, seq 638, Received a 256-byte Data record DG_TEST_00_HHE, 70 samples, 100 Hz, 2019,336,15:42:31.300000 (latency ~0.1 sec) ^CTerminating connection [lowlat-server] network socket closed $

Can you please consider including this pull request into future versions of slinktool?

Thank you,

@yen-soon

[jasondec]

Hello @yensoon2. Thank you for working on this feature. I'm also looking to use slinktool with 128 and 256-byte records. I tried running your code but hit an error during compile.

slinktool.c:126:63: warning: implicit declaration of function 'sl_recsize' is invalid in C99 [-Wimplicit-function-declaration]
    packet_handler ((char *)&slpack->msrecord, ptype, seqnum, sl_recsize (slpack, SLRECSIZE));

1 warning generated.
cc -o ../slinktool dsarchive.o archive.o slinkxml.o slinktool.o -L../libslink -L../ezxml -lslink -lezxml
Undefined symbols for architecture x86_64:
  "_sl_recsize", referenced from:
      _main in slinktool.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [../slinktool] Error 1
make: *** [all] Error 2

[OSX 10.13.6] I don't see sl_recsize defined anywhere. Are you using a modified version of libslink as well?

[chad-earthscope]

I'm working on an update to libslink for general modernization including the ability to receive and automatically detect variable record lengths. Automatic record length detection was completed in this commit. Once this is released I will update slinktool.

[jasondec]

Thanks @chad-iris