streamids limited to 100 chars

[crotwell]

As far as I can tell, the datalink protocol places no limit on the length of the streamids, other then they have to fit within the 255 header limit. But dlclient.c, about line 611 assumes max length of 100. Probably 255 would be safer.

char replystr[100]; char streamid[100]; char flags[100];

[chad-earthscope]

True, the stream ID length limit is determined by the header limit combined with the length of the other fields (packed ID, flags, packed size, ...), which also depends on whether it's a WRITE or PACKET, er, packet.

ringserver actually limits the stream IDs to 60 bytes:

/* Define a maximum stream ID string length */
#define MAXSTREAMID 60

which, honestly is quite large. This:

IU_ANMO_00_BH1/MSEED

is 20 bytes, and that can be tripled, just to visualize that:

IU_ANMO_00_BH1/MSEEDIU_ANMO_00_BH1/MSEEDIU_ANMO_00_BH1/MSEED

Even if we went full crazy with forthcoming FDSN URI-style identifiers, it's hard to imagine that 60 bytes doesn't buy a lot of head room.

So in this case ringserver is imposing a limit less than the protocol itself. I'll review making the messages bigger and think about the stream IDs too.

Increasing the length of the stream IDs in ringserver does have some implications though: each packet has a buffer for the maximum length ID (memory is not dynamically allocated for each stream ID for performance and simplicity), this directly impacts the amount of RAM and disk that is used to store the ring buffer. So for a ringserver like IRIS that is maxed out, having 16,777,215 packets in the buffer, increasing the stream ID by another 60 bytes adds 959 MiB of needed space to store the buffer.

[crotwell]

Whatever restrictions on length are fine, but would be nice also to have that in the datalink protocol documentation.

Also, there are some weirdos out there that are trying to use the ringserver to send non-fdsn, non-miniseed data, and in that case the net_sta_loc_chan/MSEED streamid pattern may not apply. Its just a generic buffer, right? :)

[chad-earthscope]

Whatever restrictions on length are fine, but would be nice also to have that in the datalink protocol documentation.

That would be the wrong place for it.

Also, there are some weirdos out there that are trying to use the ringserver to send non-fdsn, non miniseed data, and in that case the net_sta_loc_chan/MSEED streamid pattern may not apply. Its just a generic buffer, right? :)

Of course, which is why it should be documented in ringserver, which is imposing the limit.

[chad-earthscope]

In addition to documentation, there should be a clear error message if a stream ID longer than supported is submitted. Need to verify that.