Monitoring a source which has multiple RingBuffers doesn't work

[GoogleCodeExporter]

When an RBNB source has more than one RingBuffer, and a sink is monitoring
channels from the multiple RingBuffers in this source, the server will not
reliably deliver the most recent data from every channel.  Conversely,
monitoring a source with multiple channels in one RingBuffer will not have
this problem.

Example:
A source, "Foo", has 3 channels (A, B, C) which are individually flushed,
one right after the other (thus Foo will contain 3 RingBuffers - one for
each channel).  A sink wants to monitor all 3 channels from Foo.  What we
would like to see: on the first fetch, the sink retrieves the most recent
data on channel A; on the second fetch the most recent data on channel B,
and on the third fetch the most recent data on channel C.  Then the process
will repeat, getting an update on A/B/C for each three fetches.  Instead,
what we will see (due to this problem with Monitor mode) is that the single
most recent frame flushed to the source will be returned.  Thus, suppose
the source performs the three flushes A/B/C, and then the sink does a fetch
- the sink will get the most recent "C" frame (because that was the last
channel flushed).  Then the source does another set of flushes on A/B/C;
then the sink does another fetch - again, it will get the  most recent "C"
frame.  This process could continue, and the sink would only get updates on
"C".

Background: A RingBuffer is an internal RBNB data storage object which can
contain data for either a single channel or multiple channels
(multiplexed). A source can have one or more RingBuffers, as dictated by
the number of flushes performed to update the complete set of data channels
in the source.  For instance, if all channels are updated in one ChannelMap
(one flush), then the Source will only have one RingBuffer.  Alternatively,
if some of the channels are flushed at one point, and the rest of the
channels are flushed at another time, then the source will contain 2
RingBuffers.

Fixing this problem will involve RBNB server code changes.  Monitor
requests are handled by com.rbnb.api.StreamRBOListener.  I think the
problem is due to the fact that StreamRBOListener handles all channels
being monitored from an entire source - thus, it is a source-level object,
not a RingBuffer-level object.  StreamRBOListener will keep track of the
most recent frame for an entire source, not the most recent frame for each
RingBuffer.

This problem was observed when working with a source where each channel was
in its own RingBuffer (thus, each channel was individually flushed).  This
type of source/RingBuffer arrangement is very efficient for data playback.

One application which is affected by this problem is RDV (Realtime Data
Viewer; http://it.nees.org/software/rdv/index.php), which uses Monitor to
implement realtime mode.  (As an aside, an alternative to using Monitor
mode would be request/response mode in a "sleepy loop", which is what
rbnbPlot uses for realtime mode).

Here are a couple of Matlab/COMSOL functions that can be used to
demonstrate this problem:

function testSource()
src = com.rbnb.sapi.Source;
src.OpenRBNBConnection('localhost:3333','Foo');
for i=0:1000
   disp(['i = ' num2str(i)]);
   cm = javaObject('com.rbnb.sapi.ChannelMap');
   cm.Add('A');
   intVal = i*3;
   cm.PutDataAsInt32(0,intVal);
   src.Flush(cm);
   cm = javaObject('com.rbnb.sapi.ChannelMap');
   cm.Add('B');
   intVal = intVal + 1;
   cm.PutDataAsInt32(0,intVal);
   src.Flush(cm);
   cm = javaObject('com.rbnb.sapi.ChannelMap');
   cm.Add('C');
   intVal = intVal + 1;
   cm.PutDataAsInt32(0,intVal);
   src.Flush(cm);
   pause(1);
end

function testSink()
snk = com.rbnb.sapi.Sink;
snk.OpenRBNBConnection('localhost:3333','Snk');
cm = javaObject('com.rbnb.sapi.ChannelMap');
cm.Add('Foo/A');
cm.Add('Foo/B');
cm.Add('Foo/C');
snk.Monitor(cm,0);
for j=0:1000
    cm = snk.Fetch(100000);
    if (cm.NumberOfChannels > 0)
        disp(['Got ' num2str(cm.NumberOfChannels) ' channels:']);
        for k=0:(cm.NumberOfChannels - 1)
        dataStr = num2str(cm.GetDataAsInt32(k));
            disp(['    Got data on channel ' char(cm.GetName(k)) ': '
dataStr]);
    end
    end
end

Original issue reported on code.google.com by john.wil...@erigo.com on 26 Jun 2008 at 5:55

[GoogleCodeExporter]

Could you add a JUnit test to verify that the correct behavior now occurs?

Original comment by enfield....@gmail.com on 28 Aug 2008 at 1:05

[GoogleCodeExporter]

The correct behavior has been verified by the sample Matlab/COMSOL test scripts
provided in the issue description, therefore I am changing the status of this 
issue
to "Fixed".  Please note, however, that the "Fixed" status still implies that 
the
issue should be verified by the RBNB community (this is one of the reasons for 
the
new beta release today, V3.2B1).

Bill made a good suggestion about JUnit, and I'd like to open the discussion up 
to
the RBNB community.  I wonder if this case might be tricky to check using JUnit 
due
to the following:

1. The test involves multiple networked applications (Server, Source, and Sink);
looking at Bill's first JUnit test included with the RBNB, it appears that this 
is
possible.

2. This is a  Monitor issue, thus each fetch will not necessarily result in a 
clearly
defined ChannelMap - that is, it is OK if one or more channels are missing from 
each
ChannelMap.

Speaking on testing more generally, we have a fairly extensive existing 
Matlab/COMSOL
test suite, and the start of a JUnit test suite.  While the JUnit tests may 
indeed be
the direction we should go in, I don't want to abandon the Matlab test suite 
without
having us give it careful thought.  A possible topic of discussion for the 
upcoming
RBNB conference in October?

Original comment by john.wil...@erigo.com on 28 Aug 2008 at 2:05

• Changed state: Fixed

[GoogleCodeExporter]

Problems with the MATLAB test suite:

1) You have to have MATLAB.
2) It is not integrated with the build system.

Addressing your concerns:
1) Yes, it is possible.
2) I'm sure if you design your test carefully it will work.

Regression tests are necessary components of any serious system.

Original comment by enfield....@gmail.com on 28 Aug 2008 at 2:24