Rules for Connecting

[GoogleCodeExporter]

I routinely accidentally connect the benchmark twice (instead of 1 benchmark 
and 1 agent) for 
example.

The glue accepts both connections.  This I'm pretty sure is the wrong behavior.

I think if one agent/environment/experiment is connected, further connections 
from others should 
be either ignored, rejected, or something.

Original issue reported on code.google.com by bttan...@gmail.com on 24 Jul 2007 at 3:11

[GoogleCodeExporter]

Further connections can't be ignored or rejected because we allow people to swap
Agents and Environments prior to starting the experiment.  We can shut down the
original connection (and I believe we do...? I'll have to look into that.)

Original comment by andrewbutcher@gmail.com on 24 Jul 2007 at 5:48

[GoogleCodeExporter]

I think the existing code does not shut down the original connection : at least 
in my
experience with Java environments, after multiple environments connecting, they 
are
still alive which makes me think their connections were not terminated.

I want to rethink this whole line of planning though.  The auto-reconnect and
swapping of agents *seemed* like a quick and easy feature to add... but there 
are
ramifications now and I think perhaps it was ill conceived.  Perhaps whatever
behavior run-time component swapping behavior (connecting an agent, 
disconnecting
him, then connecting another one before the experiment starts) is a bad idea in
general and if necessary should be handled through RL_agent_message and other 
magics?

I see this as code bloat where we have a feature with potential side effects 
that is
not currently used for anything.

Original comment by brian.ta...@gmail.com on 24 Jul 2007 at 5:56

[GoogleCodeExporter]

It turns out I wasn't killing the original connection. Good find, thanks.
We can't send the kRLTerm message back to the user experiment that we send to 
the
agent/environment codecs to tell them to shut down because the user experiment 
won't
know how to deal with it. That should be fine, and closing the connection 
should be
enough.  Thoughts?

Original comment by andrewbutcher@gmail.com on 24 Jul 2007 at 5:57

[GoogleCodeExporter]

I think this might be a good time to scrap this feature and put it as a future
desire.  Basically, this feature was added at a time when we liked the idea of
running RL_Glue as a server. 

The scheme was, RL_Glue would run forever.  At any time, an experiment, agent, 
and
benchmark could connect together, run, and then disconnect.  The idea was also 
thrown
around that perhaps the environment could stay connected for a long time and
different agents could connect to it.

Or the same agent could stay connected and several environments could be 
connected
and disconnected for multiple environment benchmarking.

It was conceived with network competition in mind.  We're not doing that now.  I
think that to do this particular set of features *right* will require more 
planning.
     In the meantime, this sort of thing can be facilitated with the messaging protocol.

So, I propose that we take the ability for the environment to accept multiple
connections of a given type out of the code for now with the intention of 
adding that
back in later when we know what we want.  For now, I think RL-Glue should be a 
1-shot
deal.  You connect an agent/env/experiment and run until the experiment 
concludes, at
which time everything disconnects and quits.

Original comment by brian.ta...@gmail.com on 24 Jul 2007 at 6:10

[GoogleCodeExporter]

Brian and I discussed, and we think this is sorted out now.  If it arises again,
we'll add a new issue.

Original comment by andrewbutcher@gmail.com on 26 Jul 2007 at 7:52

• Changed state: Fixed