limit the time an in-channel is using

[GoogleCodeExporter]

the number of files read into a channel is already been limited.
in case of big files, the volume can be problematic.

the max number of bytes to read in can be configured in the GUI (per channel).
if this is not configured, the overall max number of bytes (from bots.ini) is 
used.

Original issue reported on code.google.com by hjebb...@gmail.com on 12 Sep 2011 at 10:37

[GoogleCodeExporter]

Hi henk-jan,
I have been thinking about this...

The problem is that communication can take a long time, but there are many 
variables. File sizes, link speed, server response speed etc. But the one thing 
we want to control is the time taken.

So instead of limiting the number of bytes, and/or number of files, I think it 
is better to limit the time. Before the file retrieval loop, calculate a cutoff 
time (now + n seconds), then check before each file and break out of the loop 
if cutoff time is exceeded. This logic ensures it adapts to varying conditions 
and can only exceed the allowed by a small amount (time while 1 file is 
transferring). Also means only 1 value needs to be configured per channel. For 
testing I have one global value ftpmaxtime in bots.ini. What do you think?

Kind Regards,
Mike

Original comment by mjg1964 on 22 Sep 2011 at 2:39

[GoogleCodeExporter]

I can see that limiting number of files could still be useful. Not so sure 
about usefulness of bytes. But looking at your code it should be easy to have 
limits on all 3 values:
maxfilesperchannel
maxbytesperchannel
maxsecondsperchannel

maxseconds would break out of the loop in the same way as maxbytes.

All 3 would have a global setting in bots.ini and could later have per-channel 
values in the database.

Kind Regards,
Mike

Original comment by mjg1964 on 22 Sep 2011 at 5:55

[GoogleCodeExporter]

yes, you have a very good point in mentioning the maxtime.
appearently is is easy to make limit, but what limit is best.

communications:
indeed it is the time you want to control.

translations:
this might be different.
volume seems to be a big thing here.
but this should be dealt with in the translation (maxbytes limit for 
translations)

I think it is best to use only maxseconds per channel,
as this is what the purpose of limiting (for channels) is.....
how do you think about this?

Original comment by hjebb...@gmail.com on 22 Sep 2011 at 12:02

[GoogleCodeExporter]

I think the only place that limits make sense is for communications channels. 
Once a file is "in bots" then it must be processed.

For me personally, a time limit would be the most useful. Maybe other people 
might prefer files or bytes? So I think it's ok to allow all 3. For adding to 
the channel table, perhaps one "limit" field and one "limit type" dropdown list 
field (files/bytes/seconds instead of 3 different limits)?

But if you want to keep it simple, then just the time limit...

If it becomes possible to do parallel runs of bots engine then probably this 
would be less needed anyway.

Original comment by mjg1964 on 22 Sep 2011 at 12:28

[GoogleCodeExporter]

yes.
but using a scheduler, would it be possible to have the engine run again when 
the run is finished?

yes, think is is better to keep it simple for now.

Original comment by hjebb...@gmail.com on 22 Sep 2011 at 12:46

[GoogleCodeExporter]

My ideas for the scheduler are based on my AS/400 system background. As well as 
the scheduler there are job queues. When a job is scheduled to run it goes onto 
a queue, and then runs in turn from that queue immediately when the previous 
job finishes. So with bots as it is now, there is only a single queue because 
bots engine cannot run parallel, But if it could, you can also have multiple 
queues (eg. fast, medium and slow queues) The schedule would specify a queue to 
run in as well as when to run.

Original comment by mjg1964 on 22 Sep 2011 at 10:35

[GoogleCodeExporter]

incoming channels are now limited by time.
deleted the limiting by volume and number of files.

Original comment by hjebb...@gmail.com on 19 Oct 2011 at 12:31

• Changed title: limit the time an in-channel is using

[GoogleCodeExporter]

Tested this, working well.

Original comment by mjg1964 on 21 Oct 2011 at 1:01

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Original comment by hjebb...@gmail.com on 10 Sep 2013 at 12:44

• Changed state: Done