search

uyar / calico

GNU General Public License v3.0
8 stars 4 forks source link

Accelerating Evaluation Procedure #14

Open canerozer opened 3 years ago

canerozer commented 3 years ago

Hello,

I noticed slowness in the evaluation procedure due to a misconfiguration related to the library pexpect. Adding the following line after the 85th line of the base.py file would accelerate the entire evaluation procedure since there will be no delay between each subsequent action.

process.delaybeforesend = None

According to pexpect, the purpose of adding the delay of 50 ms between expect and send actions is to prevent the unintended echo behavior of send. This would be an important point of consideration for a particular scenario where a test case involves a sequential series of expect and send actions and can still be used to prevent that behavior. However, an additional argument for reducing or revoking that delay would create an opportunity for processing lengthier test cases. This is not possible at the moment for test cases that involve 100 thousand per each.

Can we also provide an optional argument to redefine the delaybeforesend through the YAML file?

Best regards,

Caner Ozer

ghost commented 3 years ago

Nice catch and explanation, thank you very much. There are also other things to consider relating pexpect, especially about platform-specific behavior. I'll need to check these but I'm afraid to poke around and break a functional system.