ARCLab-MIT / kspdg

ARCLab-MIT participation in the KSPDG challenge
MIT License
4 stars 0 forks source link

Is 100 the timeout for the agents, no more than that? #3

Closed vrodriguezf closed 11 months ago

vrodriguezf commented 11 months ago

I found this code in the example_agent_runner:

if __name__ == "__main__":
    naive_agent = NaivePursuitAgent()    
    runner = AgentEnvRunner(
        agent=naive_agent, 
        env_cls=PE1_E1_I3_Env, 
        env_kwargs=None,
        runner_timeout=100,     # agent runner that will timeout after 100 seconds
        debug=True)
    runner.run()

Does that mean that the timeout for the agents is going to be 100 seconds?

OhhTuRnz commented 11 months ago

I run an agent with a timer checking the timeout. It indeed terminated reached 100 seconds starting at the moment KRPC connects with your code.

<img src="https://i.gyazo.com/9eebef127e3ef0c41d2e3697669f50a7.jpg" alt="Markdown Monster icon" style="float: left; margin-right: 5px;" />

I also checked runner's default timeout is 600 seconds:


import logging
import multiprocessing as mp

from typing import Dict

from kspdg.base_envs import KSPDGBaseEnv
from kspdg.agent_api.base_agent import KSPDGBaseAgent
from kspdg.utils.loggers import create_logger
from kspdg.agent_api.ksp_interface import ksp_interface_loop

DEFAULT_RUNNER_TIMEOUT = 600 #[s]
vrodriguezf commented 11 months ago

I also checked runner's default timeout is 600 seconds:

Well spotted! I'll ask tomorrow anyway, I'd like to know the timeout that the evaluation script uses in the runners it creates (maybe they don't want to share it)

vrodriguezf commented 11 months ago

Ross Allen said that the timeout is the one found in each environment. So 600 in case of the pe