The time taken by restler is longer than expected

[alaendro]

Description

I am using restler myself on an API I have at my disposal and one thing I have noticed is that the --time_budget parameter does not work as it should in both the original and the personally modified version. I've tested the original version on multiple devices and although I put --time_budget 0.01, the time taken is over an hour instead of 1 minute as it was a few versions ago and this problem has persisted for a few versions. I have currently encountered this problem on both restler 9.1 and 9.2 and I have noticed that this behaviour only occurs on Windows and Ubuntu while on the Macbook the time taken is as it is requested.

Steps to reproduce

Following the readme, I compile the demo_server swagger with the relevant command and execute restler_bin/restler/Restler fuzz --grammar_file Compile/grammar.py --dictionary_file Compile/dict.json --settings Compile/engine_settings.json --no_ssl --time_budget 0.01

Expected results

I expect the execution to finish 1 minute after the command has been started or shortly thereafter

Actual results

The process stops after 1 hour and some minutes

Environment details

Windows Python 3.9.13 and dotnet 7.0.202 Ubuntu Python 3.10.6 and dotnet 6.0.116 Mac Python 3.9.16 and dotnet 6.0.407

[marina-p]

Hello @alaendro,

I cannot reproduce this with the current version of RESTler. Could you please check the output in the top-level log named restler-20230602-120121.log (current date and time will be substituted in the file name) and send me the command line with the time_budget parameter as it appears in this log?

Restler Information: 0 : Launching 'python.exe -B "...../engine/restler.py" <...> --disable_checkers namespacerule --time_budget 0.001000 <...> --fuzzing_mode bfs-fast'

Thanks,

Marina

[bungdanar]

Hello, I have the same problem. I run the restler for 3 minutes but it stops after 1 hour and some extra minutes.

This is my command: .\restler\Restler.exe fuzz --grammar_file .\Compile\grammar.py --dictionary_file .\Compile\dict.json --settings .\Compile\engine_settings.json --no_ssl --time_budget 0.05

And this is the launching information log: Restler Information: 0 : Launching 'python.exe -B "D:\Projects\pribadi\rest-fuzzing\restler_bin\restler../engine/restler.py" --restler_grammar "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\grammar.py" --custom_mutations "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\dict.json" --set_version 9.2.2 --no_ssl --settings "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\engine_settings.json" --enable_checkers * --disable_checkers namespacerule --time_budget 5.000000 --include_user_agent --no_tokens_in_logs t --garbage_collection_interval 30 --fuzzing_mode bfs-fast'

Environment details Windows, dotnet 7.0.110, python 3.11.1

[bungdanar]

Hello, I have the same problem. I run the restler for 3 minutes but it stops after 1 hour and some extra minutes.

This is my command: .\restler\Restler.exe fuzz --grammar_file .\Compile\grammar.py --dictionary_file .\Compile\dict.json --settings .\Compile\engine_settings.json --no_ssl --time_budget 0.05

And this is the launching information log: Restler Information: 0 : Launching 'python.exe -B "D:\Projects\pribadi\rest-fuzzing\restler_bin\restler../engine/restler.py" --restler_grammar "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\grammar.py" --custom_mutations "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\dict.json" --set_version 9.2.2 --no_ssl --settings "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\engine_settings.json" --enable_checkers * --disable_checkers namespacerule --time_budget 5.000000 --include_user_agent --no_tokens_in_logs t --garbage_collection_interval 30 --fuzzing_mode bfs-fast'

Environment details Windows, dotnet 7.0.110, python 3.11.1

I think this is a bug (in Windows) when parsing time budget argument that contains decimal point. When you pass decimal value using dot, the prefix is ignored. For example in my case, passing 0.05 will be rendered as 5.0000. I try to pass decimal value using comma and it is rendered correctly as 0.05

This is my new command: .\restler\Restler.exe fuzz --grammar_file .\Compile\grammar.py --dictionary_file .\Compile\dict.json --settings .\Compile\engine_settings.json --no_ssl --time_budget 0,05

And this is launch information log: Restler Information: 0 : Launching 'python.exe -B "D:\Projects\pribadi\rest-fuzzing\restler_bin\restler../engine/restler.py" --restler_grammar "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\grammar.py" --custom_mutations "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\dict.json" --set_version 9.2.2 --no_ssl --settings "D:\Projects\pribadi\rest-fuzzing\restler_bin\Compile\engine_settings.json" --enable_checkers * --disable_checkers namespacerule --time_budget 0.050000 --include_user_agent --no_tokens_in_logs t --garbage_collection_interval 30 --fuzzing_mode bfs-fast'

And after that, the fuzzing process stops correctly in 3 minutes (0.05 hour)