microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.59k stars 296 forks source link

Print values while restler is running #720

Closed alaendro closed 1 year ago

alaendro commented 1 year ago

Description

While Restler is running I wanted to check some values through the print function in python but it wasn't possibile, how could I do it? I tryied writing in a txt file but it doesn't work out.

marina-p commented 1 year ago

Hello @alaendro,

Would it be possible for you to share your code snippet (python sample code with the value you attempted to print)? Just in case it's helpful, see logger.py line 91 (write function) for an example of how RESTler prints to its existing network logs.

Thanks,

Marina

alaendro commented 1 year ago

sure, this code it's taken from the render_prefix() submethod in line 379 from restler/engine/core/sequences.py `

         for i in range(len(self.requests) - 1):

            last_tested_request_idx = i
            prev_request = self.requests[i]
            prev_rendered_data, prev_parser, tracked_parameters, updated_writer_variables =\
                prev_request.render_current(candidate_values_pool,
                preprocessing=preprocessing, use_last_cached_rendering=True)
            request.update_tracked_parameters(tracked_parameters)
            # substitute reference placeholders with resolved values
            if not Settings().ignore_dependencies:
                prev_rendered_data =\
                    self.resolve_dependencies(prev_rendered_data)

            prev_req_async_wait = Settings().get_max_async_resource_creation_time(prev_request.request_id)
            prev_producer_timing_delay = Settings().get_producer_timing_delay(prev_request.request_id)

            prev_response = request_utilities.send_request_data(prev_rendered_data)
            print("prev_response json " + prev_response.json_body() + " body " + prev_response.body()) #print value returned
#and i also tried with this
                f = open("/com.docker.devenvironments.code/restler/engine/core/debugger.txt", "a")
                f.write("prev_response da sequences.py riga 396, json body " + prev_response.json_body() + " body " + prev_response.body())
                f.close()

`

marina-p commented 1 year ago

Hello @alaendro,

For your scenario, I recommend writing to the existing network logs with some prefix that you can then use to easily search for to filter out your output.

Also, there was a python error in the print statement above which caused the engine to crash - the call stack is printed in the EngineStdOut.txt file in the task directory.

Here is a line that worked for me:

                RAW_LOGGING("DEBUG: prev_response json " + prev_response.json_body + " body " + prev_response.body) #print value returned

output on demo_server:

2023-03-29 18:28:36.992: prev_response json {"id":22,"body":"my first blog post"} body {"id":22,"body":"my first blog post"}

Thanks,

Marina

alaendro commented 1 year ago

I followed your advice but I still not get the output on the EngineStdOut.txt file, there's some option I need to use before start a fuzzing or a test? This are the commands that I'm using

restler_bin/restler/Restler compile --api_spec test_8_stdout/swagger.json

restler_bin/restler/Restler test --grammar_file Compile/grammar.py --dictionary_file Compile/dict.json --settings Compile/engine_settings.json --no_ssl

restler_bin/restler/Restler fuzz-lean --grammar_file Compile/grammar.py --dictionary_file Compile/dict.json --settings Compile/engine_settings.json --no_ssl

restler_bin/restler/Restler fuzz --grammar_file Compile/grammar.py --dictionary_file Compile/dict.json --settings Compile/engine_settings.json --no_ssl --time_budget 0.03

If can be useful, I have imported the function from here from utils.logger import NetworkLog, raw_network_logging as RAW_LOGGING and I also tried with the _RAW_LOGGING imported from here from engine.core.request_utilities import _RAW_LOGGING but nothing seems to work correctly

marina-p commented 1 year ago

Hello @alaendro,

The output from RAW_LOGGING will be present in the network.testing.*.txt file. It will be interleaved with the other output from RESTler using this function, but if you include a prefix (such as 'DEBUG'), you can easily filter these and output to a different file. The advantage of including this in the network log is that you can see when your output was printed relative to the rest of the execution.

Thanks,

Marina