search

microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.59k stars 296 forks source link

ERROR: Results analyzer for logs in Fuzz fails #709

Closed juliabennett-msft closed 1 year ago

juliabennett-msft commented 1 year ago

Description

Fuzz-lean worked for me, but Fuzz fails with the following error.

Steps to reproduce

image

Expected results

No response

Actual results

No response

Environment details

No response

marina-p commented 1 year ago

Hello @juliabennett-msft,

Can you repeatedly run fuzz-lean successfully? One possibility is that there is some corruption/leftover state from fuzz-lean in your environment that is causing fuzz to fail.

It would be helpful if you could share the beginning of the network log (network.testing.*.txt) from both runs for further investigation.

Thanks,

Marina

juliabennett-msft commented 1 year ago

I retried fuzz-lean and it fails. I then reran the build-restler.py, compile, test, and fuzz-lean. Fuzz-lean failed but the rest worked fine.

juliabennett-msft commented 1 year ago

Where can I find the network.testing files?

marina-p commented 1 year ago

They are in RestlerResults\experiment*\logs

juliabennett-msft commented 1 year ago

network.testing.35504.1_FuzzLean.txt network.testing.3468.1_fuzz.txt

juliabennett-msft commented 1 year ago

Re: One possibility is that there is some corruption/leftover state from fuzz-lean in your environment that is causing fuzz to fail.

Is there a way for me to do a clean somehow?

marina-p commented 1 year ago

Hi Julia,

This issue is not with fuzzing, it is with parsing the network logs after fuzzing has completed. I am going to close this issue since it is already being tracked by #465.

On a separate note, in these runs your RESTler setup is missing custom payloads for the {subscriptionId} and {resourceGroupName} parameters (and possibly others). A coverage of 0/X often means that your API is not being reached due to some setup issue. You need to investigate the results (in speccov.json, the network logs, or the new coverage_failures_to_investigate.txt report) and improve the coverage before running fuzz-lean and fuzz modes.

Thanks,

Marina

juliabennett-msft commented 1 year ago

[like] Julia Bennett reacted to your message:

From: marina-p @.> Sent: Friday, June 2, 2023 9:28:05 PM To: microsoft/restler-fuzzer @.> Cc: Julia Bennett @.>; Mention @.> Subject: Re: [microsoft/restler-fuzzer] ERROR: Results analyzer for logs in Fuzz fails (Issue #709)

Hi Julia,

This issue is not with fuzzing, it is with parsing the network logs after fuzzing has completed. I am going to close this issue since it is already being tracked by #465https://github.com/microsoft/restler-fuzzer/issues/465.

On a separate note, in these runs your RESTler setup is missing custom payloads for the {subscriptionId} and {resourceGroupName} parameters (and possibly others). A coverage of 0/X often means that your API is not being reached due to some setup issue. You need to investigate the results (in speccov.json, the network logs, or the new coverage_failures_to_investigate.txt report) and improve the coverage before running fuzz-lean and fuzz modes.

Thanks,

Marina

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/restler-fuzzer/issues/709#issuecomment-1574335230, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A6K2QLABMRWILBK24GUKA2TXJJLGLANCNFSM6AAAAAAVUIOUOQ. You are receiving this because you were mentioned.Message ID: @.***>