Application hangs when using custom net fuzzer

[Jello]

I'm trying to fuzz an application that starts up, spawns a lot of threads, and both connects to a server and acts as a server. Using drrun with the debug mode, it hits my target function if I manually send data to it that hits the correct function. However, when using winafl with the custom net fuzzer plugin, the application seems to hang. Winafl will continually call the net fuzzer plugin to send data, but the application itself hangs and never fully finishes fully initializing to bring up all its network connections. Checking the application logs, I can see it only seems to output a few of the lines related to its initialization, but otherwise isn't continuing.

I have no idea how to even debug this to see what could be the cause of the hang, and it the application runs correctly under drrun and drcov.

[ifratric]

adding @mxmssh (author of the custom net fuzzer)

[mxmssh]

Hi Jello, thanks for the interest in winAFL and this feature. Regarding the problem, First of all, we should identify whether there is some problem with winAFL, application or both. It seems like this is not a problem with DynamoRIO (since the target is running fine under drrun) which is good.

I would try playing with the delay before actually start sending data in the socket. It can be controlled using -w parameter (see Network fuzzing section in README for more details). I would just give it 30 seconds to make sure the target finish the initialization. If it doesn't work we can continue discussing and dive deeper :)

[Jello]

I've tried setting the delay as high as several minutes. The application is getting hung during it's initialization when run with the network fuzzing, it never seems to finish bringing all its connections up and the log files don't show any output to show that it's running.

Eventually, the network fuzzing will begin, but the application seems to fail in spawning off its multiple worker threads to handle other connections. Only the first thread that makes a network connection seems to execute. It's almost like the application fails to spawn additional threads when running with the custom net fuzzer.

On Mon, Sep 30, 2019, 4:34 PM Maksim Shudrak notifications@github.com wrote:

Hi Jello, thanks for the interest in winAFL and this feature. Regarding the problem, First of all, we should identify whether there is some problem with winAFL, application or both. It seems like this is not a problem with DynamoRIO (since the target is running fine under drrun) which is good.

I would try playing with the delay before actually start sending data in the socket. It can be controlled using -w parameter (see Network fuzzing section in README for more details). I would just give it 30 seconds to make sure the target finish the initialization. If it doesn't work we can continue discussing and dive deeper :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/googleprojectzero/winafl/issues/208?email_source=notifications&email_token=AAAOLMTX6OQ52MKDH7S7LOLQMJPD5A5CNFSM4I2GPC62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7676BI#issuecomment-536739589, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLMQLCECVNKLAYBJUOBLQMJPD5ANCNFSM4I2GPC6Q .

[mxmssh]

I see, this is very weird. Are you trying client or server-mode fuzzing ?

[mxmssh]

What about your target function, are you sure that it is correctly set? Can you try to set the number of -fuzz_iterations in 1?

[mxmssh]

Ah, and I almost forget about protocol. Is it TCP or UDP?

[Jello]

The application is a client that starts up multiple threads to connect to a server. I've written a python proxy application to sit in the middle and pass traffic between the client and server, while also listening on a new port for traffic to inject into the communication stream.

I'm instrumenting the client application under winafl, using the custom net fuzzer to connect to my Python proxy to send fuzzed data to inject. With drrun, this workflow works, I can man in the middle all client/server traffic, and any packets I send to the listening port are injected into the traffic. I correctly receive multiple client connections and stateful messages like keep alives are passed between client and server, while I can inject custom packets to hit the code paths I'm interested in. Under winafl with the custom net fuzzer, the client application never seems to finish initializing as I only get a single connection. The fuzzed data will reach my Python server and be sent to the client, but the client never responds with anything and will doesn't hit the desired function for persistence mode fuzzing.

This is using tcp as the transport.

On Mon, Sep 30, 2019, 4:43 PM Maksim Shudrak notifications@github.com wrote:

I see, this is very weird. Are you trying client or server-mode fuzzing ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/googleprojectzero/winafl/issues/208?email_source=notifications&email_token=AAAOLMXRBPNFNCQRB3MKO6TQMJQF3A5CNFSM4I2GPC62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD77A4CI#issuecomment-536743433, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLMWTP72I3RRPUOQDKETQMJQF3ANCNFSM4I2GPC6Q .

[mxmssh]

Ok, It seems like I start understanding what's going on. Correct me if I am wrong. You want winAFL to act as a man-in-the-middle mutate/inject packets on the fly, right?

[Jello]

Close, I want winafl to mutate selected packet captures I provide as its input, but the man in the middle / injection functionality is handled by a Python program listening on a TCP socket for data. I can run the application as normal, have it connect to my Python mitm server, and I can send my packet captures to the Python application's listening socket and they will be successfully injected into the data stream and exercise the code I'm interested in. The packets to fuzz are static packet captures, not on the fly data. The mitm connection is necessary, because the application receives configuration information from the server, plus keep alive packets that will cause the application to exit if it doesn't receive them. With drrun, this still works, and the output shows that my target function was reached and returned from.

When running the client application under winafl with the custom net fuzzer, it will start providing fuzzed data to my Python application via the TCP connection, which will then be injected into the connection and sent to the client application. However, it is not hitting the function I selected for persistence fuzzing, and the application never seems to fully initialize. Only the first of several connections is made when running under Winafl, and the application seems to have hung. I don't see output in the application's logs corresponding to the other threads the client application normally starts.

[mxmssh]

Ok, I got it. I guess the problem is that WinAFL actually doesn't support this scenario. The network-mode fuzzing functionality is limited for the following two scenarios:

1. winAFL acts as a server accepting incoming connections on some TCP port. The client performs connection, winAFL generates data and send it over the network using the established connection.
2. winAFL acts as a client. In this case, winAFL just connect on certain port opened by the server and send data in the socket.

That's basically all. In your case, the scheme is not trivial. While I believe there is some way to make it work by slightly modifying winAFL, I don't believe it is possible out-of-the-box.

[mxmssh]

If you can replace the server (the final one that send response) with winAFL it might work.

[Jello]

It's not clear to me that would work, it seems like when running under Winafl, the client application is unable to start all its threads. I am receiving fuzzed data from the custom net fuzzer dll, but even if I cut out the Python application and replaced it with a custom fuzzer DLL that performs that mitm functionality, or otherwise replicates the server functionality, the application would still be failing to start its child threads when running under winafl.

[mxmssh]

Well, it is hard to say without understanding the actual logic of the program and debugging. What I can say for sure that if it is working under drrun/drcov, it should work under winAFL because winAFL is just a fuzzer. However, winAFL expects certain steps from the program to be done before actually start sending test cases.