microsoft / WinAppDriver

Windows Application Driver
MIT License
3.67k stars 1.4k forks source link

WinAppDriver causes memory leaks in tested application #991

Open Konop1992 opened 4 years ago

Konop1992 commented 4 years ago

Hi, I did some investigation and I'm almost certain that WinAppDriver is causing memory leaks in tested application. What I don't know is how to stop that or what is responsible for that (Windows or WinAppDriver).

I attach some simple test for Notepad that proves the problem. All it does is opens About window and closes it. Sln is in NotepadTest folder. Set NotepadMemoryLeakTests as StartupProject, build everything, run WinAppDriver.exe. You can run "ManualMode" or "UsingWinAppDriver" by commenting one of the methods.

After 4minutes - around 2Mb is added After 1h - 32mb of memory usage by empty notepad. While sending keys to application ("Manual Mode") or clicking manually there is no memory increase. MemLeakNotepad.zip

Why is that important? Because we want to use WinAppDriver to test our application for memory leaks. It is quite hard if the testing tool is causing them in the same time. Our memory leak tests are repeating the same script over and over for long period of time (e.g. 24h).

What we have tried already:

Related issues: https://github.com/microsoft/WinAppDriver/issues/425, https://github.com/microsoft/WinAppDriver/issues/547 On the first related issue mvw684 posted a picture from memory profiler and it looks like a lot of AutomationPeer objects are not released. And that's the only conclusion I can have. WinAppDriver is using AutomationPeers that are present on tested application. Those Automation Peers then keep the window in memory (because GC cannot collect them).

So is it a Windows problem or a WinAppDriver problem? Is there a way to "close" the session without closing the application (given that closing the session would actually release references to AutomationPeers, as I said before we already tried closing WinAppDriver and it didn't help)? BR

mills-andrew commented 4 years ago

What you can do is dispose of the process, then relink the process. Killing the WinAppDriver.exe will not touch your program, rather it will clear any memory it is using. the problem is, i assume you are using appium. Appium (or selenium for that matter) has 2 major functions. void close() and void quit(). Close closes the app but keeps the driver instance. (In your case you don't want this) Quit terminates the driver instance.

What I suggest is to find its process and kill it manually. Once you do this, you can re build your driver and link it to an exisiting procress. The below code will attempt to kill the process through its API but if it can't it will kill it regardless.

For example, your Process Stop function could look something like this. HostName is normally 127.0.0.1 and Port is normally 4723

    protected Process mProcess;
    public bool IsRunning => (this.mProcess != null && !this.mProcess.HasExited);
    public Uri ToUri() => new Uri(string.Format(CultureInfo.InvariantCulture, "http://{0}:{1}", this.HostName, this.Port));

    /// <summary>
    /// Will Attempt to stop the process.
    /// </summary>
    public virtual void Stop()
    {
        if(this.IsRunning)
        {
            WaitUtils.Wait(() => Terminated() == true, this.mTerminationTimeout);
            this.mProcess.WaitForExit(3000);
        }

        //Implement catch all feature just in case.
        if(this.IsRunning)
            this.mProcess.Kill();

        this.mProcess.Dispose();
        this.mProcess = null;
    }

    /// <summary>
    /// Attempts to terminate the process through the drivers API
    /// </summary>
    /// <returns></returns>
    protected bool Terminated()
    {
        HttpWebRequest request = HttpWebRequest.Create(new Uri(this.ToUri(), "/shutdown")) as HttpWebRequest;
        request.KeepAlive = false;

        using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
        {
            return response.StatusCode == HttpStatusCode.OK && response.ContentType.StartsWith("application/json", StringComparison.OrdinalIgnoreCase);
        }
    }
mills-andrew commented 4 years ago

There are some custom functions in the example, but I believe you get the idea.

mvw684 commented 4 years ago

This looks indeed very much like the issue we are experiencing as is also listed in #425.

We see that these AutomationPeers are referenced by refcounted GCHandles (via scitech memory profiler)

Note that we see this on recent windows 10, though not on older windows 10. See issue #425 Exact same issues seen when using CodedUI and Appium.

Konop1992 commented 4 years ago

Hello, @KitoCoding - I managed to replace some of the custom functions and it compiles. However there is no "/shutdown" method. I didn't find it in any API documentation and I didn't manage to invoke it. All I got was 404 as a response.

What I did managed to run was deleteing session by DELETE "http://127.0.0.1:4723/session/sessionId" then I closed WinAppDriver then opened it again and created a new session. But it didn't help. Memory is still increasing.

Changed solution with @KitoCoding proposal: NotepadTest.zip

Konop1992 commented 4 years ago

I also tried with FlaUI (https://github.com/Roemer/FlaUI) and it doesn't have this problem. For 100 iterations notepad is using steadily around 2372kb and for WinAppDriver it's increasing. After 100 iterations it is about 4260kb. I know for sure that TestComplete also doesn't have this problem.

Just to be clear - I had to change cliking on "About" menu item to sending "A" key because I don't know how to do that in FlaUI and I don't want to waste time. I did the same for WinAppDriver for comparison (results I provided are already using this). Here is the solution with changes:

NotepadTestWithFlaUI.zip