WebDriver is not deleting the profile directory after test exits

[lukeis]

Originally reported on Google Code with ID 1934

What steps will reproduce the problem?
1. On running RemoteWebDriver with an instantiated FirefoxProfile it creates 2 directories:
anonymous* and webdriver*(some random number) in “C:\Documents and Settings\<UserName>\Local
Settings\Temp”. The directory webdriver* (of ~2200K) is not removed after WebDriver
finishes its job.

What is the expected output? What do you see instead?
Either it should not create the directory or it should delete it after finishing it's
job

What version of the product are you using? On what operating system?
Windows XP
Selenium 2.0rc3

Please provide any additional information below.
I am not whether it will create a problem in WebDriver however it may slow the System
performance and will unnecessary eats up the system disk space. 

Also it seems related to Issue 1211 (http://code.google.com/p/selenium/issues/detail?id=1211).

Reported by contact@vaidabogdan.com on 2011-06-27 11:02:42

[lukeis]

Which language binding? The temp file creation and deletion is handled differently by
the different language bindings, so knowing which language you're using is important
in this case.

Reported by james.h.evans.jr on 2011-06-27 11:33:39

• Status changed: NeedsClarification

[lukeis]

I use Java for the development.

Reported by contact@vaidabogdan.com on 2011-06-27 11:44:39

[lukeis]

My experience is the same while setup combo is win7/webdriver 2.0rc3/java. Can it affect
performance of the regression run?

Reported by wantashish on 2011-06-29 10:44:51

[lukeis]

A patch is attached that fixes this issue.

Reported by barancev on 2011-06-30 22:40:12

---

• Attachment: FirefoxProfile.patch

[lukeis]

Reported by francois.reynaud on 2011-07-01 07:31:50

[lukeis]

I'm not entirely sure how this fixes the issue: the "dir" being deleted is already marked
as a temporary filesystem and should be deleted when the jvm exits. Deleting the temporary
directory before the FirefoxProfile is actually discarded will cause some stability
problems too.

Am I missing something very obvious?

Reported by simon.m.stewart on 2011-07-03 12:17:55

[lukeis]

i think the problem is for long running server in grid for instance.

Reported by francois.reynaud on 2011-07-03 12:32:34

[lukeis]

1. Yes, Francois is right, for a long running server (especially a virtual machine with
a small disk) it is a problem. Each profile can be several megabytes in size if some
FF plugins are used (e.g. firebug is about 2MB), several hundreds of tests causes "low
disk space" situation.

2. Actually, dir is a temporary directory that contains a profile extracted from json
string, but FirefoxProfile(dir) constructor copies content of this directory (webdriver*)
to a new profile  directory (anonymous*), and immediately after copying dir can be
safely removed. So, I think there is no need to wait until exit.

Reported by barancev on 2011-07-03 14:14:09

[lukeis]

Any Updates on this issue, as i am doing evaluation of the framework to adopt it in
my team. each time i run a script it creates 19MB file in temp directory which can
be disastrous for system running multiple test cases.

Reported by Talwindersingh on 2011-08-04 11:22:59

[lukeis]

I have encountered this issue as well. My long-running Selenium server, running on a
small VM with 8 GB of diskapce, is running out of diskspace due to temporary files
under /tmp.

I noticed that these files are cleaned up on JVM exit (probably because of the tmp
file status). However, for long-running server, this represents a leak, since the JVM
would not be restarted often.

Also interesting to note that I don't seem to have the same problem on Windows, only
on Linux do I see thousands of leaked profile directories (although my Linux usage
is a lot heavier, so this may not be a valid observation).

Reported by peterkwanchan on 2011-08-31 21:16:13

[lukeis]

This issue was closed by revision r13935.

Reported by barancev on 2011-09-23 14:37:03

• Status changed: Fixed

[lukeis]

Reopened since r13935 was reverted in r13956 due to test break

Reported by kristian.rosenvold on 2011-09-26 18:12:51

• Status changed: New

[lukeis]

Well, may be we have to remove the profile directory when driver.quit() is called? Will
it work for Grid? I'm having out-of-disk-space on my virtual machines almost every
day :(

Reported by barancev on 2011-09-27 13:09:43

[lukeis]

Quit sounds good to me, especially if the tests dont break ;)

Reported by kristian.rosenvold@zenior.no on 2011-09-27 16:17:27

[lukeis]

In the release note for Selenium 2.8, it is claimed that this issue of profile directory
filling up disk is fixed.

As far as I can tell, it is not fixed.

At the end of my test runs on the 2.8 Selenium server, even after all test sessions
are properly closed, a bunch of directories remain in /tmp:

drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895846/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895847/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895848/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895849/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895850/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895851/
drwxr-xr-x  6 root root  4096 Oct  9 02:00 anonymous1107044062167549014webdriver-profile/
drwxr-xr-x  6 root root  4096 Oct  9 02:00 anonymous2288986081707850643webdriver-profile/
drwxr-xr-x  6 root root  4096 Oct  9 02:00 anonymous3928139591340740449webdriver-profile/
drwxr-xr-x  6 root root  4096 Oct  9 01:58 anonymous425239302836435538webdriver-profile/
drwxr-xr-x  6 root root  4096 Oct  9 01:56 anonymous5847048534096526317webdriver-profile/

The anonymous* directories are deleted only upon JVM termination for Selenium server.

Furthermore, in the 2.8 release, I now see a new kind of leaks - the integer based
directories are not cleaned up at all.

This is after the server/JVM has terminated:

drwxr-xr-x  2 root root  4096 Oct  9 01:57 1318110895845/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895846/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895847/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895848/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895849/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895850/
drwxr-xr-x  2 root root  4096 Oct  9 01:59 1318110895851/

These integer directories are empty inside.

Reported by peterkwanchan on 2011-10-08 22:05:05

[lukeis]

Fixed (again) in r14142

Reported by kristian.rosenvold on 2011-10-12 19:32:30

• Status changed: Fixed

[lukeis]

I don't think this issue was fixed. I'm using Grid/WD version 2.11.0 and the Grid machine
dir "C:\Users\qauser\AppData\Local\Temp" is filled with the following folders "webdriver113516513935198274duplicated"
which contains my custom FF profile. Each folder is about 12MB. After a week or two
my Grid server suffered from "low disk space" and i realized the temp folder reached
19GB!!!
Now, i guess i can write a small script which deletes those folder every once in a
while but i think this should be fixed in WD.

BTW, i'm using Java if this info is important.

Reported by jongler on 2011-11-17 20:49:37

[lukeis]

@jongler, I think you are right, there are actually multiple issues hiding inside this
thread, and the /original/ issue is not fixed. I'm reopening, this is for the specific
leak associated with the method FirefoxProfile#fromJson (serialized profiles on remote
node)

Reported by kristian.rosenvold on 2011-11-18 06:33:36

• Status changed: Accepted

[lukeis]

Is there any estimation on when this bug is going to be fixed?

Reported by jongler on 2011-12-01 11:33:29

[lukeis]

Deleting files/directories upon JVM exit is unacceptable for some environments (Like
mine). I run routine firefox tests, and I load in Firebug -- Each anonymous profile
is ~22MB. And this is a service I run, not a one off application.

I've resorted to patching the FirefoxProfile to track it's profile directory, and FirefoxWebDriver's
close method to delete the dir from the FirefoxProfile (if it's anonymous).

Why isn't this the natural case?

Reported by phocis on 2011-12-14 00:06:26

[lukeis]

I just downloaded the latest C# WebDriver 2.17 and I am having this issue. I'm not happy
Bob!! The cog on the wheel is broken.

Reported by blueguitarenator on 2012-01-18 18:19:17

[lukeis]

I was having the same problem with WebDriver 2.14 and 2.18 being used inside of some
JUnit tests.

My fix was to add the following inside my tearDownAfterClass method.

@AfterClass
    public static void tearDownAfterClass() throws Exception {
        TemporaryFilesystem tempFS = TemporaryFilesystem.getDefaultTmpFS();
        tempFS.deleteTemporaryFiles();
        driver.quit();
    }

Hope this helps.

Reported by clayton.lengel-zigich@integrumtech.com on 2012-01-27 16:30:19

[lukeis]

Hi clayton,

This is a nice solution but it won't work when running tests in remote, using grid.
Am i right?

Reported by jongler on 2012-02-02 19:05:21

[lukeis]

Hello Kristian,

I think the subject of this issue should be "*Remote*WebDriver does not delete firefox
profile directory", and I think I have found the root cause of this issue (which still
exists on trunk):

org.openqa.selenium.remote.server.DefaultSession creates a temporary directory named
after the session id (and properly deletes it when the session is closed).
So I guess the assumption is that code in the session should use TemporaryFilesystem.getTmpFsBasedOn(SESSION_TMP_DIR).

But src.org.openqa.selenium.firefox.FirefoxProfile.fromJson() is not aware of this
and instead uses TemporaryFilesystem.getDefaultTmpFS() to place its profile copy, and
this never gets deleted.
However, I'm not sure that FirefoxProfile *can* be aware of the Session, since it may
or may not be running under a RemoteWebDriver.

I can't really propose how to fix this issue; maybe the FirefoxDriver needs to explicitly
remove its Profile when quitting?

Reported by wolfgang.schnerring on 2012-04-26 08:45:59

[lukeis]

Hm, this is still a pain ass bug after so many months!?!

Reported by alonecomp on 2012-04-26 21:32:08

[lukeis]

Issue 2642 has been merged into this issue.

Reported by barancev on 2012-05-05 08:38:54

[lukeis]

Issue 3155 has been merged into this issue.

Reported by barancev on 2012-05-05 14:12:01

[lukeis]

This issue was closed by revision r16883.

Reported by barancev on 2012-05-12 07:53:01

• Status changed: Fixed

[lukeis]

This bug is reproduced again in the latest release (2.23.1). Status should be updated:
"Fixed" -> "Reopened".

Reported by max.abukhovsky on 2012-06-12 04:31:03

[lukeis]

Reproduced under Win7 / FirefoxDriver (Firefox 12).

Reported by max.abukhovsky on 2012-06-12 04:32:19

[lukeis]

Is it possible to change the tmp directory to another directory? I tried to override
the "java.io.tmpdir" and it still writes to /tmp 

Reported by pauldavidchristian on 2012-06-13 21:44:42

[lukeis]

reproduced with selenium v2.23.1 on windows 7 / firefox 10.0.3

occasionally I get exceptions:

java.lang.NullPointerException
    at org.openqa.selenium.io.FileHandler.delete(FileHandler.java:189)
    at org.openqa.selenium.io.FileHandler.delete(FileHandler.java:190)
    at org.openqa.selenium.io.FileHandler.delete(FileHandler.java:190)
    at org.openqa.selenium.io.TemporaryFilesystem.deleteTempDir(TemporaryFilesystem.java:115)
    at org.openqa.selenium.firefox.FirefoxProfile.clean(FirefoxProfile.java:403)
    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.quit(NewProfileExtensionConnection.java:164)
    at org.openqa.selenium.firefox.FirefoxDriver$LazyCommandExecutor.quit(FirefoxDriver.java:299)
    at org.openqa.selenium.firefox.FirefoxDriver.stopClient(FirefoxDriver.java:257)
    at org.openqa.selenium.remote.RemoteWebDriver.quit(RemoteWebDriver.java:333)

or

Exception in thread "Thread-1" java.lang.NullPointerException
    at org.openqa.selenium.io.FileHandler.delete(FileHandler.java:189)
    at org.openqa.selenium.io.FileHandler.delete(FileHandler.java:190)
    at org.openqa.selenium.io.FileHandler.delete(FileHandler.java:190)
    at org.openqa.selenium.io.TemporaryFilesystem.deleteTemporaryFiles(TemporaryFilesystem.java:129)
    at org.openqa.selenium.io.TemporaryFilesystem$1.run(TemporaryFilesystem.java:36)

Reported by tom.goemaes on 2012-06-19 13:11:47

[lukeis]

Also seems to happen with ChromeDriver.  It leaves behind a bunch of scoped_dir* directories
within the C:\Documents And Settings\<user>\Local Settings\Temp\ directory.

Selenium 2.23.0 on Windows XP, Chrome/Chromedriver 19

Should I create a separate issue for this, or leave it here?

Reported by winter.kyle on 2012-06-20 03:34:36

[lukeis]

Selenium 2.23.1 on Win XP, Chrome 19.0184.56m/Chromedriver 20.0.1133.0
Also a lot of scoped_dir* directories in Temp.

Reported by lagin@avast.com on 2012-06-25 08:58:57

[lukeis]

any updates on the "scoped_dir* directories not cleaned up" issue with ChromeDriver?
It fills up disk space in our CI environment and is a problem we really want to tackle.

Reported by alex.ikhelis on 2012-08-01 13:21:07

[lukeis]

@alex.ikhelis: Please create a new issue in the chromedriver issue tracker: http://code.google.com/p/chromedriver/issues/list

Reported by barancev on 2012-08-01 15:31:43

[lukeis]

hmm... looks a bit strange that developers of testing tools seemingly don't write regression
tests for their bugfixes.

Reported by MarkStier on 2012-08-11 03:51:29

[lukeis]

Is there regression?

Reported by barancev on 2012-08-13 09:08:38

[lukeis]

Please reopen. This is not fixed. I just deleted tons of these folders each of them
about with a size of approximately 30 MBytes.

I'm using WebDriver 2.25.1 (.NET version) with FireFox 16.0.2. Operating system is
Windows 7 (64bit), latest patches installed.

Reported by manfredmlange on 2012-11-02 08:06:14

[lukeis]

********************
You can get this problem by failing to issue a .quit() at the end of your session.
 If you just end with a .close() it won't do a cleanup.  We were seeing this exact
same problem (FireFox on Win7 browser cache/profile being dumped to a appdata/local/temp
folder for every test run), but by adding the .quit() to the end the problem went away
and now we get clean runs.
********************

Reported by tjulius%dyn.com@gtempaccount.com on 2012-11-19 16:10:36

[lukeis]

@tjul...@dyn.com Tried what you suggested and your workaround appears to work. Thank
you!

Reported by manfredmlange on 2012-11-19 17:34:27

[lukeis]

We are seeing this issue on Win7 using selenium-server-standalone-2.26.0. We are also
calling Quit() on the driver before exiting, yet our node disks are filling up with
the following types of folders/files in our \Users\<username>\AppData\Local\Temp folder:

(1) Folders with names like "1343943823037"
I assume these are session id folders. These folders are empty, but they exist even
after shutting down selenium node.

(2) Folders with names like "anonymous1972864079409089672webdriver-profile"
These folders are about 8 MB each.

(3) Folders with names like "scoped_dir14656_4658"
These folders are mostly empty. Sometimes there is a 0 byte chrome_debug.log file.

(4) Folders with names like "webdriver-ie9198441489016361573"
These folders all seem to be empty.

Even after shutting down the selenium node, these files persist. They are consuming
a most of our drive and we have to keep deleting them. This is an issue that needs
to be resolved. Please re-open this issue.

Reported by chrisedwards357 on 2012-12-11 16:19:33

[lukeis]

I am seeing the scope_dirxxxx left around on Win7 running 2.28 (local WebDriver server,
not remote node).  When Chrome browser is started, it created two scope_dirxxx (with
different sequence number): one empty and the other containing two subfolders "Default"
and "Temp" and 5 text files (chrome_debug.log, EULA file, etc.).  

When Chrome is stopped (.close() and .quit()) called, the empty scope_dirxxx was delete
as expected, but the other folder scope_dirxxx was left behind (about 8mb).

This is repeatable.

I am running the latest chromedriver.exe.

Reported by lin.yaxiong on 2013-01-02 02:45:35

[lukeis]

@45 You should report this in the ChromeDriver tracker: 

  https://code.google.com/p/chromedriver/issues/list

Reported by jari.bakken on 2013-01-03 16:13:36

[lukeis]

I have been using a workaround that upon testcleanup we would clean out all those files.
However, now that we started using Grid, I think we are accidentally cleaning out the
folders that are also still in use. To continue to use this solution more effectively
(until the root cause is fixed), is there a way to find the folder that was created
per test? This way with that reference I would be able to delete the folder that would
no longer be used.

Reported by eric.petrowiak on 2013-02-21 18:19:50

[lukeis]

I have the same problem.

When Chrome browser is started, it create two folders scope_dirxxx (with different
sequence number): one empty and the other containing two subfolders "Default" and "Temp"
and 5 text files (chrome_debug.log, EULA file, etc.).  

When Chrome is stopped (.close() and .quit()) called, the empty scope_dirxxx was delete
as expected, but the other folder scope_dirxxx was left behind (about 8mb).

This is repeatable in interval of 15 minutes. After a few hours, i dont have no more
disk space.

I am running the latest chromedriver.exe.

Selenium version: Selenium-Java 2.35
OS: XP
ChromeDriver (v2.2)
Browser: Chrome
Browser version: 29.0.1547.57

Reported by luizalberto.cardoso on 2013-08-23 16:55:29

[lukeis]

I got a strange problem. I was running ChromeDriver on a test and found the hard disk
full the next day. After cleaning out the System Temp files using Ccleaner, I re-booted
the machine. However, something is still generating scoped_dirxxx folders and writing
to them. 

Reported by paul.delamusica on 2013-09-05 15:11:04

---

• Attachment: processes.JPG

[lukeis]

We are seeing the same issue sas well.  Is there anyone working on this?

Reported by rolandasburbulis on 2013-09-10 16:19:04

[lukeis]

No-one here is working on the chromedriver issue, it is not part of this project. If
chromedriver leaves files, it has to be fixed there, as per coment #46

Reported by kristian.rosenvold@zenior.no on 2013-09-10 16:38:28

[lukeis]

Hello, I'm working with FF and windows Server 2008 64 bits. This problem has not been
fixed, so I can't use Selenium WebDriver any more. My Hard drive is full and also My
RAM, cause selenium doesn't kill Firefox.

Regards

Reported by alexandre.safa on 2013-10-18 06:21:28