search

yvaeltercero / codesearch

Automatically exported from code.google.com/p/codesearch
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

cindex needs vast amounts of temporary space #33

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
1. Run cindex on about 9 GiB of files (where about half of them are text files)
2. Observe space usage of cindex

What is the expected output?

cindex takes a reasonable amount of temporary space and provides an option to 
specify an alternative place to store temporary files.

What do you see instead?

cindex eats up to 5 GiB of space on /tmp while generating a 600 MiB 
.csearchindex file.

What version of the product are you using? On what operating system?

tip on Linux with go tip

Please provide any additional information below.

It would be nice if cindex could provide an option to change the temporary 
directory as /tmp might not be large enough. Additionally, it might be possible 
to lower the space usage of cindex, for instance, it might be possible to merge 
(and therefore compress) temporary indices before processing the next set of 
files.

Original issue reported on code.google.com by fuz...@gmail.com on 23 Sep 2013 at 1:23

GoogleCodeExporter commented 9 years ago 
You can set the $TMPDIR environment variable before running cindex to store the 
temporary files somewhere else.

Original comment by dgryski on 24 Sep 2013 at 7:21