all files moved by zobov if handle is empty

[broukema]

SUMMARY: An empty value of handle in parameters/params.py causes zobov to shift all files from the Revolver/ main directory into a subsubdirectory and then crash.

ENVIRONMENT:

• python2.7
• Debian GNU/Linux
• Revolver commit 792322e Tue May 21 12:13:49 2019 +0100

REPRODUCE:

To reproduce the bug: compile Revolver, provide an ascii simulation file simfile (an N=32^3, XYZ T^3 simulation file of unit fundamental domain size), create a directory 'outputs/', and modify parameters.py as per the following patch, in order to analyse an example with zobov only:

diff --git a/parameters/params.py b/parameters/params.py
index 88fc4fd..b9ef372 100644
--- a/parameters/params.py
+++ b/parameters/params.py
@@ -3,14 +3,14 @@
 # distance units in this code are calculated in Mpc/h by default

 # ======= runtime options ======== #
-verbose = False  # True for more informative output statements
+verbose = True  # True for more informative output statements
 debug = False    # True for output checks during reconstruction
 nthreads = 1    # set to the number of CPUs you have
 # ================================ #

 # ========= file handling options ========= #
 handle = ''         # string; used to identify the sample and set filenames
-output_folder = ''  # /path/to/folder/ where output should be placed
+output_folder = 'outputs/'  # /path/to/folder/ where output should be placed
 # ========================================= #

 # ========== cosmology ============ #
@@ -18,7 +18,7 @@ omega_m = 0.308  # used for reconstruction and to convert redshifts to distances
 # ================================= #

 # ======= reconstruction options ========== #
-do_recon = True     # if False, no reconstruction is performed and other recon options are ignored
+do_recon = False     # if False, no reconstruction is performed and other recon options are ignored
 nbins = 256     # the number of grid cells per side of the box
 padding = 200.  # for survey data, the extra 'padding' for the cubic box, in Mpc/h
 smooth = 10.    # smoothing scale in Mpc/h
@@ -28,10 +28,10 @@ niter = 3       # number of iterations in the FFT reconstruction method, 3 is su
 # ========================================= #

 # ======= input tracer data options =========== #
-tracer_file = ''    # /path/to/file with input data
-is_box = False      # True if tracers cover a cubic simulation box with periodic boundaries; False for survey data
-box_length = 1500.  # if is_box, the box side length in Mpc/h; else ignored
-boss_like = True    # True if the input data file is in FITS format with same data fields as BOSS data
+tracer_file = 'simfile'    # /path/to/file with input data
+is_box = True      # True if tracers cover a cubic simulation box with periodic boundaries; False for survey data
+box_length = 1.  # if is_box, the box side length in Mpc/h; else ignored
+boss_like = False    # True if the input data file is in FITS format with same data fields as BOSS data
 special_patchy = False  # set True if input array is in the special PATCHY format provided by Hector
 # if not boss_like or special_patchy, data file must contain array data in ASCII or NPY format
 posn_cols = [0, 1, 2]  # columns of tracer input array containing 3D position information
@@ -65,7 +65,7 @@ randoms_file = ''   # /path/to/file containing randoms data: must be formatted s
 # =========================== #

 # ========== void-finding choices ============= #
-run_voxelvoids = True  # watershed void-finding based on particle-mesh density field interpolation in voxels
+run_voxelvoids = False  # watershed void-finding based on particle-mesh density field interpolation in voxels
 run_zobov = True   # watershed void-finding (using ZOBOV) based on Voronoi tessellation
 # note that these two options are not mutually exclusive - 2 sets of voids can be produced if desired

EXPECTED OUTPUT:

Using the command

python revolver.py --par=parameters/params.py

you should get

Loading parameters from parameters/params.py

 ==== Starting the void-finding with ZOBOV ====
Loading tracer positions from file simfile
32768 tracers found
MPI run: calling voz1b1 and voztie to do the tessellation...
Tessellation done.
mv: cannot move './outputs' to a subdirectory of itself, 'outputs/rawZOBOV/./outputs'
Post-processing voids ...
Traceback (most recent call last):
  File "revolver.py", line 202, in <module>
  File "/scratch/Revolver/python_tools/zobov.py", line 859, in postprocess_voids
  File "/scratch/.local/lib/python2.7/site-packages/numpy/lib/npyio.py", line 962, in loadtxt
    fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
  File "/scratch/.local/lib/python2.7/site-packages/numpy/lib/_datasource.py", line 266, in open
    return ds.open(path, mode, encoding=encoding, newline=newline)
  File "/scratch/.local/lib/python2.7/site-packages/numpy/lib/_datasource.py", line 624, in open
    raise IOError("%s not found." % path)
IOError: outputs/rawZOBOV/.txt not found.

ANALYSIS:

The error can be understood by checking what's in the working directory, in outputs/, and in outputs/rawZOBOV:

boud:/scratch/Revolver$ ls -l
total 4
drwxr-xr-x 4 boud boud 4096 Aug  8 17:59 outputs/
boud:/scratch/Revolver$ ls -l outputs/
total 788
drwxr-xr-x 2 boud boud   4096 Aug  8 17:58 log/
-rw-r--r-- 1 boud boud 786436 Aug  8 17:58 _pos.dat
drwxr-xr-x 9 boud boud   4096 Aug  8 17:59 rawZOBOV/
-rw-r--r-- 1 boud boud    129 Aug  8 17:58 sample_info.txt
boud:/scratch/Revolver$ ls -l outputs/rawZOBOV/
total 108
drwxr-xr-x 2 boud boud   4096 Aug  8 17:50 bin/
drwxr-xr-x 3 boud boud   4096 Aug  8 17:50 build/
-rwxr-xr-x 1 boud boud     27 Aug  8 17:50 __init__.py*
-rw-r--r-- 1 boud boud  35149 Aug  8 17:49 LICENSE
-rwxr-xr-x 1 boud boud    456 Aug  8 17:50 Makefile*
drwxr-xr-x 2 boud boud   4096 Aug  8 17:50 masks/
drwxr-xr-x 2 boud boud   4096 Aug  8 17:58 parameters/
drwxr-xr-x 2 boud boud   4096 Aug  8 17:51 python_tools/
drwxr-xr-x 7 boud boud   4096 Aug  8 17:50 qhull/
-rw-r--r-- 1 boud boud   3599 Aug  8 17:50 README.md
-rwxr-xr-x 1 boud boud  10894 Aug  8 17:50 revolver.py*
lrwxrwxrwx 1 boud boud 884590 Aug  8 17:53 simfile
drwxr-xr-x 2 boud boud   4096 Aug  8 17:50 src/
-rw-r--r-- 1 boud boud   4969 Aug  8 17:55 wisdom.256.1

The problem seems to be related to handle being empty.

SUGGESTED FIX:

One solution is to put a default handle if handle is left as an empty string; another could be to warn the user and give a fatal error.

[seshnadathur]

I'll apply a version of the fix you suggest along with some other changes.