Minimum working example

[alex-koe]

Is there a minimum working example of the particle reduction? I tried to get the reduction_mail.py run with the following bash line:

python reduction_main.py -hdf simData_filtered_190000.h5 -hdf_re out.h5 -ration 0.003

There is no output from the program i could post. The output file out.h5 seems to be not written. I guess i did not correctly call the program. So, how do i have to run the program? :-)

[KseniaBastrakova]

You can read help, by calling python3 reduction_main.py -h

So, for your example, you can run reduction like this: python3 reduction_main.py simData_filtered_190000.h5 -hdf_re out.h5 -algorithm random -ratio_deleted_particles 0.1 (with random thinnig algorithm).

[alex-koe]

That is helping a lot. I have just another question: Where can i select the particle species, e.g., en, en_all, e, ...? Currently, the example keeps failing for my case.

[KseniaBastrakova]

Currently, the example keeps failing for my case.

could you send me error message or link to the file? it will help me a lot

[alex-koe]

As discussed offline with @PrometheusPi , I tested now with an original file direct from PIC simulation.

python reduction_main.py  -hdf simData_orig_60000.h5 -hdf_re out.h5 -ratio_deleted_particles 0.5 -algorithm random

I got the following output:

Series constructor called with explicit iteration suggests loading a single file with groupBased iteration encoding. Loaded file is fileBased.
Series.set_software_version is deprecated. Set the version with the second argument of Series.set_software
Traceback (most recent call last):
  File "reduction_main.py", line 790, in <module>
    base_reduction_function(args.hdf, args.hdf_re, "random", parameters)
  File "reduction_main.py", line 207, in base_reduction_function
    process_iteration_group(algorithm, current_iteration, series_hdf, series_hdf_reduction, reduction_iteration)
  File "reduction_main.py", line 226, in process_iteration_group
    process_patches_in_group_v2(iteration.particles[name_group], series_hdf,
  File "reduction_main.py", line 660, in process_patches_in_group_v2
    data, dict_data_indexes, last_idx = get_data(series_hdf, particle_species, weights, idx_start, idx_end)
  File "reduction_main.py", line 615, in get_data
    if len(absolute_values[0]) == 1:
IndexError: index 0 is out of bounds for axis 0 with size 0
Speicherzugriffsfehler

Speicherzugriffsfehler means seg fault. The file out.h5 is written but too small to be complete. I ran the script with Python 3.8.5 on taurus.

[PrometheusPi]

@KseniaBastrakova Under what conditions could absolute_values be not an array?

[KseniaBastrakova]

@KseniaBastrakova Under what conditions could absolute_values be not an array?

it only values from input openPMD file, multiply on unit_SI or (for position) with adding positionOffset

Can you send me a link to the fie? I will reproduce and find the bug

[PrometheusPi]

@KseniaBastrakova for overing to test the file. :+1: The file is on the taurus cluster at ZIH/TU Dresden - @alex-koe Could you send me the path to the file - then I will transfer the data (via GridFTP).

[alex-koe]

@PrometheusPi this is the original file: /scratch/ws/1/s5960712-LPWFA_2020/runs_LWFA/010_LWFA_doping=0.005_Laguerre_k80/simOutput/h5/simData_beta_60000.h5

[PrometheusPi]

@alex-koe Thank you - surprisingly the file was not as big as expected - thus GridFTP is not really needed - I just copy it with scp.

[PrometheusPi]

The file can be fund on hemera in : /bigdata/hplsim/scratch/rp6038/testParticleReduction/simData_beta_60000.h5.

[KseniaBastrakova]

Thank you for example. Now, reduction works normally.

But, I noticed that there aren't any fields in simData_beta_60000.h5 file. I ask it because it necessary for conversion: I count rmacro for gdf file based on mesh grid size.

Also, i reproduced floating Segmentation fault. I think, problem can be in openPMD-api, not in file or in my code. (now, i"m working on localize)

[KseniaBastrakova]

I added additional parameter ""-grid_size"" to openPMD to gdf converter: For gdf file we store "rmacro" parameter -- radius of macroparticle. If openPMD file has Fields, we count rmacro by multiplying particle shape on cell size, but if the file doesn't have fields, you should define "-grid_size"" -- size of grid cell in SI.

[PrometheusPi]

@alex-koe Could you try the new option?

[alex-koe]

@PrometheusPi i will see how much time i have during beam time.

[alex-koe]

@KseniaBastrakova @PrometheusPi somehow, i cannot make it work. 1st: I run

$> python reduction_main.py -hdf_re reduced_60000.h5 -hdf simData_orig_60000.h5 -grid_size 0.00008  -ratio_deleted_particles 0.003 -algorithm random` and got ```usage: reduction_main.py [-h] [-algorithm algorithm] [-hdf hdf_file]
                         [-hdf_re hdf_file_reduction]
                         [-ratio_deleted_particles ratio_deleted_particles]
                         [-momentum_tol tolerance_momentum]
                         [-position_lol tolerance_position]
                         [-leveling_coefficient leveling_coefficient]
reduction_main.py: error: unrecognized arguments: -grid_size 0.00008

and i did not found in any file the word grid_size by grep grid_size *.py.

2nd: I run

$> python reduction_main.py -hdf_re reduced_60000.h5 -hdf simData_orig_60000.h5  -ratio_deleted_particles 0.003 -algorithm random

Series constructor called with explicit iteration suggests loading a single file with groupBased iteration encoding. Loaded file is fileBased.

And there was no file generated. :-(

[KseniaBastrakova]

@KseniaBastrakova @PrometheusPi somehow, i cannot make it work. 1st: I run

$> python reduction_main.py -hdf_re reduced_60000.h5 -hdf simData_orig_60000.h5 -grid_size 0.00008  -ratio_deleted_particles 0.003 -algorithm random` and got ```usage: reduction_main.py [-h] [-algorithm algorithm] [-hdf hdf_file]
                         [-hdf_re hdf_file_reduction]
                         [-ratio_deleted_particles ratio_deleted_particles]
                         [-momentum_tol tolerance_momentum]
                         [-position_lol tolerance_position]
                         [-leveling_coefficient leveling_coefficient]
reduction_main.py: error: unrecognized arguments: -grid_size 0.00008

Oh, sorry for the misunderstanding! I just reproduced full user case (reduction + conversion) and noticed, that is necessary parameter for my GDFconverter ( https://github.com/ComputationalRadiationPhysics/openPMD-converter-GDF ), so i added this parameter to converter, not to particle reduction.

[KseniaBastrakova]

2nd: I run

$> python reduction_main.py -hdf_re reduced_60000.h5 -hdf simData_orig_60000.h5  -ratio_deleted_particles 0.003 -algorithm random

Series constructor called with explicit iteration suggests loading a single file with groupBased iteration encoding. Loaded file is fileBased.

And there was no file generated. :-(

I think, I understood your problem: you file in openPMD format is file-based. It means, that openPMD-api expected group of files, with one iteration in each, in the format simData_orig_%T.h5, where T - the number of iteration. And my redutor output has to be in the same format, as input, in the format reduced_%T.h5

So, first, do: cd particle_reduction git fetch git pull (i added "iteration parameter" for reduction specify iteration)

And next, try to modify python reduction_main.py -hdf_re reduced_60000.h5 -hdf simData_orig_60000.h5 -ratio_deleted_particles 0.003 -algorithm random to python reduction_main.py -hdf_re reduced_%T.h5 -hdf simData_orig_%T.h5 -ratio_deleted_particles 0.003 -algorithm random -iteration 60000 (if you want to reduce only iteration 60000) or to python reduction_main.py -hdf_re reduced_%T.h5 -hdf simData_orig_%T.h5 -ratio_deleted_particles 0.003 -algorithm random (if you want to reduce all iterations in current directory)

[alex-koe]

2nd: I run

$> python reduction_main.py -hdf_re reduced_60000.h5 -hdf simData_orig_60000.h5  -ratio_deleted_particles 0.003 -algorithm random

Series constructor called with explicit iteration suggests loading a single file with groupBased iteration encoding. Loaded file is fileBased.

And there was no file generated. :-(

I think, I understood your problem: you file in openPMD format is file-based. It means, that openPMD-api expected group of files, with one iteration in each, in the format simData_orig_%T.h5, where T - the number of iteration. And my redutor output has to be in the same format, as input, in the format reduced_%T.h5

So, first, do: cd particle_reduction git fetch git pull (i added "iteration parameter" for reduction specify iteration)

And next, try to modify python reduction_main.py -hdf_re reduced_60000.h5 -hdf simData_orig_60000.h5 -ratio_deleted_particles 0.003 -algorithm random to python reduction_main.py -hdf_re reduced_%T.h5 -hdf simData_orig_%T.h5 -ratio_deleted_particles 0.003 -algorithm random -iteration 60000 (if you want to reduce only iteration 60000) or to python reduction_main.py -hdf_re reduced_%T.h5 -hdf simData_orig_%T.h5 -ratio_deleted_particles 0.003 -algorithm random (if you want to reduce all iterations in current directory)

Thanks for your detailed description. I pulled the new version of your code. It worked well and the file was written. How can i open the file "reduced_%T.h5"? I did two things: 1) I have tried h5py reduced_60000.h5 but it fails and tell me "unable to open file". 2) I have tried python and h5py and it also fails with: "Unable to open file (bad object header version number)". The h5 file a real h5 file?

[KseniaBastrakova]

1. h5py

Which h5py version do you have? I tried:

 f = h5py.File("reduced_60000.h5", 'r')
 print(f['data'].keys())

and get: <KeysViewHDF5 ['60000']>

could you try to open result file with openPMD-api? like this:

series_hdf = Series("reduced_60000.h5", Access_Type.read_only)
for iteration in series_hdf.iterations:
print(iteration)

[alex-koe]

Thanks a lot @KseniaBastrakova. It is now working! The mistake was an issue. For testing, i just ran it on taurus (without sbatch) and then it quited silently. The reason was, i guess, that i piped the output to a file for further reference:

$> python reduction_main.py -hdf_re reduced_60000.h5 -hdf simData_orig_60000.h5  -ratio_deleted_particles 0.003 -algorithm random 2&>1 | tee output.txt

However, it ended without any output at all in 'output.txt', because all messages were not written into the file but only buffered. This means that even no indication of exceeding taurus runtime was left. By using job submission, it now works. Again, thanks a lot!

[alex-koe]

To be complete, this is the submit script on taurus i used:

#!/bin/bash

#SBATCH --time=02:00:00
#SBATCH --job-name=part-red
#SBATCH --nodes=1
#SBATCH  --mem-per-cpu=4000
#SBATCH --ntasks=2

# send me mails on BEGIN, END, FAIL, REQUEUE, ALL,
# TIME_LIMIT, TIME_LIMIT_90, TIME_LIMIT_80 and/or TIME_LIMIT_50
#SBATCH --mail-type=ALL
#SBATCH --mail-user=XXX@hzdr.de

export PYTHONPATH="$HOME/anaconda3/"
export PATH="$PYTHONPATH/bin:$PATH"

python reduction_main.py -hdf simData_orig_%T.h5 -hdf_re simData_for_GPT_%T.h5 -iteration 60000  -ratio_deleted_particles 0.9970685060813841 -algorithm random

It completed after < 30min. XXX has to be replaced for a working address in case someone copies the code.