pelahi / VELOCIraptor-STF

Galaxy/(sub)Halo finder for N-body simulations
MIT License
20 stars 26 forks source link

Error, net size 0 with row,col=0,0 #4

Closed DCunnama closed 7 years ago

DCunnama commented 7 years ago

Hi Pascal,

Still having the same issue with the following calculation:

Now get average in grid cell and find outliers

I've included the makefile, config and output:

Thanks, Dan

velociraptor.tar.gz

pelahi commented 7 years ago

weirdly this seems like either the local velocity density calculation is failing or the center of mass and the mean halo velocity is off. Maybe try Search_for_substructure=0 FoF_Field_search_type=5 just to see if the properties it calculates for the halo make sense. The next thing to try is Search_for_substructure=1 Output_den=1 with the STRUCDEN compile flag off along with HALODEN then check the file your_output_filename.localden to see if these are positive floats.

On 23 January 2017 at 19:32, DCunnama notifications@github.com wrote:

Hi Pascal,

Still having the same issue with the following calculation:

Now get average in grid cell and find outliers

I've included the makefile, config and output:

Thanks, Dan

velociraptor.tar.gz https://github.com/pelahi/VELOCIraptor-STF/files/723416/velociraptor.tar.gz

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4, or mute the thread https://github.com/notifications/unsubscribe-auth/ALeQS8aK3TNt0ubliGAmqoPRyTFiW7HIks5rVI-_gaJpZM4Lq795 .

DCunnama commented 7 years ago

Ok, so it doesn't look like the properties make sense, getting negative masses.

snap_m25n256.snap_134.VELOCIraptor.properties.tar.gz

pelahi commented 7 years ago

I wonder if this is an issue with the Endian of the dataset. Does the input look fine in hdfview (or h5dump)? I'm a bit careless when checking the endian of input data.

On 23 January 2017 at 20:47, DCunnama notifications@github.com wrote:

Ok, so it doesn't look like the properties make sense, getting negative masses.

snap_m25n256.snap_134.VELOCIraptor.properties.tar.gz https://github.com/pelahi/VELOCIraptor-STF/files/723580/snap_m25n256.snap_134.VELOCIraptor.properties.tar.gz

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4#issuecomment-274479706, or mute the thread https://github.com/notifications/unsubscribe-auth/ALeQS8JL5U7qMkkaf7QhIGuhjDUEfvRQks5rVKFogaJpZM4Lq795 .

DCunnama commented 7 years ago

Hdf5 by construction should remove any endian issues from within the library.

But as it is, the fact that its reading everything else fine, the velocities and positions, and even the masses look reasonable, with the exception of the negative.

The massed are stored in a Mass block not in the mass table, but that shouldn't be an issue should it?

pelahi commented 7 years ago

It shouldn't be an issue. Could you add after line 483 in hdfio.cxx (which as LN=) cout<<hdf_header_info[ifirstfile].mass[1]<<" "<<mscale<endl; and compile. See if, for reasons that still aren't clear to me, one of these is negative?

On 23 January 2017 at 21:02, DCunnama notifications@github.com wrote:

Hdf5 by construction should remove any endian issues from within the library.

But as it is, the fact that its reading everything else fine, the velocities and positions, and even the masses look reasonable, with the exception of the negative.

The massed are stored in a Mass block not in the mass table, but that shouldn't be an issue should it?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4#issuecomment-274482528, or mute the thread https://github.com/notifications/unsubscribe-auth/ALeQS31_Wn56la2v4Z-dcVO52m88HNQ7ks5rVKT3gaJpZM4Lq795 .

DCunnama commented 7 years ago

0 1.470588235

pelahi commented 7 years ago

so the mass header for DM is zero? Is that correct? what about at line 763 adding in cout<<ifloat<<" "<<floatbuff[0]<<" <<doublebuff[0]<<endl; so you can see what it s reading in the mass data set.

On 23 January 2017 at 21:15, DCunnama notifications@github.com wrote:

0 1.470588235

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4#issuecomment-274485033, or mute the thread https://github.com/notifications/unsubscribe-auth/ALeQSzitWl-dten1sx7vg7-5Ywa068I3ks5rVKf5gaJpZM4Lq795 .

DCunnama commented 7 years ago

Yeah, the mass header is 0, that's correct.

Seems that floatbuff[0] i the issue:

1 -35102.88281 0.68

On Mon, Jan 23, 2017 at 3:26 PM, Pascal Jahan Elahi < notifications@github.com> wrote:

so the mass header for DM is zero? Is that correct? what about at line 763 adding in cout<<ifloat<<" "<<floatbuff[0]<<" <<doublebuff[0]<<endl; so you can see what it s reading in the mass data set.

On 23 January 2017 at 21:15, DCunnama notifications@github.com wrote:

0 1.470588235

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4# issuecomment-274485033, or mute the thread https://github.com/notifications/unsubscribe- auth/ALeQSzitWl-dten1sx7vg7-5Ywa068I3ks5rVKf5gaJpZM4Lq795

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4#issuecomment-274487219, or mute the thread https://github.com/notifications/unsubscribe-auth/AMagN87BeUrJe185EHyRqB1R6B5nPuCzks5rVKqGgaJpZM4Lq795 .

pelahi commented 7 years ago

Which value seems correct? I would expect 0.00776 (1e10 Msun/h). Neither is what I would expect for a 25 Mpc/h 256^3 simulation. Are the units solar masses? or 1e10 solar masses? The code thinks the data space contains float32 but does h5dump think the data type is a float64? Or float 32?

Oh and I noticed that you have units of G that would be 1e10 Msun, kpc, and km/s but you have Hubble in km/s/Mpc (should be 0.1 for km/s/kpc).

On 23 January 2017 at 22:50, DCunnama notifications@github.com wrote:

Yeah, the mass header is 0, that's correct.

Seems that floatbuff[0] i the issue:

1 -35102.88281 0.68

On Mon, Jan 23, 2017 at 3:26 PM, Pascal Jahan Elahi < notifications@github.com> wrote:

so the mass header for DM is zero? Is that correct? what about at line 763 adding in cout<<ifloat<<" "<<floatbuff[0]<<" <<doublebuff[0]<<endl; so you can see what it s reading in the mass data set.

On 23 January 2017 at 21:15, DCunnama notifications@github.com wrote:

0 1.470588235

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4# issuecomment-274485033, or mute the thread https://github.com/notifications/unsubscribe- auth/ALeQSzitWl-dten1sx7vg7-5Ywa068I3ks5rVKf5gaJpZM4Lq795

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4# issuecomment-274487219, or mute the thread https://github.com/notifications/unsubscribe-auth/ AMagN87BeUrJe185EHyRqB1R6B5nPuCzks5rVKqGgaJpZM4Lq795 .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4#issuecomment-274507292, or mute the thread https://github.com/notifications/unsubscribe-auth/ALeQS2kpcP2Z5Lmq3AyQ_Savcr45Obj9ks5rVL4ugaJpZM4Lq795 .

DCunnama commented 7 years ago

None of them seem correct, the Mass of the DM particles is 0.0065132.

I see the units were incorrect, I've corrected them now. We're in kpc and 1e10 Msol:

Units: L=0.001, M=1e+10, V=1, G=43021.1

Anyway, I'm still getting the same value read in:

1 -35102.88281 0.68

The datatype is H5T_IEEE_F32LE

pelahi commented 7 years ago

so then it should try to read a float as the code is doing. Is the hdf file somewhere I could get access to so I can see why it is loading in this negative value?

On 24 January 2017 at 15:34, DCunnama notifications@github.com wrote:

None of them seem correct, the Mass of the DM particles is 0.0065132.

I see the units were incorrect, I've corrected them now. We're in kpc and 1e10 Msol:

Units: L=0.001, M=1e+10, V=1, G=43021.1

Anyway, I'm still getting the same value read in:

1 -35102.88281 0.68

The datatype is H5T_IEEE_F32LE

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4#issuecomment-274730142, or mute the thread https://github.com/notifications/unsubscribe-auth/ALeQS5B-MlddQR_f6vpdiqPVdrilcMTKks5rVamAgaJpZM4Lq795 .

DCunnama commented 7 years ago

Thanks Pascal,

I stuck the file on castor: /home/nifty2014/ClustersOfGalaxies/PLAYGROUND/dcunnama/data/

Sorry about this.

pelahi commented 7 years ago

I know what is the issue, DM particles in illustris format don't have a masses data set names[itemp++]=H5std_string("Coordinates"); names[itemp++]=H5std_string("Velocities"); names[itemp++]=H5std_string("ParticleIDs"); if (hdfnametype==HDFILLUSTISNAMES) { names[itemp++]=H5std_string("Potential");

So you should add names[itemp++]=H5std_string("Masses"); after line 186. Fix everything

On 24 January 2017 at 18:23, DCunnama notifications@github.com wrote:

Thanks Pascal,

I stuck the file on castor: /home/nifty2014/ClustersOfGalaxies/PLAYGROUND/ dcunnama/data/

Sorry about this.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pelahi/VELOCIraptor-STF/issues/4#issuecomment-274764519, or mute the thread https://github.com/notifications/unsubscribe-auth/ALeQS-uEz7SQpSoEcFXeNcgf5Z4srLTFks5rVdEzgaJpZM4Lq795 .

DCunnama commented 7 years ago

Ha, bingo.

Sorry I didn't see that sooner.

Thanks, seems to be reading properly now.