Open rbmetcalf opened 7 years ago
Hi Ben,
I have been working on this and I have two questions. Let me explain first. I decided to do it in the following way.
In lightcone_construction.h I changed
Konsole output size_tNlines= 0,Nblocks=0; while(!feof(pFile)){ // loop through blocks
//long Nbatch = unit.scan_block(blocksize,pFile); longNbatch= unit.scan_blockH5(blocksize,snap_filenames[i_file]);
so that Nbatch is changed to the new version I included to read hdf5. I also added to the struct ASCII_XMRT12 a new way of reading:
Konsole output structASCII_XMRRT12:public ASCII_XMRRT{ size_tscan_block(size_tblocksize,FILE*pFile){
// read in a block of points size_ti=0; inttmp;
points.resize(blocksize);
while(i < blocksize && fscanf(pFile,"%lf %lf %lf %lf %lf %lf %i" ,&points[i].x[0],&points[i].x[1],&points[i].x[2] ,&points[i].mass,&points[i].r_max,&points[i].r_scale,&tmp) != EOF){ if(tmp == 1 || tmp == 2) ++i; } points.resize(i);
for(auto&h: points){ h.r_max /= 1.0e3; h.r_scale /= 1.0e3; } returni; }
//CHANGED ***** MARCOS size_tscan_blockH5(size_tblocksize,H5std_stringfilename){ // read in a block of points size_ti=0, j=0; intk=6; hsize_tnrows, ncols=7, offset=0; doubledata; data = common_scan_blockH5(filename, "/dset", blocksize, ncols, offset, &nrows); points.resize(nrows); for(i = 0; i < nrows; i++) { if(((size_t)data[incols+k] == 1) || ((size_t)data[incols+k] == 2)) { points[j].x[0] = data[incols ]; points[j].x[1] = data[incols+1]; points[j].x[2] = data[incols+2]; points[j].mass = data[incols+3]; points[j].r_max = data[incols+4]; points[j].r_scale = data[i*ncols+5]; j++; } } points.resize(j); deletedata;
for(auto&h: points){ h.r_max /= 1.0e3; h.r_scale /= 1.0e3; } returnj; }
Konsole output doublecommon_scan_blockH5(H5std_stringfilename , H5std_stringdataset_name , hsize_tn_rows , hsize_tn_cols , hsize_toffset_rows , hsize_tread_rows );
}; //CHANGED ***** MARCOS
Which calls a function called common_scan_blockH5() that I included at the end of MultiPlane/lightcone_construction.cpp. I am able to compile glamer but, when I try to compile cones.cpp I get some error that it's not finding the function common_scan_blockH5().
My questions are the following:
1) Where should I put the function common_scan_blockH5() so that
lightcone_construction.h sees it. I also need to add some header
(
2) I need to include the following flags
Konsole output -lhdf5_cpp -lhdf5
so that the compiler sees the libraries I need to use hdf5. I have been unable to find the place to include them in the Makefile, where do you put them?
I could pull the code to github but as it is not working yet, I thought it was better not to.
Cheers! Marcos.
PD. I attach the files I changed.
On 07/03/2017 09:08 PM, Ben Metcalf wrote:
@mpellejero https://github.com/mpellejero lets continue our discussion of this here.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/glenco/SLsimLib/issues/131, or mute the thread https://github.com/notifications/unsubscribe-auth/AVsIK2TKyXemjYSONUj9A49uJY4TAkLoks5sKUomgaJpZM4OMpVb.
ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a http://www.iac.es/disclaimer.php WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult http://www.iac.es/disclaimer.php?lang=en
// // lightcone_construction.h // GLAMER // // Created by Ben Metcalf on 26/10/16. // //
using std::cout; using std::endl;
using namespace H5;
/* \brief The LightCones namespace is for classes and functions related to making light cones and weak lensing maps from 3D snapshots of particles or halos. / namespace LightCones{
struct DataRockStar{ unsigned int id; Point_3d x; double mass; double Rvir; // in comoving units double Rscale; // in comoving units double Vmax; };
struct DatumXM{ Point_3d x; double mass; }; struct DatumXMRmRs{ Point_3d x; double mass; double r_max; double r_scale; }; struct DatumXMR{ Point_3d x; double mass; double r; };
void ReadLightConeNFW(std::string filename,COSMOLOGY &cosmo ,std::vector<LensHalo* > &lensVec ,PosType &theta_max);
void ReadLightConeParticles(std::string filename,COSMOLOGY &cosmo ,std::vector<LensHaloParticles *> &lensVec ,int Nplanes ,float particle_mass ,float particle_size ,bool angular_sizes = false ,bool verbose = false);
void ReadLightConeParticles(std::string filename,COSMOLOGY &cosmo ,std::vector<LensHaloMassMap *> &lensVec ,int Nplanes ,float particle_mass ,float angular_resolution ,bool verbose = false);
/** \brief Class for constructing light cones form simulation boxes. / class LightCone{ public: LightCone(double angular_radius);
void WriteLightCone(std::string filename,std::vector<DataRockStar> &vec);
void WriteLightCone(std::string filename,std::vector<Point_3d> &vec);
friend class MultiLightCone;
template <typename T>
void select(Point_3d xo,Point_3d v,double Length,double rlow,double rhigh
,T* begin
,T* end
,Utilities::LockableContainer<std::vector<T> > &incone
,bool periodic_boundaries = true);
private:
double r_theta; // angular radius of light cone (radians)
double sin_theta_sqrt;
};
/// select the halos from the box that are within the light cone
template
if(begin == end) return;
Point_3d dx,x;
double r2,xp;
Point_3d n;
double rhigh2 = rhigh*rhigh;
double rlow2 = rlow*rlow;
T b;
v /= v.length();
// standard coordinates so cone is always centered on the x-axis
Point_3d y_axis,z_axis;
y_axis[0] = -v[1];
y_axis[1] = v[0];
y_axis[2] = 0;
y_axis /= y_axis.length();
z_axis = v.cross(y_axis);
//std::cout << v*y_axis << " " << v*z_axis << " " << y_axis*z_axis << std::endl;
// we can do better than this !!!!
int n0[2] = {(int)((rhigh + xo[0])/Length),(int)((xo[0] - rhigh)/Length) - 1 };
int n1[2] = {(int)((rhigh + xo[1])/Length),(int)((xo[1] - rhigh)/Length) - 1 };
int n2[2] = {(int)((rhigh + xo[2])/Length),(int)((xo[2] - rhigh)/Length) - 1 };
if(!periodic_boundaries){
n0[0] = n0[1] = 0;
n1[0] = n1[1] = 0;
n2[0] = n2[1] = 0;
}
for(auto a = begin ; a != end ; ++a){
dx[0] = (*a).x[0] - xo[0];
dx[1] = (*a).x[1] - xo[1];
dx[2] = (*a).x[2] - xo[2];
int count = 0;
for(n[0] = n0[1]; n[0] <= n0[0] ; ++n[0]){
for(n[1] = n1[1]; n[1] <= n1[0] ; ++n[1]){
for(n[2] = n2[1]; n[2] <= n2[0] ; ++n[2]){
x = dx + n*Length;
xp = x*v;
if(xp > 0){
r2 = x.length_sqr();
if(r2 > rlow2 && r2 < rhigh2){
if( r2*sin_theta_sqrt > r2 - xp*xp){
//std::cout << n << " | " << xo << " | "
// << v << " | " << x << std::endl;
b = *a;
//b.x = x;
// rotate to standard reference frame
b.x[0] = xp;
b.x[1] = y_axis*x;
b.x[2] = z_axis*x;
//std::cout << "b = " << b.x << std::endl;
incone.push_back(b);
++count;
}
}
}
}
}
}
if(count > 1) std::cout << " Warning: Same halo appears " << count << " times in light-cone." << std::endl;
}
}
class MultiLightCone{
public:
MultiLightCone(
double angular_radius
,const std::vector
/** Read the points in from a snapshot of the halos created by RockStar.
The output will be the halos in the cone in coordinates where the x-axis is
the center of the cone. Changing the observer, xo, and direction of view, V,
effectively translates and rotates the box, respectively. Only halos with radii
between rlow and rhigh are added.
*/
void ReadBoxRockStar(std::string filename
,double rlow,double rhigh
,std::vector<std::vector<LightCones::DataRockStar> > &conehalos
,bool periodic_boundaries = true
,bool allow_subhalos = false);
void ReadBoxXYZ(std::string filename
,double rlow,double rhigh
,std::vector<std::vector<Point_3d> > &conehalos
,double hubble
,double BoxLength
,bool periodic_boundaries = true
);
void ReadBoxToMaps(std::string filename
,double rlow,double rhigh
,std::vector<std::vector<PixelMap> > &maps
,double hubble
,double BoxLength
,bool periodic_boundaries = true
);
/* double common_scan_blockH5(H5std_string filename
, H5std_string dataset_name
, hsize_t n_rows
, hsize_t n_cols
, hsize_t offset_rows
, hsize_t *read_rows
); */
private:
std::vector<Point_3d> xos;
std::vector<Point_3d> vs;
std::vector<LightCone> cones;
/* double common_scan_blockH5(H5std_string filename
, H5std_string dataset_name
, hsize_t n_rows
, hsize_t n_cols
, hsize_t offset_rows
, hsize_t *read_rows
); */
};
using Utilities::Geometry::Quaternion; using Utilities::Geometry::SphericalPoint;
/ templated functions for projecting mass onto planes /
/ templated functions for reading a block from a file /
template
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf %lf"
,&points[i][0],&points[i][1],&points[i][2],&tmpf,&tmpf,&tmpf) != EOF)
++i;
points.resize(i);
return i;
}
/ structures to implement templated FastLightCones<>() /
struct ASCII_XV{
std::vector<Point_3d> points;
size_t scan_block(size_t blocksize,FILE *pFile){
double tmpf;
// read in a block of points
size_t i=0;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf %lf"
,&points[i][0],&points[i][1],&points[i][2],&tmpf,&tmpf,&tmpf) != EOF)
++i;
points.resize(i);
return i;
}
static void fastplanes_parallel(
Point_3d *begin
,Point_3d *end
,const COSMOLOGY &cosmo
,std::vector<std::vector<Point_3d> > &boxes
,std::vector<Point_3d> &observers
,std::vector<Quaternion> &rotationQs
,std::vector<double> &dsources
,std::vector<std::vector<PixelMap> > &maps
,double dmin
,double dmax
,double BoxLength
);
}; struct ASCII_XM{
std::vector<DatumXM> points;
size_t scan_block(size_t blocksize,FILE *pFile){
// read in a block of points
size_t i=0;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf"
,&points[i].x[0],&points[i].x[1],&points[i].x[2],&points[i].mass) != EOF)
++i;
points.resize(i);
return i;
}
static void fastplanes_parallel(
DatumXM *begin
,DatumXM *end
,const COSMOLOGY &cosmo
,std::vector<std::vector<Point_3d> > &boxes
,std::vector<Point_3d> &observers
,std::vector<Quaternion> &rotationQs
,std::vector<double> &dsources
,std::vector<std::vector<PixelMap> > &maps
,double dmin
,double dmax
,double BoxLength
);
};
struct ASCII_XMR{
std::vector<DatumXMR> points;
size_t scan_block(size_t blocksize,FILE *pFile){
// read in a block of points
size_t i=0;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf"
,&points[i].x[0],&points[i].x[1],&points[i].x[2]
,&points[i].mass,&points[i].r) != EOF)
++i;
points.resize(i);
return i;
}
static void fastplanes_parallel(
DatumXMR *begin
,DatumXMR *end
,const COSMOLOGY &cosmo
,std::vector<std::vector<Point_3d> > &boxes
,std::vector<Point_3d> &observers
,std::vector<Quaternion> &rotationQs
,std::vector<double> &dsources
,std::vector<std::vector<PixelMap> > &maps
,double dmin
,double dmax
,double BoxLength
);
//CHANGED ********************************************************* MARCOS
/*size_t scan_blockHDF5(size_t blocksize,H5std_string filename){
// read in a block of points
size_t i=0, j=0;
int k=5;
hsize_t nrows, ncols=7, offset=0;
double *data;
data = common_scan_blockH5(filename, "/dset", blocksize, ncols, offset, &nrows);
points.resize(nrows);
for (i = 0; i < nrows; i++) {
if (((size_t)data[i*ncols+k] == 1) || ((size_t)data[i*ncols+k] == 2)) {
points[j].x[0] = data[i*ncols ];
points[j].x[1] = data[i*ncols+1];
points[j].x[2] = data[i*ncols+2];
points[j].mass = data[i*ncols+3];
points[j].r_max = data[i*ncols+4];
//points[j].r_scale = data[i*ncols+5];
j++;
}
}
points.resize(j);
delete data;
for(auto &h: points){
h.r_max /= 1.0e3;
h.r_scale /= 1.0e3;
}
return j;
}*/
};
struct ASCII_XMRRT{
std::vector<DatumXMRmRs> points;
size_t scan_block(size_t blocksize,FILE *pFile){
// read in a block of points
size_t i=0;
int tmp;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf %lf %i"
,&points[i].x[0],&points[i].x[1],&points[i].x[2]
,&points[i].mass,&points[i].r_max,&points[i].r_scale,&tmp) != EOF)
++i;
points.resize(i);
for(auto &h: points){
h.r_max /= 1.0e3;
h.r_scale /= 1.0e3;
}
return i;
}
static void fastplanes_parallel(
DatumXMRmRs *begin
,DatumXMRmRs *end
,const COSMOLOGY &cosmo
,std::vector<std::vector<Point_3d> > &boxes
,std::vector<Point_3d> &observers
,std::vector<Quaternion> &rotationQs
,std::vector<double> &dsources
,std::vector<std::vector<PixelMap> > &maps
,double dmin
,double dmax
,double BoxLength
);
};
struct ASCII_XMRRT12:public ASCII_XMRRT{ size_t scan_block(size_t blocksize,FILE *pFile){
// read in a block of points
size_t i=0;
int tmp;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf %lf %i"
,&points[i].x[0],&points[i].x[1],&points[i].x[2]
,&points[i].mass,&points[i].r_max,&points[i].r_scale,&tmp) != EOF){
if(tmp == 1 || tmp == 2) ++i;
}
points.resize(i);
for(auto &h: points){
h.r_max /= 1.0e3;
h.r_scale /= 1.0e3;
}
return i;
}
//CHANGED ********************************************************* MARCOS
size_t scan_blockH5(size_t blocksize,H5std_string filename){
// read in a block of points
size_t i=0, j=0;
int k=6;
hsize_t nrows, ncols=7, offset=0;
double *data;
data = common_scan_blockH5(filename, "/dset", blocksize, ncols, offset, &nrows);
points.resize(nrows);
for (i = 0; i < nrows; i++) {
if (((size_t)data[i*ncols+k] == 1) || ((size_t)data[i*ncols+k] == 2)) {
points[j].x[0] = data[i*ncols ];
points[j].x[1] = data[i*ncols+1];
points[j].x[2] = data[i*ncols+2];
points[j].mass = data[i*ncols+3];
points[j].r_max = data[i*ncols+4];
points[j].r_scale = data[i*ncols+5];
j++;
}
}
points.resize(j);
delete data;
for(auto &h: points){
h.r_max /= 1.0e3;
h.r_scale /= 1.0e3;
}
return j;
}
double *common_scan_blockH5(H5std_string filename
, H5std_string dataset_name
, hsize_t n_rows
, hsize_t n_cols
, hsize_t offset_rows
, hsize_t *read_rows
);
}; //CHANGED ***** MARCOS
/ /
void random_observers(std::vector
struct DkappaDz{ DkappaDz(const COSMOLOGY &cos,double zsource):cosmo(cos),zs(zsource){ rho = cosmo.getOmega_matter()*cosmo.rho_crit(0); };
double operator()(double z){
double x = 1+z;
return cosmo.drdz(x)*rho*x*x*x/cosmo.SigmaCrit(z,zs)/cosmo.getHubble();
}
const COSMOLOGY &cosmo;
double zs;
double rho;
};
using Utilities::Geometry::Quaternion; using Utilities::Geometry::SphericalPoint; /** \brief This function goes directly from snapshots to lensing maps with Born approximation and linear propogations.
The template parameter allows this function to use different input data formats and
different ways of distributing the mass into grid cells. The current options are
LightCones::ASCII_XV -- for 6 column ASCII file with position and velocity
LightCones::ASCII_XM -- for 5 column ASCII file with position and mass
LightCones::ASCII_XMR -- for 6 column ASCII file with position, mass and size
LightCones::ASCII_XMRRT -- for 7 column ASCII file with position, mass,Rmax,Rscale and an integer denoting type, the halos are rendered as truncated NFWs
LightCone::ASCII_XMRRT12 -- same as LightCones::ASCII_XMRRT but only takes entries with the 7th column equal to 1 or 2
<\p>
*/
template
It would be better to derive a new class and redefine scan_block()
instead of making a new function scan_blockH5()
because FastLightCone
is template to accept a structureASCII_ ...
and uses what function it has called scan_block()
. I did this so that the other parts of FastLightCone
don't need to be changes for different situations. FastLightCone
can be used for halos or particles or for different input formats by just changing the template argument. This makes it much easier to use from the outside and reduces duplicate code.
@ntessore can you help us with linking the HDF5 library through cmake?
Is this going to be an optional dependency going on the master branch? If so I will add a PR for it
I have pushed a branch to the glamer repository where ENABLE_HDF5 exists, which should then build projects with HDF5 support. Will HDF5 be necessary for SLsimLib itself?
Hi Nicolas,
yes I am using HDF5 routines at SLsimLib to be able to read in this format.
Cheers! Marcos.
On 07/07/2017 11:09 AM, Nicolas Tessore wrote:
I have pushed a branch to the glamer repository where ENABLE_HDF5 exists, which should then build projects with HDF5 support. Will HDF5 be necessary for SLsimLib itself?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/glenco/SLsimLib/issues/131#issuecomment-313642630, or mute the thread https://github.com/notifications/unsubscribe-auth/AVsIK5MuP3M2q9epwkj_vbZhQ8-z9BMdks5sLgPngaJpZM4OMpVb.
ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a http://www.iac.es/disclaimer.php WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult http://www.iac.es/disclaimer.php?lang=en
Ok, to test this, try and do the following three steps:
update and checkout the test branch for the glamer folder:
$ cd glamer
$ git pull && git checkout nt/hdf5
update and checkout the test branch for the SLsimLib folder:
$ cd SLsimLib
$ git pull && git checkout nt/hdf5
add the (unfortunately necessary) definitions to your project's CMakeLists.txt:
# somewhere in file CMakeLists.txt
...
find_package(GLAMER NO_MODULE REQUIRED)
include_directories(${GLAMER_INCLUDE_DIRS})
add_definitions(${GLAMER_DEFINITIONS}) # <-- this line is new
...
Then you should be able to use cmake .. -DENABLE_HDF5=1
when building GLAMER to find the HDF5 library. Subsequently building your project should no longer need any intervention.
Hi Ben,
I finally have a working code inside glamer that reads HDF5 in chuncks of a million (that can be changed) where I followed your advice and used another class for reading this:
Konsole output structHDF5_XMRRT12:public ASCII_XMRRT{
using scan_block(). Because of the structure of how H5 is coded I needed to change a bit the arguments I passed to scan_block() so now I am passing
Konsole output scan_block(size_tblocksize,FILEpFile,H5std_stringfilename,hsize_toffset_rows,boolH5eof)
I made sure that it still works in the other ACII_ structures so I guess this is not a big problem (it might give some warnings because we are not using some arguments when reading from ASCII but that won't be a problem, I agree it might not be the most beautiful way).
Something that I don't like is that I call a function (common_scan_blockH5() that reads in H5 format a set of data) from the structure HDF5_XMRRT12 and, as I didn't know where to put it, it is now at lightcones_construction.h, just before overriding scan_block(). Having a function at a .h file is very ugly. Could you tell me where can I put that function so that lightcones_construction sees it and can call it?
Summarizing, I've got a code, and it works, but it is still ugly, it needs some polishing. Can I "git pull" it? Is it as simple as typing "git pull" at the SLsimlib folder? I am attaching the code anyway.
Cheers! Marcos.
** On 07/07/2017 11:59 AM, Nicolas Tessore wrote:
Ok, to test this, try and do the following three steps:
1.
update and checkout the test branch for the *glamer* folder: |$ cd glamer $ git pull && git checkout nt/hdf5 |
2.
update and checkout the test branch for the *SLsimLib* folder: |$ cd SLsimLib $ git pull && git checkout nt/hdf5 |
3.
add the (unfortunately necessary) definitions to *your project*'s CMakeLists.txt: |# somewhere in file CMakeLists.txt ... find_package(GLAMER NO_MODULE REQUIRED) include_directories(${GLAMER_INCLUDE_DIRS}) add_definitions(${GLAMER_DEFINITIONS}) # <-- this line is new ... |
Then you should be able to use |cmake .. -DENABLE_HDF5| when building GLAMER to find the HDF5 library. Subsequently building your project should no longer need any intervention.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/glenco/SLsimLib/issues/131#issuecomment-313651746, or mute the thread https://github.com/notifications/unsubscribe-auth/AVsIK4ry4UjmcxU430IeDXJf2cdlFhR6ks5sLg9_gaJpZM4OMpVb.
ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a http://www.iac.es/disclaimer.php WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult http://www.iac.es/disclaimer.php?lang=en
// // lightcone_construction.h // GLAMER // // Created by Ben Metcalf on 26/10/16. // //
using std::cout; using std::endl;
using namespace H5;
/* \brief The LightCones namespace is for classes and functions related to making light cones and weak lensing maps from 3D snapshots of particles or halos. / namespace LightCones{
struct DataRockStar{ unsigned int id; Point_3d x; double mass; double Rvir; // in comoving units double Rscale; // in comoving units double Vmax; };
struct DatumXM{ Point_3d x; double mass; }; struct DatumXMRmRs{ Point_3d x; double mass; double r_max; double r_scale; }; struct DatumXMR{ Point_3d x; double mass; double r; };
void ReadLightConeNFW(std::string filename,COSMOLOGY &cosmo ,std::vector<LensHalo* > &lensVec ,PosType &theta_max);
void ReadLightConeParticles(std::string filename,COSMOLOGY &cosmo ,std::vector<LensHaloParticles *> &lensVec ,int Nplanes ,float particle_mass ,float particle_size ,bool angular_sizes = false ,bool verbose = false);
void ReadLightConeParticles(std::string filename,COSMOLOGY &cosmo ,std::vector<LensHaloMassMap *> &lensVec ,int Nplanes ,float particle_mass ,float angular_resolution ,bool verbose = false);
/** \brief Class for constructing light cones form simulation boxes. / class LightCone{ public: LightCone(double angular_radius);
void WriteLightCone(std::string filename,std::vector<DataRockStar> &vec);
void WriteLightCone(std::string filename,std::vector<Point_3d> &vec);
friend class MultiLightCone;
template <typename T>
void select(Point_3d xo,Point_3d v,double Length,double rlow,double rhigh
,T* begin
,T* end
,Utilities::LockableContainer<std::vector<T> > &incone
,bool periodic_boundaries = true);
private:
double r_theta; // angular radius of light cone (radians)
double sin_theta_sqrt;
};
/// select the halos from the box that are within the light cone
template
if(begin == end) return;
Point_3d dx,x;
double r2,xp;
Point_3d n;
double rhigh2 = rhigh*rhigh;
double rlow2 = rlow*rlow;
T b;
v /= v.length();
// standard coordinates so cone is always centered on the x-axis
Point_3d y_axis,z_axis;
y_axis[0] = -v[1];
y_axis[1] = v[0];
y_axis[2] = 0;
y_axis /= y_axis.length();
z_axis = v.cross(y_axis);
//std::cout << v*y_axis << " " << v*z_axis << " " << y_axis*z_axis << std::endl;
// we can do better than this !!!!
int n0[2] = {(int)((rhigh + xo[0])/Length),(int)((xo[0] - rhigh)/Length) - 1 };
int n1[2] = {(int)((rhigh + xo[1])/Length),(int)((xo[1] - rhigh)/Length) - 1 };
int n2[2] = {(int)((rhigh + xo[2])/Length),(int)((xo[2] - rhigh)/Length) - 1 };
if(!periodic_boundaries){
n0[0] = n0[1] = 0;
n1[0] = n1[1] = 0;
n2[0] = n2[1] = 0;
}
for(auto a = begin ; a != end ; ++a){
dx[0] = (*a).x[0] - xo[0];
dx[1] = (*a).x[1] - xo[1];
dx[2] = (*a).x[2] - xo[2];
int count = 0;
for(n[0] = n0[1]; n[0] <= n0[0] ; ++n[0]){
for(n[1] = n1[1]; n[1] <= n1[0] ; ++n[1]){
for(n[2] = n2[1]; n[2] <= n2[0] ; ++n[2]){
x = dx + n*Length;
xp = x*v;
if(xp > 0){
r2 = x.length_sqr();
if(r2 > rlow2 && r2 < rhigh2){
if( r2*sin_theta_sqrt > r2 - xp*xp){
//std::cout << n << " | " << xo << " | "
// << v << " | " << x << std::endl;
b = *a;
//b.x = x;
// rotate to standard reference frame
b.x[0] = xp;
b.x[1] = y_axis*x;
b.x[2] = z_axis*x;
//std::cout << "b = " << b.x << std::endl;
incone.push_back(b);
++count;
}
}
}
}
}
}
if(count > 1) std::cout << " Warning: Same halo appears " << count << " times in light-cone." << std::endl;
}
}
class MultiLightCone{
public:
MultiLightCone(
double angular_radius
,const std::vector
/** Read the points in from a snapshot of the halos created by RockStar.
The output will be the halos in the cone in coordinates where the x-axis is
the center of the cone. Changing the observer, xo, and direction of view, V,
effectively translates and rotates the box, respectively. Only halos with radii
between rlow and rhigh are added.
*/
void ReadBoxRockStar(std::string filename
,double rlow,double rhigh
,std::vector<std::vector<LightCones::DataRockStar> > &conehalos
,bool periodic_boundaries = true
,bool allow_subhalos = false);
void ReadBoxXYZ(std::string filename
,double rlow,double rhigh
,std::vector<std::vector<Point_3d> > &conehalos
,double hubble
,double BoxLength
,bool periodic_boundaries = true
);
void ReadBoxToMaps(std::string filename
,double rlow,double rhigh
,std::vector<std::vector<PixelMap> > &maps
,double hubble
,double BoxLength
,bool periodic_boundaries = true
);
private:
std::vector<Point_3d> xos;
std::vector<Point_3d> vs;
std::vector<LightCone> cones;
};
using Utilities::Geometry::Quaternion; using Utilities::Geometry::SphericalPoint;
/ templated functions for projecting mass onto planes /
/ templated functions for reading a block from a file /
template
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf %lf"
,&points[i][0],&points[i][1],&points[i][2],&tmpf,&tmpf,&tmpf) != EOF)
++i;
points.resize(i);
return i;
}
/ structures to implement templated FastLightCones<>() /
struct ASCII_XV{
std::vector<Point_3d> points;
size_t scan_block(size_t blocksize,FILE *pFile,H5std_string filename,hsize_t offset_rows,bool *H5eof){
double tmpf;
// read in a block of points
size_t i=0;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf %lf"
,&points[i][0],&points[i][1],&points[i][2],&tmpf,&tmpf,&tmpf) != EOF)
++i;
points.resize(i);
return i;
}
static void fastplanes_parallel(
Point_3d *begin
,Point_3d *end
,const COSMOLOGY &cosmo
,std::vector<std::vector<Point_3d> > &boxes
,std::vector<Point_3d> &observers
,std::vector<Quaternion> &rotationQs
,std::vector<double> &dsources
,std::vector<std::vector<PixelMap> > &maps
,double dmin
,double dmax
,double BoxLength
);
}; struct ASCII_XM{
std::vector<DatumXM> points;
size_t scan_block(size_t blocksize,FILE *pFile,H5std_string filename,hsize_t offset_rows,bool *H5eof){
// read in a block of points
size_t i=0;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf"
,&points[i].x[0],&points[i].x[1],&points[i].x[2],&points[i].mass) != EOF)
++i;
points.resize(i);
return i;
}
static void fastplanes_parallel(
DatumXM *begin
,DatumXM *end
,const COSMOLOGY &cosmo
,std::vector<std::vector<Point_3d> > &boxes
,std::vector<Point_3d> &observers
,std::vector<Quaternion> &rotationQs
,std::vector<double> &dsources
,std::vector<std::vector<PixelMap> > &maps
,double dmin
,double dmax
,double BoxLength
);
};
struct ASCII_XMR{
std::vector<DatumXMR> points;
size_t scan_block(size_t blocksize,FILE *pFile,H5std_string filename,hsize_t offset_rows,bool *H5eof){
// read in a block of points
size_t i=0;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf"
,&points[i].x[0],&points[i].x[1],&points[i].x[2]
,&points[i].mass,&points[i].r) != EOF)
++i;
points.resize(i);
return i;
}
static void fastplanes_parallel(
DatumXMR *begin
,DatumXMR *end
,const COSMOLOGY &cosmo
,std::vector<std::vector<Point_3d> > &boxes
,std::vector<Point_3d> &observers
,std::vector<Quaternion> &rotationQs
,std::vector<double> &dsources
,std::vector<std::vector<PixelMap> > &maps
,double dmin
,double dmax
,double BoxLength
);
};
struct ASCII_XMRRT{
std::vector<DatumXMRmRs> points;
size_t scan_block(size_t blocksize,FILE *pFile,H5std_string filename,hsize_t offset_rows,bool *H5eof){
// read in a block of points
size_t i=0;
int tmp;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf %lf %i"
,&points[i].x[0],&points[i].x[1],&points[i].x[2]
,&points[i].mass,&points[i].r_max,&points[i].r_scale,&tmp) != EOF)
++i;
points.resize(i);
for(auto &h: points){
h.r_max /= 1.0e3;
h.r_scale /= 1.0e3;
}
return i;
}
static void fastplanes_parallel(
DatumXMRmRs *begin
,DatumXMRmRs *end
,const COSMOLOGY &cosmo
,std::vector<std::vector<Point_3d> > &boxes
,std::vector<Point_3d> &observers
,std::vector<Quaternion> &rotationQs
,std::vector<double> &dsources
,std::vector<std::vector<PixelMap> > &maps
,double dmin
,double dmax
,double BoxLength
);
};
struct ASCII_XMRRT12:public ASCII_XMRRT{ size_t scan_block(size_t blocksize,FILE pFile,H5std_string filename,hsize_t offset_rows,bool H5eof){
// read in a block of points
size_t i=0;
int tmp;
points.resize(blocksize);
while(i < blocksize &&
fscanf(pFile,"%lf %lf %lf %lf %lf %lf %i"
,&points[i].x[0],&points[i].x[1],&points[i].x[2]
,&points[i].mass,&points[i].r_max,&points[i].r_scale,&tmp) != EOF){
if(tmp == 1 || tmp == 2) ++i;
}
points.resize(i);
for(auto &h: points){
h.r_max /= 1.0e3;
h.r_scale /= 1.0e3;
}
return i;
}
};
struct HDF5_XMRRT12:public ASCII_XMRRT{
// added by Marcos Pellejero & Antonio Dorta*** /Modified by Marcos and Antonio 4th July 2017/
double common_scan_blockH5(H5std_string filename, H5std_string dataset_name, hsize_t n_rows, hsize_t n_cols, hsize_t offset_rows, hsize_t read_rows)
{
const int RANK = 2;
double data_out = NULL;
read_rows = 0;
// Try block to detect exceptions raised by any of the calls inside it
try
{
/*
Define the memory space to read dataset.
*/
hsize_t file_rows = dims[0], file_cols = dims[1];
bool readALL = false;
if (n_cols != file_cols) { cout << "ERROR!!! You specified " << n_cols << " cols, but file contains " << dims[1] << " cols.\n"; return data_out; }
if (offset_rows > file_rows) {
// Reading out of data
cout << "ERROR!!! File has " << file_rows << " rows and you wanted to read till row " << offset_rows + n_rows << "\n";
return data_out;
}
else if ((offset_rows == 0) && (file_rows >= n_rows)) {
// Read ALL data
count_rows = n_rows;
readALL = (count_rows == file_rows);
cout << "Reading " << count_rows << "\n";
}
else if (offset_rows + n_rows > file_rows) {
// We are out of bound, limit the max rows
count_rows = file_rows - offset_rows;
cout << "Reading LIMIT ROWS: " << count_rows << " FROM ROW " << offset_rows << "\n";
}
else {
// Inside bounds
count_rows = n_rows;
cout << "Reading INSIDE\n";
}
/*
Read dataset back and display.
*/
// Allocate only the required memory data_out = new double [count_rowsfile_cols]; if (data_out == NULL) { cout << "NOT ENOUGH MEMORY!!!\n"; read_rows = 0; return NULL; }
// READ DATA!!! (we are going to read count_rows x file_cols) if (readALL) { DataSpace memspace(RANK, dims); dataset.read(data_out, PredType::NATIVE_DOUBLE, memspace, filespace); } else { hsize_t count[RANK]; count[0] = count_rows; count[1] = file_cols; offset[0] = offset_rows; offset[1] = 0; DataSpace memspace(RANK, count); filespace.selectHyperslab(H5S_SELECT_SET, count, offset); dataset.read(data_out, PredType::NATIVE_DOUBLE, memspace, filespace); } *read_rows = count_rows;
} // end of try block // catch failure caused by the H5File operations catch( FileIException error ) { error.printError(); read_rows = 0; return NULL; } // catch failure caused by the DataSet operations catch( DataSetIException error ) { error.printError(); read_rows = 0; return NULL; } // catch failure caused by the DataSpace operations catch( DataSpaceIException error ) { error.printError(); *read_rows = 0; return NULL; } return data_out; }
//CHANGED ***** MARCOS // read in a block of points size_t scan_block(size_t blocksize,FILE pFile,H5std_string filename,hsize_t offset_rows,bool H5eof){ // HDF5 needs filename // Offset (rows) // Since it does NOT use pointer to file, we need to tell when the EOF is reached. // We could return -1 when reaching EOF, but since site_t could be unsigned, that would NOT work // Instead of that, we return H5eof -> true when EOF size_t i=0, j=0; int k=6; H5eof = false; hsize_t nrows, ncols=7; double data; cout << "Reading " << blocksize << " from offset " << offset_rows << "\n"; H5eof = false; data = common_scan_blockH5(filename, "/dset", blocksize, ncols, offset_rows, &nrows); if ((data == NULL) || (nrows == 0)) { H5eof = true; return 0; } points.resize(nrows); for (i = 0; i < nrows; i++) { if (((size_t)data[incols+k] == 1) || ((size_t)data[incols+k] == 2)) { points[j].x[0] = data[incols ]; points[j].x[1] = data[incols+1]; points[j].x[2] = data[incols+2]; points[j].mass = data[incols+3]; points[j].r_max = data[incols+4]; points[j].r_scale = data[incols+5]; j++; } } points.resize(j); delete data;
for(auto &h: points){ h.r_max /= 1.0e3; h.r_scale /= 1.0e3; } return j; }
}; //CHANGED ***** MARCOS
/ /
void random_observers(std::vector
struct DkappaDz{ DkappaDz(const COSMOLOGY &cos,double zsource):cosmo(cos),zs(zsource){ rho = cosmo.getOmega_matter()*cosmo.rho_crit(0); };
double operator()(double z){ double x = 1+z; return cosmo.drdz(x)rhoxxx/cosmo.SigmaCrit(z,zs)/cosmo.getHubble(); }
const COSMOLOGY &cosmo; double zs; double rho; };
using Utilities::Geometry::Quaternion; using Utilities::Geometry::SphericalPoint; /** \brief This function goes directly from snapshots to lensing maps with Born approximation and linear propogations.
The template parameter allows this function to use different input data formats and
different ways of distributing the mass into grid cells. The current options are
LightCones::ASCII_XV -- for 6 column ASCII file with position and velocity
LightCones::ASCII_XM -- for 5 column ASCII file with position and mass
LightCones::ASCII_XMR -- for 6 column ASCII file with position, mass and size
LightCones::ASCII_XMRRT -- for 7 column ASCII file with position, mass,Rmax,Rscale and an integer denoting type, the halos are rendered as truncated NFWs
LightCone::ASCII_XMRRT12 -- same as LightCones::ASCII_XMRRT but only takes entries with the 7th column equal to 1 or 2
<\p>
*/
template
}
Hi Nicolas,
sorry, I don't yet control github and am like a monkey trying to figure out how it works. I tried what you mentioned in point one but it complains about the following:
Konsole output fatal: Not a git repository (or any parent up to mount point /home/mpi) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
/home/mpi is myself by the way. Right now what I do to compile with HDF5 is go to the code we made and, after trying to compile it with the usual make that fails (obviously) I take the linking bit of the products the make trial produces and add -lhdf5_cpp -lhdf5. I compile with something like:
Konsole output /usr/lib64/ccache/c++ -std=c++11 CMakeFiles/Cones.dir/cones.cpp.o -o Cones -rdynamic /home/mpi/Documents/Weak_lensing_project/glamer-master/build/SLsi\ mLib/libSLsimLib.a /home/mpi/Documents/Weak_lensing_project/glamer-master/build/CosmoLib/libCosmoLib.a /home/mpi/Documents/Weak_lensing_project/glamer-master\ /build/NR/libNR.a -lpthread -lCCfits /usr/local/lib64/libcfitsio.so -lfftw3 -Wl,-rpath,/usr/local/lib64 -lhdf5_cpp -lhdf5
Cheers! Marcos.
On 07/07/2017 11:59 AM, Nicolas Tessore wrote:
Ok, to test this, try and do the following three steps:
1.
update and checkout the test branch for the *glamer* folder: |$ cd glamer $ git pull && git checkout nt/hdf5 |
2.
update and checkout the test branch for the *SLsimLib* folder: |$ cd SLsimLib $ git pull && git checkout nt/hdf5 |
3.
add the (unfortunately necessary) definitions to *your project*'s CMakeLists.txt: |# somewhere in file CMakeLists.txt ... find_package(GLAMER NO_MODULE REQUIRED) include_directories(${GLAMER_INCLUDE_DIRS}) add_definitions(${GLAMER_DEFINITIONS}) # <-- this line is new ... |
Then you should be able to use |cmake .. -DENABLE_HDF5| when building GLAMER to find the HDF5 library. Subsequently building your project should no longer need any intervention.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/glenco/SLsimLib/issues/131#issuecomment-313651746, or mute the thread https://github.com/notifications/unsubscribe-auth/AVsIK4ry4UjmcxU430IeDXJf2cdlFhR6ks5sLg9_gaJpZM4OMpVb.
ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a http://www.iac.es/disclaimer.php WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult http://www.iac.es/disclaimer.php?lang=en
You are probably not using a version-controlled source then. Perhaps @rbmetcalf could test it instead.
Hi Marcos. You can make a pull request with your changes and we can review it for you. I have been away, but maybe this coming week we can sit down in a video con and work through how to use some of these development tools.
Hi Ben,
I created a pull-request but I am not sure if I did it correctly. Actually, I created two of them so one of them is wrong for sure. Can you check it please?
If it's wrong I'll try again.
Cheers! Marcos.
On 07/08/2017 05:07 PM, Ben Metcalf wrote:
Hi Marcos. You can make a pull request with your changes and we can review it for you. I have been away, but maybe this coming week we can sit down in a video con and work through how to use some of these development tools.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/glenco/SLsimLib/issues/131#issuecomment-313865059, or mute the thread https://github.com/notifications/unsubscribe-auth/AVsIKyMvkiiChva97jjsU2xGuO67tgcQks5sL6lYgaJpZM4OMpVb.
ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a http://www.iac.es/disclaimer.php WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult http://www.iac.es/disclaimer.php?lang=en
@mpellejero lets continue our discussion of this here.