The coordinates.

[qinyuxian]

Hi John,

The coordinates and densities of gases which are used to generate the photons is in comoving or in proper? The photon list is saved in comoving or in proper? Thank you.

[jzuhone]

The coordinates are converted to proper when the list is saved. They don't come into account when it is generating the photons.

[jzuhone]

Densities are proper as well.

[qinyuxian]

So the pixel coordinates are from proper photon coordinates?

[jzuhone]

Yes, that's correct. Is something incorrect or unexpected?

[qinyuxian]

Before I use pyxsim, I shall traslate the gas particle parameters (i.e. coordiates, densities) into proper according to the redshift? Almost of the simulations save their data in comoving.

[jzuhone]

No, yt handles this for you completely, so there is no need to do it on your own.

[qinyuxian]

But I don't use yt to load the data.

[jzuhone]

pyXSIM only works with yt--it needs yt to figure out the various fields (emission measure, temperature, metallicity, etc). I would check out some of the cookbook examples to see how it works.

http://hea-www.cfa.harvard.edu/~jzuhone/pyxsim/cookbook/index.html

pyXSIM works fine with both grid-based and particle data. What kind of dataset do you have?

[qinyuxian]

IllustrisTNG. If I use yt, my machine may crash.

[jzuhone]

I've been using yt-4.0 (it's in beta) on TNG300 data fairly regularly without issue. Feel free to email me at jzuhone@gmail.com and I can give you some general pointers.

[qinyuxian]

I know. It seems costs a lot time when it loads the data. I just use the python script given by TNG team to read the data.

[qinyuxian]

So the original data read from the snapshot is in comoving. I should translate them in proper if I want to observe at high redshift.

[qinyuxian]

The densities shall be scaled by (1+z)**3 if the snapshot is at z. Do you think so?

[jzuhone]

I think so. But the way pyXSIM is designed, there's no way to get the data into it without yt.

I think you may be having memory problems with yt if you're using yt-3.x, which does not do Lagrangian data well and does not really support Arepo data at all. What version did you try using?

[jzuhone]

Did you download a single halo from TNG? With yt-4.0 and pyXSIM it creates an observation of such a halo in just a few minutes.

[qinyuxian]

I want to observe large scale structure. Single halo is not enough.

[jzuhone]

Ah, ok. I still think yt-4.0 would work for you, unless you have tried it already. yt-3.x was really bad with memory for Lagrangian datasets. Are you trying to do a light cone simulation (with several snapshots) or just a single snapshot?

[qinyuxian]

Light cone is my goal.

[qinyuxian]

By the way, when use angular diameter distance to calculate the angle, the coordinates shall be in proper?

[jzuhone]

Yes, that's correct.

So unfortunately I think that we have to decide if you want to use yt or not--if you do not, that's fine, but there's no way to use pyXSIM without it. There's no entry point into pyXSIM where you open the data in some other way, as doing this would require you to rewrite about half of pyXSIM.

[jzuhone]

I really do think we could get it to work for your use case with the latest code--it works very well compared to previous versions, and it would save you a ton of time.

[qinyuxian]

That is the question. It requires the parameters should be translated in proper before they go into pyxsim.

[jzuhone]

Not if you use yt--it doesn't require that at all. Yt knows that it's looking at a cosmo sim and can handle it. I can set up an example script later tonight if you like

[qinyuxian]

In fact, I have rewrote a lot code for pyxsim before you released the new version. A lot of mock observations have been done. Unfortunately, I now realize the question. I am crazy. Maybe my paper can not be submit.

[jzuhone]

Sorry, what do you mean by "rewrote code"?

[qinyuxian]

I didn’t use yt to load the data.

[qinyuxian]

All the parameters are in comoving.

[jzuhone]

I guess my point is that yt handles all of this stuff about units for you, regardless of your simulation type, so it should be used. My offer to give you an example to do it using yt without using lots of memory still stands. But I don't know how to help you otherwise, and I am sorry about that.

[qinyuxian]

Does the current version work for light cone?

[jzuhone]

We have a light cone module, yes. It will take some work to get it to run with TNG, because it needs some parameters. But I am happy to help figure that out, since I have wanted to get it to work for some time and just haven't found the time.

We will have to use yt though--the light cone stuff in pyXSIM requires it.

[qinyuxian]

In fact, I have done some work to generate light cone. But now I must revise my former codes.

[qinyuxian]

Hi John,

When I want to use ds.sphere to select the data, the radius is in comoving or proper?

[jzuhone]

It actually doesn't matter which units you put the sphere in, because yt converts them to proper for you at the end.

[qinyuxian]

It takes a lot of time in loading a complete snapshot.

[jzuhone]

Have you tried it with yt-4.0? I'm happy to help you do that. It shouldn't take a lot of time in that case. Yt also doesn't load the whole snapshot into memory unless you request it--it only uses the particles you need for the sphere, for example.

There is no point in trying to get pyXSIM to work without yt. It's not designed to work without it.

[qinyuxian]

I just set the center of sphere to the original coordinate which did not need to correct using h or 1+z, right?

[qinyuxian]

However, according to your example, I must first load the snapshot, and then choose the sphere. What’s more, I need more than one snapshot to build the light cone.

[jzuhone]

If you're using yt, yes, that's correct.

I do not feel I will be able to help you unless you want to use yt-4.0. I've asked you several times if you want to use it, but you haven't answered my question. If you do not, that's fine. But I don't think I will be able to help you with pyXSIM if you do not.

So if you'd like me to help you with yt-4.0, please let me know, but if not I don't feel that there is anything I can do. Because I don't know enough about your code and the way you're applying it to help without assuming something that may be wrong.

[jzuhone]

I'm happy to provide you a light cone example with yt, but I won't be able to do it until I go back to work tomorrrow.

[jzuhone]

See my message above:

https://github.com/jzuhone/pyxsim/issues/32#issuecomment-846560189

[qinyuxian]

Yes, I am trying yt 4.0 now. Sorry for replying late. At the beginning, yt crashed too many times for tng. I had to spend a lot time in rewriting the interface.

[jzuhone]

Tonight I will work on an example script for you. If you have tracebacks that would be helpful in diagnosing any problems.

[qinyuxian]

http://hea-www.cfa.harvard.edu/~jzuhone/pyxsim/light_cone.html Is this the light cone example? Can it work for tng?

[jzuhone]

Yes, it is. But we don't have the parameter file that TNG used, so we'll have to construct one. It's something I've been meaning to do for a long time. I will try to make this work tonight.

[jzuhone]

TNG300-1, 2, or 3?

[qinyuxian]

TNG100-1 and TNG300-1.

[jzuhone]

Closing for now. If there is a specific issue please file another one.