TDE models - Difference in flux normalisation for spectra with different wavelength ranges

[saultyevil]

In attempt to remove any possible edge effects from some weird spectra, I re-ran the simulation but with a much larger wavelength range for the spectrum.

But, I found that when I compared the two models, the restricted range and the larger range, that the flux normalisation was a bit off.

tdeCV.pf.txt

Uh oh! The above spectra were generated in py83. Just to make sure it wasn't an issue with py83, I've re-run the same runs in py82i...

tdeCV.pf.txt

and py82k...

Uh oh! Same problem in all three problems.

I tried the same tests with a different TDE model based on the standard AGN pf, and there was no problem here.

py83 tdeAGN.pf.txt

py82i tde.pf.txt

Apart from a bit of noise and some edge effects, everything seems fine!

So now I'm quite lost how changing spectrum.wavemax and spetrum.wavemin is changing the normalisation in the CV model but not the AGN model. Does anyone have any ideas?

P.S. I'm currently running a CV model without a binary component since there are some numerical issues with setting up Roche lobes. So this should eliminate if that is anything to do with it.

[saultyevil]

Ok, so removing the binary component by using a star system seems to have the same problem.

tdeCV.pf.txt

[Higginbottom]

I have done some very fast runs, with only 10000 photons, two ionisation cycles. I run those two and then restart from the same windsavefile for a restricted and wide range

Here is my small vs big 62.5 degree

plot

There are small differences. If I look at the wind photons only

there is clearly not much difference

But- these are the disk generated protons.

One thing that does occur to me is that maybe if you change the spectral range, you could remove certain annuli of the disk from emitting anything, so you could change the spectral shape of the continuum quite a lot. This seems to be a similar effect to what ed is seeing...

How to check this...

[saultyevil]

I think I've been seeing the same sort of thing in my longer runs as well. Generated wind photons between the small and large range are different (solid line), but there is a larger gap for disk photons (dotted line) between the two runs.

[Higginbottom]

Ok, so Ed has sent me his spectral files, plotting each of the columns in the spec file, I note one particularly interesting one. This is the 'created' photons plot

or on a log scale

by extending the spectral range we are just including the gigantic hydrogen recombination photon contribution from the wind. Basically none of these get out as we see from the 'wind' spectrum

but I could well imagine off things happening. For example the 'disk' spectrum

might this be the different annuli being allowed to emit..

[Higginbottom]

So, as Ed noted, there is no significant difference in the AGN spectrum

This is the emitted spectrum...

compared to CV

but look at the difference in created photons between the CV and and AGN in the wide band

There is that odd feature that I initially thought was recombination photons in the agn model.

[jhmatthews]

Guys, is this a macro-atom model or non-macro?

[saultyevil]

Non-macro

[kslong]

My suggestion is to run another model with a lower wind velocity and see if we can sort out what this is. I think recombination is a good bet.I would reduce m_dot_wind at the same time so that the density in the wind remains constant.

Additionally, I would, as I suggested, extract photons for unscattered, scattered once, and scattered two events … or something like that

[kslong]

OK, I have now run some of these models, and in particular tdeCV.pf above. I have also run a model with a wider spectral range. I confirm the types of things Ed was seeing. But I have some comments as well. First, here is the full spectrum as calculated int he last ionization cycle:

As you can see it has lots of recombination edges. My suspicion is that is also what we are seeing near 3,500 A in the detailed spectra, although I did not believe we could get an Hbeta edge with this ionization mode. Someone should check on this.

Recombination dominates the luminosity of the wind, as shown in this extract from the last ionization cycle.

Rotational wind velocities here are quite high, approaching 1/3 of the speed of light in this model. This is not the case in our standard AGN models, because we start the wind at about 50 rstar in those models.

Nearly all photons are being scattered and most of them are being scattered multiple times. Some photons scatter almost 500 times. Comparing the total number of scatters in the diag file to the numbers of resonant scatters, it appears that electron scattering is dominant. With this number of scatters (and the high speed of the wind) it is not surprising that features are very broad.

The wind is being truncated significantly by wind rmax, which is set to 1e16, while the acceleration length is set to 5e16. Unless one believes something is cutting the wind off, one needs to set wind rmax to something significantly greater than 5e16, something like 5e17. (But one will probably need to increase the dimension, currently 30x30, in the wind when you do this.

The temperatures radiation and electron look plausible. The electron temperatures are less than the radiation temperatures by a factor of up to two, but this is something we have seen before.

[kslong]

@saultyevil There was a problem with renormalization after restarts see #503, but this should now be solved. Can you please determine how much of this issue was associated with the renormalization problem, and how much remains. Note that if we are still getting normalization issues that have to do with the wavelength range chosen in the final spectra, I suspect we at least want to determine when this happens and warn the user, maybe by estimating how much of the detailed spectra is set by edge effects. I do think we should figure out what is needed to close out this issue, something that is not apparent from the discussion.

[saultyevil]

I re-ran the tdeCV.pf model posted at the very top and the issue still seems to be here.

Just to be rigorous, I tried the same thing above with a different TDE model.

tde_normalisation.pf.txt

Thankfully, both runs agree for this model. But now I'm wondering what's happening with the other model. The issue does not appear to be related to the renormalisation issue which was recently fixed, but something else. The wavelength range in both plots is different, so I'll double check to see if that makes a difference first otherwise I'll have a closer look at the tdeCV model...

[kslong]

Hmm, a difference between these two models is the amount of interaction. I would check carefully where there are lot's of photons being lost somehow in the first example, and whether it is different between the two runs.

I would also try to figure out whether it is the weights of the photons which

I might also run a model with an intermediate wavelength range, say 500-3000 A to verify that that this is some how associated with the exact wavelength range (which is what I would expect)

Are these models run from scratch? or using restart?

[saultyevil]

My recent runs are all from scratch so there's no interference from something like issue #515. I've re-run tde_normalisation for an intermediate wavelength range and that seems fine apart from at the edges.

I was also curious if it was maybe related to the possible recombination edge seen. So I re-ran another TDE model where I've seen a similar feature and it was the same as above.

I think what's causing the problem is related to an error from spectrum_create,

Error: spectrum_create: Fraction of photons lost: 0.08 wi/ freq. low, 0.19 w/freq hi

I believe this tracks the number of photons which have have their frequencies shifted outside the provided spectrum range. The fraction of photons lost is higher for the smaller wavelength range, which would explain the lower flux. I see similar behaviour in the tde_normalisation model, but less photons are being lost which would explain why it seems fine for the most part apart from near the edges. Maybe we want to also include a call to photon_checks after photon transport in the spectral cycles to warn the user about photons which have been shifted outside the range of the spectrum.

[kslong]

@saultyevil I am losing track here. Is your opinion now that the scaling is actually correct, and that this particular problem is due to edge photons. That is something I could easily believe (and would be separate from the new issue #515 or the old #503, which I believe I solved.

[saultyevil]

Yes, I think so. Although I can still replicate the issue in the first post of this thread, I feel like the large velocities in the tdeCV model, and the fact that it seems to be causing a decent fraction of photons to become lost in the spectrum (mostly at the edges) is the problem. It would probably be sensible to add a clear warning about normalisation when Python detects a large fraction of photons are lost or either intelligently adapt the spectrum range (which could be an advanced mode) to account for the lost photons.

[kslong]

@saultyevil - I think given the long thread of information here, we need a clear restatement of whether we understand the problem fully, or if there are unresolved issues. If everything is due getting a spectra in a limited range, we need to decide if the user needs to be warned when the problem has occured.

[saultyevil]

Ok, so..

The problem I encountered was that for two different spectral wavelength ranges, the normalisation of the flux is different between the two ranges; at least in the model originally posted. Specifically, I encountered this problem when I generated a spectrum over a restricted wavelength range (1100 - 2600 A) and again for a broader range (500 - 5000 A). I found that the spectrum with the broad range had a larger flux than the spectrum with the restricted range.

I believe one of the causes of this issue is due to edge effects. Effectively, as the velocity is large (0.2-0.5c) in the wind, photon frequiences are significantly shifted outside the spectral wavelength range. This results in the error spectrum_create: Fraction of photons lost: 0.10 wi/ freq. low, 0.19 w/freq hi as a fairly significant amount of photons are not being binned in the restricted spectrum wavelength range - effectively losing their contribution to the spectrum at the edges.

But it also appears that a a decent fraction of weight is removed due to photons being lost to "too many scatters" (photons which scatter > 500). To test this, I ran a model with MAXSCAT = 50. This model has significantly lower flux than the other two models. Closer inspection of the broad and restricted models revealed that photons in the restricted model were undergoing more scatters and more photons were removed due to MAXSCAT. But why does this model have more scatters anyway..? When I removed the MAXSCAT limit, the model was slightly closer to the broad spectrum range but not perfect - it doesn't look like this is the only cause.

For anyone interested, I have updated the parameter file showing this problem to work with the current version of Python. Note that there are currently no spectral cycles in the pf however, and that there may be a few issues due to superluminal motion :-).

tde_flux.pf.txt

[saultyevil]

I have created a wiki entry to document this behaviour as we decided during our Southampton meet up, hence I am going to close this issue.