Closed twfang closed 4 years ago
Unfortunately, the model starts to produce NaNs after a longer test. George and I will have to look into the problem before we make any changes.
NaN traced back to -ve Te at low altitudes for flux-tube mp=73, lp=46. The -ve Te then traced to an issue with q-coordinate interpolation in the ExB transport routine. Investigating the issue as it calls into question all of the q-coordinate interpolation in general (ie, has Q been calculated correctly for the IPE grid?)
Here are the q values at low altitudes for the flux-tubes around mp73 lp46
Alt km mp73 lp46 mp73 lp47 mp74 lp46 mp74 lp47
0 90.0000 0.521460 0.499284 0.517652 0.495333
1 92.0000 0.520902 0.498723 0.517095 0.494774 2 94.0000 0.520344 0.498163 0.516538 0.494215 3 96.0000 0.519787 0.497603 0.515982 0.493657 4 98.0000 0.519230 0.497043 0.515426 0.493099 5 100.000 0.518673 0.496485 0.514871 0.492542 6 102.000 0.518117 0.495926 0.514316 0.491985 7 104.000 0.517562 0.495369 0.513762 0.491428 8 106.000 0.517007 0.494811 0.513209 0.490873 9 108.000 0.516453 0.494255 0.512656 0.490317 10 110.000 0.515899 0.493698 0.512103 0.489762 11 112.000 0.515346 0.493143 0.511551 0.489209 12 114.000 0.514793 0.492588 0.511000 0.488655 13 116.000 0.514241 0.492033 0.510449 0.488102 14 118.000 0.513690 0.491480 0.509899 0.487550 15 120.000 0.513139 0.490926 0.509349 0.486998 16 122.000 0.512588 0.490373 0.508800 0.486446 17 124.000 0.512038 0.489821 0.508251 0.485895 18 126.000 0.511489 0.489269 0.507703 0.485344 19 128.000 0.510940 0.488717 0.507155 0.484795
OK, here, for a given longitude slice (in this case mp=73), are the points in the ExB transport q interpolation that have not solved properly and left problematic factors (greater than 1). All below 200km, more towards the equator, and all in the Northern hemisphere:
Very clear analysis. Good stuff George.
I would suggest just making the interpolation consistent with setting Ne,.... Te, Ti values below 90 km equal to the value at 90 km. So it still has make a change. I know it is not as simple as that because it still tries to interpolate in q below 90 km which would have to be suppressed.
I don't understand the N/S asymmetry. We assume the south is the same as the north, i.e., we don't even do it in the south.
Tim
On Tue, May 19, 2020 at 4:07 PM gmillward notifications@github.com wrote:
OK, here, for a given longitude slice (in this case mp=73), are the points in the ExB transport q interpolation that have not solved properly and left problematic factors (greater than 1). All below 200km, more towards the equator, and all in the Northern hemisphere:
[image: incorrect_q_interp_locations] https://user-images.githubusercontent.com/24465054/82383270-b49ff400-99ea-11ea-856a-0c47ddb5ce21.png
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/NOAA-SWPC/IPE/issues/30#issuecomment-631109131, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5BFOA43PXMZ2EY4WROP53RSL7IDANCNFSM4MUJGS6A .
Tim
I have a fix and will implement today - I think my fix is what you are saying - if the q interpolation comes from a place below 90km
The same thing does happen in the south (kinda) - but the way the code is written (looping over q values) - the applicable q interpolation factors are defaults of 1.0 and 0.0 already - so it just so happens that it uses 90km correctly in the South - it's just in the North where the looping 'runs off the end' as it were
Anyhow, I had the fix implemented and running at the end of yesterday - and running the March 2015 storm where Tzu-Wei saw the NaNs - so just about to check how that went and then hopefully can push the fix up to the develop branch.
Incidentally - the way the q-value search is done is very inefficient computationally - I can sort that too, but it's lower priority because my fix will not make any difference in the numbers - just faster [I can explain this over Zoom, hard to write down]
Cheers
George.
On Wed, May 20, 2020 at 9:03 AM timfullerrowell notifications@github.com wrote:
Very clear analysis. Good stuff George.
I would suggest just making the interpolation consistent with setting Ne,.... Te, Ti values below 90 km equal to the value at 90 km. So it still has make a change. I know it is not as simple as that because it still tries to interpolate in q below 90 km which would have to be suppressed.
I don't understand the N/S asymmetry. We assume the south is the same as the north, i.e., we don't even do it in the south.
Tim
On Tue, May 19, 2020 at 4:07 PM gmillward notifications@github.com wrote:
OK, here, for a given longitude slice (in this case mp=73), are the points in the ExB transport q interpolation that have not solved properly and left problematic factors (greater than 1). All below 200km, more towards the equator, and all in the Northern hemisphere:
[image: incorrect_q_interp_locations] < https://user-images.githubusercontent.com/24465054/82383270-b49ff400-99ea-11ea-856a-0c47ddb5ce21.png
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/NOAA-SWPC/IPE/issues/30#issuecomment-631109131, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AH5BFOA43PXMZ2EY4WROP53RSL7IDANCNFSM4MUJGS6A
.
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/NOAA-SWPC/IPE/issues/30#issuecomment-631532074, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF2U5HTJX2IT2UGQPZ3Z37DRSPWMPANCNFSM4MUJGS6A .
Dr. George Millward Cooperative Institute for Research in Environmental Sciences (CIRES) University of Colorado, Boulder
Visiting scientist at NOAA Space Weather Prediction Center Boulder, CO
fixed the problem ^^ - now any ExB convection that tracks to points lower than 90km just come from that 90km, which 'should' work fine. Just tested with a 2 day run of the 20150316 storm - no NaNs anywhere so hopeful the results will look good.
Excellent. Great job George. Tim
On Thu, May 21, 2020 at 8:33 AM gmillward notifications@github.com wrote:
fixed the problem ^^ - now any ExB convection that tracks to points lower than 90km just come from that 90km, which 'should' work fine. Just tested with a 2 day run of the 20150316 storm - no NaNs anywhere so hopeful the results will look good.
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/NOAA-SWPC/IPE/issues/30#issuecomment-632119999, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5BFOHLQYWP2OMWFKMQIH3RSU3SPANCNFSM4MUJGS6A .
Just need to look at the results - hopefully, all of that 'mess' around 200km and below will now have gone
The 200km problem has been resolved with the latest transport code. See details on PR 34. https://github.com/NOAA-SWPC/GSMWAM-IPE/pull/34
In order to understand the impact of 200km constrain on the model result, I have done two different runs. The default one is the current version of WAM-IPE while the other one results with removing the line of code.
All the results are the WAM-IPE simulation from 2013031600 to 2013031612 with time-varying drivers. This runs with self-consistent electrodynamics with Weimer 2005 at high-latitudes, the aurora is turned on, no plasma depletion is included.
It looks like everything behaves much better without the constrain. Also, the nighttime density becomes much larger when the constrain is taken away.
TEC: Default (with 200km constrain)![TEC_default](https://user-images.githubusercontent.com/22968399/80653469-7fc9fe00-8a37-11ea-9db6-573686b0a7e0.png)
Modified (no 200km constrain)![TEC_no200km](https://user-images.githubusercontent.com/22968399/80653484-88bacf80-8a37-11ea-8388-88282d9c91d8.png)
NE at 150km: Default (with 200km constrain)![NE150km_default](https://user-images.githubusercontent.com/22968399/80653496-93756480-8a37-11ea-8add-cb87f92866d7.png)
Modified (no 200km constrain)![NE150km_no200km](https://user-images.githubusercontent.com/22968399/80653523-a12aea00-8a37-11ea-9fb1-b42e7cf98c42.png)
O+ density at 0316 6UT Default (with 200km constrain)![oplus_201303160600_default](https://user-images.githubusercontent.com/22968399/80654090-c409ce00-8a38-11ea-8313-e2f85a7f48e9.png)
Modified (no 200km constrain)![oplus_201303160600_no200km](https://user-images.githubusercontent.com/22968399/80654107-ce2bcc80-8a38-11ea-9f2f-2978dc5e5c22.png)
Note that the scales are different in each plot!
O+ velocity at 0316 6UT Default (with 200km constrain)![voplus_201303160600_default](https://user-images.githubusercontent.com/22968399/80654144-e0a60600-8a38-11ea-8146-ebae662cf6f2.png)
Modified (no 200km constrain)![voplus_201303160600_no200km](https://user-images.githubusercontent.com/22968399/80654163-e865aa80-8a38-11ea-86f0-3f4d90702fc8.png)
O+ density at 0316 12UT Default (with 200km constrain)![oplus_201303161200_default](https://user-images.githubusercontent.com/22968399/80654192-f87d8a00-8a38-11ea-9840-ca2e5bc85690.png)
Modified (no 200km constrain)![oplus_201303161200_no200km](https://user-images.githubusercontent.com/22968399/80654203-fddad480-8a38-11ea-8b26-f19f14f01393.png)
O+ velocity at 0316 12UT Default (with 200km constrain)![voplus_201303161200_default](https://user-images.githubusercontent.com/22968399/80654232-092e0000-8a39-11ea-8eba-70eef3707c89.png)
Modified (no 200km constrain)![voplus_201303161200_no200km](https://user-images.githubusercontent.com/22968399/80654243-0e8b4a80-8a39-11ea-99e5-d9defdcb9461.png)