Time series plots to run for every cycle

[rtodling]

How do the time series plots work?

It seems to me that, when it comes to what shows in the FP (radmom standard page), the plots have a latency of a few days - 4 to 5. I think, as I monitoring tool, RadMom has to run every cycle. Now the question is how to get it to be efficient.

I don't know the inner guts of RadMom - but I hope some of you know enough to comment and make suggestions so we can get this to do what I propose.

I get the impression that every time Radmom runs, it starts afresh - it regenerates all the stats it needs without using calculates done previously. That makes it inefficient particularly because it needs to re-read the data from the original files.

So is there a way to change RadMom so it saves what it needs to produce the plots for one time in such a way that plots for the next time only need to generate the stats corresponding to that new time, and now everything else back in the past? This would allow to have the sliding window of the plots controlled by the plotting programs themselves, not the data files (cycles).

Joe: do you know enough of the inner guts of RadMom to work of this? Or do you need someone else who might know more (Meta? Or Jianjun?)

[gmao-jjin3]

Ricardo, I don't know how it works in the FP. In my practice, I make a tarball of the stats files after making those figures, so that I can check and avoid re-generating those stats files next time.

[gmao-msienkie]

Ricardo, I think that Mark is taking care of the operational software and generation of Pyradmon plots.

It is possible to run pyradmon using a directory with only diag.txt files previously generated. I just verified this (unwittingly) by rerunning some plots I had created last week, in order to test the legend code that Jianjun contributed.

It looks like the time series pyradmon plotting is triggered in the ops version of the 'fvarchive' script. See /home/dao_ops/f5294_fp/run/bin/fvarchive.j line 635. The most recent plots at https://gmao.gsfc.nasa.gov/forecasts/radmon_diagnostics.php are from 20220610-20220908 - although from looking at the script in /home/dao_ops/pyradmon/scripts/radmon-f5294_fp.20220610.20220908.rc it looks like the plotting was set to run from 20220908 00z.

[gmao-msienkie]

Joe is on leave until October 14. So changing his scripts would need to wait on his return. It could be possible to just create the diagbin2txt files each synoptic time on the fly, then use a copy of one of Joe's bin2img job scripts and comment out the bin2txt.pl call then use a template to fill in start date and end date and run that to get the plots. (Yes, I have done this before when repeatedly running the plots to check a change I made in the code.) I don't know about if the time series plots really need to be done each synoptic time - it is not easy to see changes right at the end of the time series and I would have thought once a day (which we have now) would suffice. I can see doing the map plots at each synoptic time as it might be easier to see a glitch on those plots.

[gmao-msienkie]

It could be possible to just create the diagbin2txt files each synoptic time on the fly

I had completely forgotten about setting the GSI_DIAG2TXT variable in the FVDAS_Run_Config to automatically generate the diagbin2txt files for each cycle... I noticed my comment on one of the other 'issues' in this repository.

[gmao-msienkie]

Note that it is not enough to set GSI_DIAG2TXT to '1' in the run config or job script, it needs to have the command (with optional arguments) to execute. E.g. "$FVROOT/bin/gsidiag_bin2txt -passivebc -npred 12"