Custom integration time and exact start times with make_grid

[billetd]

Hi there, I'm wondering if what I'm trying to do is possible or if my understanding of some make_grid functionality is off.

I'm trying to create .grdmap files with a very specific integration time (in seconds). When I use the -i flag I can do this and the outputs produced are the correct time integration for the whole period I'm looking at, although the start time chosen for the first record seems a bit random. I've also tried -tl to ignore scan flags and -st to specify a start time, but neither seem to actually have an effect on the start time chosen by make_grid in this scenario (I'm also not sure I understand -tl too well).

For example: make_grid -vb -i 458 20140223.2001.00.pgr.fitacf > pgr.grdmap produces two records, both 458s in length, one after the other:

Storing:2014-2-23 20:7:58 20:15:36 pnts=14 (6)
Storing:2014-2-23 20:15:36 20:23:14 pnts=18 (6)

However, when I specify a start time: make_grid -vb -i 458 -st 20:09 20140223.2001.00.pgr.fitacf > pgr.grdmap or make_grid -vb -i 458 -st 20:04 20140223.2001.00.pgr.fitacf > pgr.grdmap the records are the same as without the -st flag. I can mess around with the integration and start times, but I can't find a pattern in when the records seem to be starting whilst using an unusual integration time.

Is there a way to force the records exactly to the start time I specify whilst keeping my unusual integration time? I understand the answer likely involves cutting up or dropping scans, but I'm not sure how this works myself.

Dan

[egthomas]

Hi Dan - You're absolutely correct that the timing logic of make_grid is quite convoluted. I don't know the answer to your question right away but I can try to point you in the direction of where to look. The start and end times of the grid records are determined in GridTableMap of gtable.c:

https://github.com/SuperDARN/rst/blob/master/codebase/superdarn/src.lib/tk/gtable.2.0/src/gtable.c#L444-L472

which is called by make_grid here:

https://github.com/SuperDARN/rst/blob/master/codebase/superdarn/src.bin/tk/tool/make_grid.2.0/make_grid.c#L865

I believe the -i flag controls the duration of the record written to the grid file, while the -tl flag comes into play earlier in make_grid when deciding which FITACF data to load into the RadarScan structure for purposes of median filtering or mapping onto the grid.

[billetd]

Thanks Evan,

It seems like the culprit might be in line 469 of gtable.c: ptr->st_time=tlen*( (int) (tm/tlen)); which under normal circumstances will drop the first scan. Changing this line to force the input start time from make_grid seems to remedy the problem: ptr->st_time=scan->st_time; and output records now start at the specified time with integration times given by -i.

My main worry now is that the original line would have skipped out the first scan for a reason, and now I might be changing something for the worse without really understanding what's going on.

[egthomas]

@billetd - glad you were able to track this down. To be honest I won't have much time to look into this over the next few weeks but we can discuss it further when I visit Lancaster next month.

[billetd]

Just to update on this issue from your visit Evan, the solution posted in my previous comment seems to solve the issue and I haven't run into any compatibility issues yet. Unless this breaks anything else which I've not encountered yet then this issue can be closed when implemented.

[egthomas]

Hi Dan - thanks for following up on this. I believe I've made the change you proposed in both the C and IDL versions of make_grid here: 82fa757 and added it to pull request #133.

I tested it out using your original example from this issue and it seems to be working fine to me.

[egthomas]

Fixed in #133.