Added the option to write the measurement errors to the secondary output file

[garethethomas]

It can be quite useful to see what values for measurement error are actually being used in the retrieval, so this adds the option to request the diagonals of the Sy covariance matrix be output to the seondary.nc file.

[adamcpovey]

After 20 minutes I worked out how to push changes to your branch and update this thing! @garethethomas please check that those still address the problems you were having (I went looking for the causes of the missed status flag rather than circumvent the whole system).

Further, I am beginning to question a bunch of stuff in here:

• Why are we allocating SPixel arrays each time when they just get copied into local arrays in invert_marquadt()? We probably save a bunch of processing time by not bothering to reallocate.
• Given the output arrays are initialised to fill values, why do we call prepare_output_XXX() for skipped retrievals? Surely doing nothing outputs what we want (namely nothing)?

[garethethomas]

I agree with your inclusion of SPixel%spixel_y_to_ctrl_y_index - from what I've seen it isn't generally necessary (it's a one-to-one match), but I guess this might not always be the case! Regarding your other change - it should be ok, so long as SPixel%Ny = 0 means that the do loop at line 338 of prepare_output_secondary.F90 doesn't get executed. Is that how Fortran works?

I was also surprised at the fact that all of the SPixel arrays are reallocated for every pixel - like you say, it seems rather inefficient. "Fixing" this would be a big job though!

[adamcpovey]

SPixel%spixel_y_to_ctrl_y_index is mostly important for the cloud retrieval, as one of the effective radius channels is always missing when multiple are present.

To my knowledge, Fortran checks the validity of the for condition at the start of the loop so do i=1,0 should immediately terminate.

[garethethomas]

Ok, finally managed to actually test this as-is. All seems good, so I'm happy for it to be merged, if you are Adam.

If you're wondering why it took quite so long, do you know if it would be possible/easy to make the "read_ctrl" provide more feedback than flex scanner jammed if it runs into an unknown driver file line?

[adamcpovey]

I never managed to work out how to get sensible errors out of flex. You can get it to at least throw a line number and slightly more information (almost always "syntax", but others can happen) by uncommenting the last two lines of config/arch.inc