NCAR / DART

Data Assimilation Research Testbed
https://dart.ucar.edu/
Apache License 2.0
189 stars 142 forks source link

Performance issue? obs_def gpsro key vs. obs_seq key #512

Open hkershaw-brown opened 1 year ago

hkershaw-brown commented 1 year ago

Describe the bug

Anderson found this when running obs_sequence_tool.

  1. List the steps someone needs to take to reproduce:
  1. What was the expected outcome? input obs_sequence.processed === output obs_sequence.processed

  2. What actually happened?
    The gpsroref keys are different, and gpsref keys ininput obs_sequence.processed do not start from 1.

when you call initialize_module for obs_def_gps_mod the keycount is set to 0. https://github.com/NCAR/DART/blob/1b76f3afa5978469d6a119710bb638c1c077ae20/observations/forward_operators/obs_def_gps_mod.f90#L134-L135

what gets passed in to write_obs_def is the key from the obs_sequence. Which is not the same as the gpsroref key.

obs_sequence_tool -> write_obs -> write_obs_def https://github.com/NCAR/DART/blob/1b76f3afa5978469d6a119710bb638c1c077ae20/assimilation_code/modules/observations/obs_sequence_mod.f90#L2509

/glade/scratch/hkershaw/DART/Bugs/obs_sequence_tool
grep gps obs_seq.out | sort -k2 -n |head  -n 1
gpsroref  646965

grep gps obs_seq.processed | sort -k2 -n | head  -n 1
gpsroref   81405

Error Message

No error, I think using the key means you allocate more memory than you need to in obs_def_gps_mod (and obs_def_rttov_mod which also used module data accessed by key). Also, tripped up Anderson when trying to compare two obs_sequences which is fair enough. I believe (not checked with Anderson's code) that a simple read/write you get the gpsro starting at 1.

sanity checked by changing GPSREF value (they are all zero all the time) that the obs keeps the gpsref data /glade/scratch/hkershaw/DART/Bugs/obs_sequence_tool

diff obs_seq.out.X obs_seq.out
10815868c10815868
<   0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  1.230000000000000E+000 GPSREF
---
>   0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000 GPSREF

Which model(s) are you working with?

its a obs sequence file from a cam-fv experiment.

Version of DART

Which version of DART are you using? v10.8.0

Have you modified the DART code?

No

Build information

Please describe:

  1. Cheyenne
  2. intel
hkershaw-brown commented 5 months ago

note Anderson's test code is here: https://github.com/NCAR/DART/compare/main...achauphan:DART:obs_seq_harness