Wavy Pattern in AVHRR CLAVRx reprojections

[kathys]

GINA reported a problem with the AVHRR CLAVRx P2G reprojections as displayed in AWIPS. They reported a couple of times where the displays showed a strange wavy pattern in the products. I downloaded the L1B files on one of the passes where they had seen this behavior and I was able to reproduce their results when creating GeoTIFFs. Dave said this is the pattern that results from the EWA resampling in FORNAV when it is not looking far enough for valid pixels. We will need to investigate further.

Dave - bumi:/data/users/kathys/test_data/clavrx/gina

[djhoese]

@kathys Could you post some of the screenshots they sent?

[kathys]

There are also GeoTIFFs that I created included at the bumi site.

[djhoese]

The longitude array has a series of 0s at the top:

I'm not sure what I can do when the data Polar2Grid is provided has non-fill value coordinates that are wrong. Now if we wanted to say "oh this is the anti-meridian" then yeah sure, but there is clearly a series of 0s in the upper left of the longitude data that I don't think makes sense here. Or is that the prime meridian on the other side of the globe?

This was generated in python from the level2 file bumi:/data/users/kathys/test_data/clavrx/gina/output1/clavrx_hrpt_M01_20191129_2117_37353.l1b.level2.hdf. I used Satpy to load the longitude and latitude arrays and plotted them with matplotlib.

@wstraka3 I mentioned I'd mention you in whatever issue talked about this.

[djhoese]

Close, but it hasn't quite gone over the north pole yet. Here's the first 20 rows and columns of the latitude:

In [18]: lats[:20, :20]
Out[18]:
array([[        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [        nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan,         nan],
       [85.1109347 , 85.13840136, 85.16586802, 85.19058801, 85.21805467, 85.24552133, 85.27298799, 85.30045465, 85.3279213 , 85.35538796, 85.38010796, 85.40757461, 85.43504127, 85.45976127, 85.48722792, 85.51469458, 85.54216124, 85.56688123, 85.59434789, 85.61906788],
       [85.10544137, 85.13290803, 85.16037469, 85.18784135, 85.215308  , 85.24277466, 85.26749466, 85.29496131, 85.32242797, 85.34989463, 85.37461462, 85.40208128, 85.42954794, 85.4570146 , 85.48173459, 85.50920125, 85.53666791, 85.5613879 , 85.58885456, 85.61632122],
       [85.09994804, 85.1274147 , 85.15488136, 85.18234801, 85.20981467, 85.23728133, 85.26474799, 85.28946798, 85.31693464, 85.3444013 , 85.37186796, 85.39658795, 85.42405461, 85.45152127, 85.47898793, 85.50370792, 85.53117458, 85.55589457, 85.58336123, 85.61082789],
       [85.09720137, 85.12466803, 85.14938802, 85.17685468, 85.20432134, 85.231788  , 85.25925466, 85.28672132, 85.31144131, 85.33890797, 85.36637463, 85.39384129, 85.41856128, 85.44602794, 85.47349459, 85.49821459, 85.52568125, 85.5531479 , 85.5778679 , 85.60533456],
       [85.09170804, 85.1191747 , 85.14664136, 85.17410802, 85.19882801, 85.22629467, 85.25376133, 85.28122799, 85.30869464, 85.33341464, 85.3608813 , 85.38834795, 85.41581461, 85.4405346 , 85.46800126, 85.49272126, 85.52018791, 85.54765457, 85.57237457, 85.59984122],
       [85.08621471, 85.11368137, 85.14114803, 85.16861469, 85.19608134, 85.223548  , 85.248268  , 85.27573465, 85.30320131, 85.33066797, 85.35538796, 85.38285462, 85.41032128, 85.43504127, 85.46250793, 85.48997459, 85.51469458, 85.54216124, 85.56688123, 85.59434789],
       [85.08346804, 85.10818804, 85.1356547 , 85.16312135, 85.19058801, 85.21805467, 85.24552133, 85.27024132, 85.29770798, 85.32517464, 85.3526413 , 85.37736129, 85.40482795, 85.43229461, 85.4570146 , 85.48448126, 85.50920125, 85.53666791, 85.56413457, 85.58885456],
       [85.07797471, 85.10544137, 85.13290803, 85.15762802, 85.18509468, 85.21256134, 85.240028  , 85.26749466, 85.29221465, 85.31968131, 85.34714797, 85.37186796, 85.39933462, 85.42680128, 85.45152127, 85.47898793, 85.50645459, 85.53117458, 85.55864124, 85.58336123],
       [85.07248138, 85.09994804, 85.1274147 , 85.15488136, 85.18234801, 85.20706801, 85.23453467, 85.26200132, 85.28946798, 85.31418798, 85.34165463, 85.36912129, 85.39384129, 85.42130794, 85.44602794, 85.47349459, 85.50096125, 85.52568125, 85.5531479 , 85.5778679 ],
       [85.06698805, 85.09445471, 85.12192137, 85.14938802, 85.17685468, 85.20432134, 85.22904133, 85.25650799, 85.28397465, 85.30869464, 85.3361613 , 85.36362796, 85.38834795, 85.41581461, 85.44328127, 85.46800126, 85.49546792, 85.52018791, 85.54765457, 85.57237457]])

[djhoese]

Something else about the anti-meridian is making EWA resampling not happy though. If I force all longitude values greater than zero to -180, I still get some odd looking images.

Edit: Maybe it is just best to switch AVHRR processing to use nearest neighbor instead of trying to make EWA do something it wasn't designed to do (processing non-scan based data as one giant scanline).

[djhoese]

So good news, I did a little playing around and I may have a reasonable solution for this. Basically I take the changes I made in #238 and expand it. Right now I do a fallback of search distance (my understanding) of the distance_max parameter which is 1.0 this processing. I did a little test and increased this fallback number to 8.0 and the image no longer has the missing pixels/wavy pattern. I'll need to walk through this code tomorrow and decide what the best fallback value really is.

[kathys]

Just as a final note on this issue:

I spoke with Andy about CLAVRx AVHRR products. I told him what I understood to be the problem. That although there were valid Lat/Lon lines of data in the L1B AVHRR file, the CLAVRx files contained NaNs or some other invalid data in them. I told him that my understanding was that this has to do with something unrelated to geolocation. He told me that he would take an action to look at this, and if this is what is happening, then he will make sure this behavior is changed.