openradar / PyDDA

Pythonic multiple doppler code based off the 3D variational technique
http://openradarscience.org/PyDDA
Other
91 stars 44 forks source link

The w component obtained by inversion is not correct. #93

Closed chentryto1 closed 1 year ago

chentryto1 commented 1 year ago

When I use the function get dd wind _ field and set the parameter engine = ' tensorflow ', the w component obtained by inversion is abnormally large. When using engine = ' scipy ', I get the following tips :

d:\codetools\anaconda\home\envs\py38\lib\site-packages\pydda-1.3.0-py3.8.egg\pydda\cost_functions_cost_functions_numpy.py:215: RuntimeWarning: invalid value encountered in divide grad_w = grad_w / dz The gradient of the cost functions is nan Nfeval | Jvel | Jmass | Jsmooth | Jbg | Jvort | Jmodel | Jpoint | Max w
10| nan| nan| nan| 0.0000| 0.0000| 0.0000| 0.0000| nan The gradient of the cost functions is nan Nfeval | Jvel | Jmass | Jsmooth | Jbg | Jvort | Jmodel | Jpoint | Max w
20| nan| nan| nan| 0.0000| 0.0000| 0.0000| 0.0000| nan Applying low pass filter to wind field... Done! Time = 11.4

At this time, the w component obtained by inversion is particularly small.

chentryto1 commented 1 year ago

Now I update to pydda1.3.1, the engine = ' scipy ' error is not reported, but the engine = ' tensorflow ' problem still exists.

rcjackson commented 1 year ago

It sounds like there is a nan present in your input radar data that may be preventing the optimization loop from starting. One thing you should do is, for every input grid, make sure that the NaNs are masked with a statement like this: grid.fields['corrected_velocity']['data'] = np.ma.masked_invalid(grid.fields['corrected_velocity']['data'] )

PyDDA will exclude points that are masked, so masking invalid points in your inputs should get PyDDA to start the optimization loop.

chentryto1 commented 1 year ago

I checked my input data, very unfortunately, did not find this problem you said, maybe other problems. At the same time, I noticed the example you gave in this link :

https://openradarscience.org/PyDDA/source/auto_examples/plot_fun_with_constraints.html#sphx-glr-source-auto-examples-plot-fun-with-constraints-py

I found that your third and fourth figures did not retrieve the wind, I looked at the output of your program, but also there is no execution loop this situation, which seems unusual.

rcjackson commented 1 year ago

If you could provide the input data and settings you are using I could take a look at what is causing your retrieval to not start.

I will also take a look at the examples, chances are the constants in example 3 and 4 are not suitable for the retrieval. I will make a note of this in the documentation which will explain why PyDDA is not starting and the importance of choosing the correct weights.

Get Outlook for iOShttps://aka.ms/o0ukef


From: Chenlaoban @.> Sent: Tuesday, August 29, 2023 9:54:20 AM To: openradar/PyDDA @.> Cc: Jackson, Robert @.>; Comment @.> Subject: Re: [openradar/PyDDA] The w component obtained by inversion is not correct. (Issue #93)

I checked my input data, very unfortunately, did not find this problem you said, maybe other problems. At the same time, I noticed the example you gave in this link :

https://openradarscience.org/PyDDA/source/auto_examples/plot_fun_with_constraints.html#sphx-glr-source-auto-examples-plot-fun-with-constraints-py

I found that your third and fourth figures did not retrieve the wind, I looked at the output of your program, but also there is no execution loop this situation, which seems unusual.

— Reply to this email directly, view it on GitHubhttps://github.com/openradar/PyDDA/issues/93#issuecomment-1697605381, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFIQA5FGOBVTSUVGNQIBJMTXXX7BZANCNFSM6AAAAAA34NPJUQ. You are receiving this because you commented.Message ID: @.***>

chentryto1 commented 1 year ago

Thank you very much for your help on this, I provided the radar file that I used to retrieve the wind field. The parameters I used are as follows : Cx = 1e-4, Cy = 1e-4, Cz = 1e-4, Cm = 256, Co = 1e-2, engine = ' tensorflow ' Then I didn 't use the initial field and the background field

The w component of the wind field I retrieved is particularly large, and I was able to start the optimization loop.

radar.zip

rcjackson commented 1 year ago

Hello,

What is happening is that you are generating artificial updrafts at the lobe edges due to the lack of a background and zero initialization. Therefore, your background wind is zero outside the lobes which is causing artificially high updrafts. I would highly recommend using a sounding or model input field as an initialization and/or constraint then try running the retrieval again.

From: Chenlaoban @.> Date: Tuesday, August 29, 2023 at 10:57 PM To: openradar/PyDDA @.> Cc: Jackson, Robert @.>, Comment @.> Subject: Re: [openradar/PyDDA] The w component obtained by inversion is not correct. (Issue #93)

Thank you very much for your help on this, I provided the radar file that I used to retrieve the wind field. The parameters I used are as follows : Cx = 1e-4, Cy = 1e-4, Cz = 1e-4, Cm = 256, Co = 1e-2, engine = ' tensorflow ' Then I didn 't use the initial field and the background field

The w component of the wind field I retrieved is particularly large, and I was able to start the optimization loop.

radar.ziphttps://github.com/openradar/PyDDA/files/12471518/radar.zip

— Reply to this email directly, view it on GitHubhttps://github.com/openradar/PyDDA/issues/93#issuecomment-1698450509, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFIQA5DJ3WGWBZNJBMYVDD3XX22ZBANCNFSM6AAAAAA34NPJUQ. You are receiving this because you commented.Message ID: @.***>

chentryto1 commented 1 year ago

Oh, thank you for helping me find the problem and give me advice, I lack of theoretical knowledge of the retrieval of wind field, I will follow your advice to improve, thank you for providing this very useful tool