airavata-courses / VignyaanDwaarNirman

Micro-service Architecture based Radar Data Visualization Web Application. Tools and Technologies: Jetstream, Kubernetes, Docker, RabbitMQ, Jenkins, RESTful, JAVA, Node JS, MongoDB
http://149.165.168.66:30012
0 stars 2 forks source link

Develop bottom-level microservices for data-manipulation #2

Closed AnshulV98 closed 4 years ago

AnshulV98 commented 4 years ago

Developing data-retrieval/model execution microservices using Python

devanshjain14 commented 4 years ago

I have tried using pyart library to map NEXRAD data. But I am now stuck at a time where I get this error:

2020/01/30/KIND/KIND
<Key: noaa-nexrad-level2,2020/01/30/KIND/KIND20200130_235132_V06_MDM>
/tmp/tmp_qkjbe05
after s3
Traceback (most recent call last):
  File "test.py", line 80, in <module>
    radar = pyart.io.read_nexrad_archive(localfile.name)
  File "/home/devansh/anaconda3/lib/python3.7/site-packages/pyart/io/nexrad_archive.py", line 118, in read_nexrad_archive
    nfile = NEXRADLevel2File(prepare_for_read(filename))
  File "/home/devansh/anaconda3/lib/python3.7/site-packages/pyart/io/nexrad_level2.py", line 154, in __init__
    raise IOError('unknown compression record')
OSError: unknown compression record
devanshjain14 commented 4 years ago

I am now trying to use MetPy library. I have shared the details with Viral. Using metpy on the NEXRAD data, we will be able to generate plots for the following attributes,

300 hPa heights and winds 500 hPa heights and absolute vorticity Surface temperatures Precipitable water

for any particular day in the past.

devanshjain14 commented 4 years ago

This is another option that we have, we can also use the NEXRAD Level 2 data with,

https://unidata.github.io/MetPy/latest/examples/formats/NEXRAD_Level_2_File.html#sphx-glr-examples-formats-nexrad-level-2-file-py

but this is again followed by the same error


Traceback (most recent call last):
  File "test3.py", line 20, in <module>
    ref_hdr = f.sweeps[sweep][0][4][b'REF'][0]
IndexError: tuple index out of range
devanshjain14 commented 4 years ago

I was successfully able to map reflexivity and velocity scans for any radar station and any date using pyart and metpy library. I was also able to save all the graphs to png file.

1

2

ViralPrajapati commented 4 years ago

I am also trying to map reflexivity and velocity using pyart and metpy but there seems to be some error in the library since I am able to generate the plot but can't impose it on the respective state map.

devanshjain14 commented 4 years ago

Can you post your error here? I think, I had one issue, which I could resolve easily.

AnshulV98 commented 4 years ago

As we are done with the initial build of the microservices closing this issue, will open new issues with newer requirements in the future