EHN: Fan Plot using User Input Data

[carleyjmartin]

Scope

This pull request hopes to give an option to anyone who needs more flexibility to the fan plots. You can essentially plot whatever data you want at any time for any radar, given you can make the data be in the right format.

You will need to make a 2D list of data that matches the FOV of the target radar, where no data is given as NaN values, data is given as a float, and an array of 0,1 in the same dimensions will act as groundscatter with the dimensions [ranges, beams].

issue: to close #387

Approval

Number of approvals: 2

Test

matplotlib version: 3.9.1 Note testers: please indicate what version of matplotlib you are using

Note: If you come across an error that traces back to numpy or pydarnio, please downgrade numpy, it is out of scope of pyDARN to fix this. There will also likely be a merge conflict, so be aware.

import matplotlib.pyplot as plt
import pydarn
import datetime as dt
import numpy as np

# ==========================================================================
# Test completely made up data for SAS
# Made up data in format [75, 16]
data_array = [(np.ones(16) * (x - 36) * 10).tolist() for x in range(75)]
# Add some areas with no data
for i in range(65,70):
    data_array[i] = (np.empty(16) * np.nan).tolist()

# Made up groundscatter boolean array
data_groundscatter = [(np.zeros(16)).tolist() for x in range(75)]
for i in range(30,40):
    data_groundscatter[i] = (np.ones(16)).tolist()

stid = 5
data_datetime = dt.datetime(2024,1,1,0,0)

pydarn.Fan.plot_fan_input(data_array=data_array,
                          data_datetime=data_datetime,
                          stid=stid,
                          data_groundscatter = data_groundscatter,
                          data_parameter='v',
                          zmin=-400,zmax=400, lowlat=50, coastline=True)
plt.show()

# ==========================================================================
# Test completely made up data with bigger fan size for BKS
# Made up data in format [110, 24]
data_array = [(np.ones(24) * (x - 36) * 10).tolist() for x in range(110)]
# Add some areas with no data
for i in range(65,70):
    data_array[i] = (np.empty(24) * np.nan).tolist()

# Made up groundscatter boolean array
data_groundscatter = [(np.zeros(24)).tolist() for x in range(110)]
for i in range(30,40):
    data_groundscatter[i] = (np.ones(24)).tolist()

stid = 33
data_datetime = dt.datetime(2024,1,1,0,0)

pydarn.Fan.plot_fan_input(data_array=data_array,
                          data_datetime=data_datetime,
                          stid=stid,
                          data_groundscatter = data_groundscatter,
                          data_parameter='v',
                          zmin=-400,zmax=400, lowlat=30, coastline=True)
plt.show()

# ==========================================================================
# Test completely made up data with non-standard fan size?
# Made up data in format [110, 24]
data_array = [(np.ones(24) * (x - 36) * 10).tolist() for x in range(110)]
# Add some areas with no data
for i in range(65,70):
    data_array[i] = (np.empty(24) * np.nan).tolist()

# Made up groundscatter boolean array
data_groundscatter = [(np.zeros(24)).tolist() for x in range(110)]
for i in range(30,40):
    data_groundscatter[i] = (np.ones(24)).tolist()

stid = 5
data_datetime = dt.datetime(2024,1,1,0,0)

pydarn.Fan.plot_fan_input(data_array=data_array,
                          data_datetime=data_datetime,
                          stid=stid,
                          data_groundscatter = data_groundscatter,
                          data_parameter='v',
                          zmin=-400,zmax=400, lowlat=50, coastline=True)
plt.show()

# ==========================================================================
# Test with camping beam real data from FHE
file = "/Users/carley/Documents/data/20231122.0402.00.fhe.fitacf"
SDarn_read = pydarn.SuperDARNRead(file)
fitacf_data = SDarn_read.read_fitacf()

# find the data you want to plot int eh file using a dateitme
scan_time = dt.datetime(2023,11,22,4,16)
matching_records = pydarn.find_records_by_datetime(fitacf_data, scan_time,
                                                   dt.timedelta(seconds=120))

# Make empty data array to correct size
data_array = [(np.empty(22) * np.nan).tolist() for x in range(110)]
data_groundscatter = [(np.zeros(22)).tolist() for x in range(110)]

# Do fun stuff to the camped beam (THIS IS JUST AN EXAMPLE OF WHAT CAN BE DONE
# DONT @ ME FOR THE SCIENTIFIC PURPOSE OF DOING THIS SPECIFICALLY TO THE DATA)
camped_beam = 10
camped_count = 0

for i in range(len(matching_records)):
    for g, gate in enumerate(matching_records[i]['slist']):
        if matching_records[i]['bmnum'] != camped_beam:
            data_array[gate][matching_records[i]['bmnum']] =\
                matching_records[i]['v'][g]
            data_groundscatter[gate][matching_records[i]['bmnum']] =\
                matching_records[i]['gflg'][g]
        else:
            if matching_records[i]['gflg'][g] == 0:
                if np.isnan(data_array[gate][matching_records[i]['bmnum']]):
                    data_array[gate][matching_records[i]['bmnum']] =\
                        matching_records[i]['v'][g]
                else:
                    data_array[gate][matching_records[i]['bmnum']] =\
                        data_array[gate][matching_records[i]['bmnum']] +\
                        matching_records[i]['v'][g]
    if matching_records[i]['bmnum'] != camped_beam:
        camped_count+=1

for g in range(110):
    data_array[g][camped_beam] = data_array[g][camped_beam] / camped_count
# This is ignoring groundscatter for beam 10, and assuming that it is okay to
# average the data collected over the two minutes, I would probably personally
# choose one of the records to plot rather than averaging all the data in it
# but it's really up to the user how they present and interpret their data
# please don't blame me haha! There's also probably a better way to make this
# data array, I'm just playing with it.

# plot using new input
pydarn.Fan.plot_fan_input(data_array=data_array,
                          data_datetime=scan_time,
                          stid=fitacf_data[0]['stid'],
                          data_groundscatter = data_groundscatter,
                          data_parameter='v',
                          zmin=-600,zmax=600, lowlat=30, coastline=True)
plt.show()

# Traditional plot to compare to, with all records in camped beam overplotted
rtn = pydarn.Fan.plot_fan(fitacf_data, scan_index=scan_time, groundscatter=True,
                          coastline=True, lowlat=40, zmin=-600, zmax=600,
                          tolerance=dt.timedelta(seconds=120))
plt.show()

Lots of plots, not much science, mainly for examples!

[Doreban]

Tested on Windows 10 with Matplotlib version 3.9.1 and numpy version 1.26.4

I was able to recreate the same plots as shown by Carley using the test code provided. The plot_fan_input also worked for different projections. Additionally when a improperly formatted array was passed in as the data array, I got the following warning which seems appropriate: UserWarning: The data you have inputted to the method does not match the expected number of beams for this radar. The data will still plot, but the position of the extra beams may not be as expected.

Also able to plot FOVs alongside this new function.