Closed DAndrewA closed 1 year ago
I've made some further changes to the code in ipx.Read
and have been able to succesfully load in an ATL09 file to an xarray database.
The code I've changed within the file is:
# line 407, in function _add_vars_to_ds
else:
import re
print(f'NEW DEBUG: ipx.READ: _add_vars_to_ds: {grp_path=}')
'''ORIGINAL CODE, DO NOT DELETE
gt_str = re.match(r"gt[1-3]['r','l']", grp_path).group()
spot = is2ref.gt2spot(gt_str, is2ds.sc_orient.values[0])
'''
profile_str = re.match(r'profile_[1-3]', grp_path).group()
#rate_str = re.match(r"*['high','low']_rate", grp_path).group()
print(f'NEW DEBUG: ipx.READ: _add_vars_to_ds: {profile_str=}')
#print(f'NEW DEBUG: ipx.READ: _add_vars_to_ds: {rate_str=}')
gt_str=profile_str
spot = int(profile_str[-1])
# add a test for the new function (called here)!
grp_spec_vars = [...
By completely bypassing the function is2ref.gt2spot()
I've been able to get a database that extracts the values I'm looking for. The output database has the structure:
It still uses the gt
and spot
notation used in the ATL06 data formats, but the data itself seems to be valid.
My solution as-is would be incompatible with the ATL09 data (and possibly other data products).
Hello @DAndrewA! Thank you very much for this great bug report! We have also encountered this issue with ATL11, but I'm struggling to keep up with bug fixes and new development simultaneously, so the implementing a fix has been slow. There are currently the first portions of a fix for this particular issue in the atl11 branch, which is quite similar to the solution you shared (but tries to generalize to work for spot, profile, and pt cases). I'll be continuing to work on this later this week and next and am happy to ping you when we've got an updated release out. Would you consider reviewing a pull request with this fix in it? Thanks!
Hi @JessicaS11 , thanks for getting back to me! I've never reviewed a pull request before but I'd be more than happy to give it a go!
I've never reviewed a pull request before but I'd be more than happy to give it a go!
@DAndrewA Thanks for this offer! If you're new to the process or would like some help getting oriented around the GitHub pull request (PR) interface/pulling the branch to test locally, I'd be happy to arrange a call to get you going. For some high level info on reviewing code and why it's so critical and helpful, The Turing Way has some great resources. You can also check out some of the already merged PRs in this repo. In the meantime, I will send you an invite (which you will have to accept) to the icesat2py organization that icepyx lives in so that you have the right permissions to review.
Summary
The
ipx.Read.load()
method seems to expect profiles that match the regular expression formgt[1-3]['l','r']
. ATL09 data is labelled according to profiles that match the regular expressionprofile_[1-3]/['high','low']_rate
. This means that loading in variables from ATL09 data files isn't currently possible.Replication
I'm trying to load in ATL09 data for analysing clouds with. I've used the subsetting of downloaded granules, which can be replicated with the below code:
I was then able to use
.order_granules
and.download_granules
to download the data. I've accessed the data using the h5py module, and to the best of my knowledge, none of the data appears to be erroneous or missing.I wrote a simple script to test the loading functionality of icepyx, which goes as:
This produces an error message that reads as
I've identified that the error comes from the regular expression matching. In the ATL09 files, the data is labelled according to the regular expression format
profile_[1-3]/['high','low']_rate
rather thangt[1-3]['l','r']
. I tried changing this in the function, so that it read:However, this simply throws an error of the form