Closed jpnIII closed 8 months ago
jpnIII
commented
8 months ago Another related question: In G2G version 1.2.b, --list-products-all returns the information below.
Is there a way I can know where these products are actually defined within G2G? Thank you again for your help!
*true_color_night
cimss_cloud_type cimss_cloud_type_raw cimss_green cimss_green_sunz cimss_green_sunz_rayleigh cimss_true_color *cimss_true_color_sunz . . .
jpnIII
commented
8 months ago Another question cropped up: If I accidentally have a duplicate product name (but with different product characteristics from the core definition) defined within composites/abi.yaml, do you know which product will take precedence within G2G (core product or my product)? In other words, which one will get displayed?
jpnIII
commented
8 months ago And an idea, which may or may not be practical to implement: What about having a utility script that would run through all available products, and warn the user about any duplicates? Then the user could rename some things to fix the problem(s).
Now I'll shut up...
djhoese
commented
8 months ago because it appears that many G2G composite products are actually coded well within the bowels of G2G, in places I am not familiar with.
This is a constant issue I keep bumping into. I could rewrite things to try to duplicate/replicate the low-level configs (ex. those in Satpy) and make them visible to the user and include Geo2Grid's customizations as well, but all in one high-level directory (GEO2GRID_HOME/etc/). But I'm worried this would be overwhelming and confusing to users as there are a lot of composites and enhancements in Satpy. It also isn't risk free and could still cause odd conflicts with Satpy's builtin directories.
On a fresh install of G2G, doing --list-products-all might be the best bet for detecting if the custom composites you want to add conflict with any Satpy or Geo2Grid builtins.
do you know which product will take precedence within G2G (core product or my product)? In other words, which one will get displayed?
The ones defined via the GEO2GRID_HOME/etc directory should always have priority. Even if they have the same name. There may be small cases where if you don't overwrite every piece of the definition in Satpy in your custom definition then that piece would still be applied to your definition. For example, if a composite in Satpy has "optional_prerequisites" and you overwrite this composite with your own of the same name but don't redefine optional_prerequisites: [] (meaning no optionals) then Satpy's definition's optional_prerequisites will still be used. This is what I remember at least, but it also rarely happens. Bottom line is make sure you define the regular prerequisites and a standard_name and you're probably fine.
Is there a way I can know where these products are actually defined within G2G?
If it is under the "Non-standard Satpy Products" then it is definitely defined in Satpy which is in that "hidden" python_runtime/lib/pythonX.Y/site-packages/satpy/... directory and should probably not be touched. If it is a custom user product then you added it and should know where that is. Otherwise any other products could be split between Geo2Grid's definitions or Satpy. Most of the time though Geo2Grid doesn't define its own composites and instead uses Satpy's definitions.
What about having a utility script that would run through all available products, and warn the user about any duplicates? Then the user could rename some things to fix the problem(s).
I don't dislike this idea, but I'm not sure it would be a priority for our funding. I'd also like to work on (and I think I've finished it already) rewriting how the YAML files are placed in the Geo2Grid tarball so that they are always copies of the versions in Geo2Grid's builtin configs and Satpy's builtin configs. That way the user modifying things never breaks the original copies of these YAML files.
kathys
commented
8 months ago @jpnIII Hi Jim. First I want to thank you for making the packages better. You have been an invaluable source for testing and feedback. I just want us all to keep in mind the main G2G/P2G use case. Which is to make it easy for users to create high quality images from direct broadcast data, especially those that are not technically savvy. Those who want to delve deeper and do more complicated things with imagery can always use Satpy. We will never replicate all of the functionality that is available in that library.
jpnIII
commented
8 months ago Thank you, Kathy!
Sincerely,
Jim
From: kathys @.> Sent: Wednesday, January 17, 2024 9:48 AM To: ssec/polar2grid @.> Cc: James P. Nelson @.>; Mention @.> Subject: Re: [ssec/polar2grid] Some confusion over where various Geo2Grid composite products are defined? (Issue #675)
@jpnIIIhttps://github.com/jpnIII Hi Jim. First I want to thank you for making the packages better. You have been an invaluable source for testing and feedback. I just want us all to keep in mind the main G2G/P2G use case. Which is to make it easy for users to create high quality images from direct broadcast data, especially those that are not technically savvy. Those who want to delve deeper and do more complicated things with imagery can always use Satpy. We will never replicate all of the functionality that is available in that library.
— Reply to this email directly, view it on GitHubhttps://github.com/ssec/polar2grid/issues/675#issuecomment-1896086341, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AARELF5VJ6WURAKYHX3G3QLYO7XBFAVCNFSM6AAAAABB54Q3F2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJWGA4DMMZUGE. You are receiving this because you were mentioned.Message ID: @.***>
jpnIII
commented
8 months ago Hi Dave – Thank you for your detailed replies! I know I usually just flail about, and I appreciate your ongoing patience with my inquires. ☺ Just think of me as the lowest common denominator of users…
Take care.
Sincerely,
Jim
From: David Hoese @.> Sent: Wednesday, January 17, 2024 8:57 AM To: ssec/polar2grid @.> Cc: James P. Nelson @.>; Author @.> Subject: Re: [ssec/polar2grid] Some confusion over where various Geo2Grid composite products are defined? (Issue #675)
because it appears that many G2G composite products are actually coded well within the bowels of G2G, in places I am not familiar with.
This is a constant issue I keep bumping into. I could rewrite things to try to duplicate/replicate the low-level configs (ex. those in Satpy) and make them visible to the user and include Geo2Grid's customizations as well, but all in one high-level directory (GEO2GRID_HOME/etc/). But I'm worried this would be overwhelming and confusing to users as there are a lot of composites and enhancements in Satpy. It also isn't risk free and could still cause odd conflicts with Satpy's builtin directories.
On a fresh install of G2G, doing --list-products-all might be the best bet for detecting if the custom composites you want to add conflict with any Satpy or Geo2Grid builtins.
do you know which product will take precedence within G2G (core product or my product)? In other words, which one will get displayed?
The ones defined via the GEO2GRID_HOME/etc directory should always have priority. Even if they have the same name. There may be small cases where if you don't overwrite every piece of the definition in Satpy in your custom definition then that piece would still be applied to your definition. For example, if a composite in Satpy has "optional_prerequisites" and you overwrite this composite with your own of the same name but don't redefine optional_prerequisites: [] (meaning no optionals) then Satpy's definition's optional_prerequisites will still be used. This is what I remember at least, but it also rarely happens. Bottom line is make sure you define the regular prerequisites and a standard_name and you're probably fine.
Is there a way I can know where these products are actually defined within G2G?
If it is under the "Non-standard Satpy Products" then it is definitely defined in Satpy which is in that "hidden" python_runtime/lib/pythonX.Y/site-packages/satpy/... directory and should probably not be touched. If it is a custom user product then you added it and should know where that is. Otherwise any other products could be split between Geo2Grid's definitions or Satpy. Most of the time though Geo2Grid doesn't define its own composites and instead uses Satpy's definitions.
What about having a utility script that would run through all available products, and warn the user about any duplicates? Then the user could rename some things to fix the problem(s).
I don't dislike this idea, but I'm not sure it would be a priority for our funding. I'd also like to work on (and I think I've finished it already) rewriting how the YAML files are placed in the Geo2Grid tarball so that they are always copies of the versions in Geo2Grid's builtin configs and Satpy's builtin configs. That way the user modifying things never breaks the original copies of these YAML files.
— Reply to this email directly, view it on GitHubhttps://github.com/ssec/polar2grid/issues/675#issuecomment-1895982183, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AARELFZ2VJKXLIXQZFXOSSLYO7RCTAVCNFSM6AAAAABB54Q3F2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJVHE4DEMJYGM. You are receiving this because you authored the thread.Message ID: @.***>
Hello. My apologies if this has been asked before, but I am concerned about adding new Geo2Grid composite image products, and would appreciate any advice. We (mainly Tim Schmit, with a little assistance from me) have done this before, but I am always confused in general when I need to do it again. The latest G2G version we have is 1.2.b, from October 2023. The following 3 files appear to be the files I should be dealing with when adding a new G2G product, and I am thinking that composites/abi.yaml is the most likely location for a new product definition. Any enhancement(s) required could be added to either the enhancements/abi.yaml or enhancements/generic.yaml file. This is supported by: https://www.ssec.wisc.edu/software/geo2grid/compositors.html
geo2grid_v_1_2_b/geo2grid-swbundle-20231006-171312/etc/polar2grid/composites/abi.yaml geo2grid_v_1_2_b/geo2grid-swbundle-20231006-171312/etc/polar2grid/enhancements/abi.yaml geo2grid_v_1_2_b/geo2grid-swbundle-20231006-171312/etc/polar2grid/enhancements/generic.yaml
However, there are many new products released with 1.2.b, and I imagine others will come along in the future. I worry about accidentally redefining an already-existing product, because it appears that many G2G composite products are actually coded well within the bowels of G2G, in places I am not familiar with.
1.) Would you say that my first check of already-existing products should be gotten by running geo2grid.sh using either of the following -p flags? Then I could make sure that whatever I called a new product, it would not have any counterpart in either of these product lists:
That's enough for now, I think. I'll see where I stand after I read/think about your reply.
Thank you!
Sincerely,
Jim