Closed joergklausen closed 3 years ago
joergklausen
commented
4 years ago Branch created at https://github.com/wmo-im/wmds/commit/b9cfcbf6bcbdba0a44fb57ac270698eda97351c2#diff-a066c50c20373fc15bcd759e58529c31867ae12caecaa7d36ad7627f5e689622. @tomkralidis, @ejwelton Please review proposal and confirm or comment.
tomkralidis
commented
4 years ago Thanks @joergklausen. Do deprecated entries have a lifecycle (i.e. will they be deleted? If yes, when)?
ejwelton
commented
4 years ago Folks, sorry I didn't get to this till now. I was swamped with work last week. I don't recall anyone asking me if its ok to deprecate GALION or GALIONNDACC. Removing the latter (GALIONNDACC) makes sense since NDACC is a GALION network. However, I would prefer to not remove GALION. If that were removed from the programs listing then there would be no way to search for GALION networks. An OSCAR user would have to know what programs were contributing to GALION. I find it odd that the reason to deprecate is that its considered a "coordination mechanism" and not a program? GAW itself is a coordination mechanism, and there are many entries in the programAffiliation code list that are not actual programs: GAW Contributing Networks, GAW other elements, globalAOD, etc .... these exist to provide search fields. SO removing them does not make much sense to me. What is driving this request?
joergklausen
commented
4 years ago Good question, I think we need more information by WMO on how this is handled in general. In this particular case, it is a correction and the term should not have entered the code list in the first place. In other cases, where a term has been used in the past, it cannot be simply removed without losing the history, but adding (deprecated) would indicate that it must not be used anymore.
Sent from my iPhone
On 17 Oct 2020, at 01:34, Tom Kralidis notifications@github.com wrote:
Thanks @joergklausenhttps://github.com/joergklausen. Do deprecated entries have a lifecycle (i.e. will they be deleted? If yes, when)?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/wmo-im/wmds/issues/189#issuecomment-710697024, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIFWXY5DSVR7U6VOU2YYF6TSLDKAZANCNFSM4QJSSE3A.
amilan17
commented
3 years ago Decision dependant on #199
joergklausen
commented
3 years ago @ejwelton @Netcheva GALION needs to be removed from the WMDR code list, because one cannot affiliate an observation with GALION directly. GALION could remain as a grouping in OSCAR/Surface if WMO agrees, so that one can still search for stations that are coordinated within GALION. This would mean, GALION needs to be listed directly under GAW, because it is not a GAW Contributing network.
It remains unclear to me how we should deal with NDACC in this regard. NDACC is one program and they see themselves as such. If the same NDACC is listed under GALION and under GAW Other Elements, then the consequence is that a search for GALION will also return all NDACC stations that have no lidar observations. The only solution to cleanly separate these parts of NDACC is to actually split it up into, say NDACC-main and NDACC-lidar. @ejwelton Could you engage with Jeannette to find a solution?
ejwelton
commented
3 years ago You can definitely assign an observation to GALION, I've done this in the XML files I was generating for MPLNET. Even while not done with that process yet, I've tested doing so and it works fine. So I don't understand that comment. As I've said, there are other "programs" listed in the programAffiliation code list that are not individual programs (like global AOD). I don't understand why there is such focus on GALION right now. I've identified many other much more pressing vocabulary / code list issues that have not been addressed yet.
Regarding NDACC: I don't see a problem here. Whoever is responsible for updating NDACC in OSCAR would just add GALION affiliation for their sites that have lidar. Just like I could add or remove GALION affiliation for any of my MPLNET sites if one of them did not want to be listed as GALION.
fstuerzl
commented
3 years ago In agreement with Jörg I would like to propose the following approach to deal with this problem:
Proposal Deprecate the entry GALION and add "(GALION)" to every Program that is coordinated by GALION. This leads to the following structure:
GAW >GAW Contributing networks
> GAW Other elements
...
Reason The programs that are coordinated by GALION are categorised as GAW Contributing networks (in case of ADNet, CIS-LiNet, EARLINET, LALINET and MPLNET) and as GAW Other elements (NDACC Lidar, notation: GALIONNDACC). Therefore the existing subgroup by the name "GALION" under "GAW Contributing networks" does not make sense, as it implies that GALIONNDACC is also a contributing network, which is not the case. It is however necessary that users are still able to find all GALION programs, even when the entry "GALION" is deprecated. Therefore the the information "(GALION)" should be added, so that users can still search for this expression.
Branch https://github.com/wmo-im/wmds/tree/Issue-%23189
View differences: https://github.com/wmo-im/wmds/commit/2268f3f82094c36b7f37be9c16c532057962c190#diff-178d0ba48282279e1a03f1d2dc6739a7ff8999a4f14acc531aa8144af5383986
ejwelton
commented
3 years ago I am not sure what these GitHub branches are, I cannot access those links. However, I am still opposed to the idea of deprecating GALION. It makes no sense to me, and given the many other serious problems and issues in OSCAR I don't understand why time is being wasted on this issue. To be clear, WMO, GAW and/or GALION do not coordinate any lidar network. Our networks existed prior to the creation of GALION, and will likely outlast it given attempts like this to remove it. We (the leads of the lidar networks), coordinate GALION! Not the other way around. It is improper to add the word (GALION) to our network names as that implies that we are some sort of WMO funded GAW project. We are independent networks that have agreed to collaborate with GAW, and that collaboration vehicle is GALION. If GALION is deprecated within OSCAR then the entire concept of GALION is erased. What is the point of it? As the GALION co-chair I'm 100% against this action. This whole action seems to have begun due to the GALIONNDACC entry. Why not just clean up NDACC in system and leave GALION as is.
joergklausen
commented
3 years ago Hi @ejwelton, my mandate here is to reflect the reality and correct previous mistakes in the official WIGOS code tables. How OSCAR/Surface can deal with this is a related but not the same issue. According to WMO (Oksana), there is no written MoU or LoA that would authorize a code list entry GALION in the program/network codelist - if you want to dispute this, I refer you to WMO. According to your own reasoning, the lidar networks presently listed under GALION are not part of an official GALION network. Therefore, this node cannot be listed, and a direct affiliation with GALION is improper. However, you also insist that you would like to preserve GALION in this tree somehow. The proposal to mention GALION in parentheses is an attempt to preserve GALION as a collaboration vehicle, so that stations that collaborate under this acronym can be searched and found collectively in OSCAR/Surface. The individual networks are directly listed as indicated in the comment above, where NDACC has always collaborated with GAW but never signed an agreement to become a GAW Contributing Network. NDACC is very broad and only a subset of stations perform lidar observations, and that is why the proposal would list the lidar part of NDACC separately. This, BTW, would need to be approved by jeannette.wild@noaa.gov. At this point, I am at a loss as to what could be an acceptable solution, and I would personally welcome a comment from @Netcheva. Finally, the focus on GALION in this issue not to be confused with a focus on GALION as opoosed to other correction of the code list. We are just trying to identify manageable parts.
jdwild
commented
3 years ago I do not see why this has to be so complicated. Keep GALION as a separate entity from NDACC. There are sites associated with NDACC that have no GALION instruments. There are sites associated with GALION that have no NDACC instruments. Just like SHADOZ and NDACC have some shared sites, and some distinct to each. There is no formal MOU between GALION and NDACC, so GALION should not be considered a subset of NDACC ( or any other network). If there were an MOU, GALION would be a Cooperating Network in our nomenclature. That would mean it is a fully operating independent network. Why can't WMO/OSCAR provide a tool to search GALION sites, and/or NDACC sites? That logic seems simple enough. If they are truly independent that should be easy to execute.
ejwelton
commented
3 years ago If an MOU is required to add a word to a code list in your metadata system, then I think there are larger problems to sort out that have nothing to do with me. So it seems to me that the decision to deprecate GALION was already made, yet for some reason this issue was created in GitHub for commentary? The opening comment on this issue indicated that I had agreed to this decision, and my responses thus far have simply been to say that I did not even know about this proposal and continue to disagree with this decision. Thus far I've heard from a representative of AD-Net and they are indifferent to these changes. Jeanette can represent NDACC, and I have no response from EARLINET or LALINET yet. I will just reiterate one last time that the existing implementation of GALION within OSCAR works just fine from a user point of view. It is very easy to either affiliate GALION to a site with a lidar, or choose not to do so. There is no problem from my end. The GALIONNDACC entry should be removed, its unnecessary and confusing. Other than that, since this issue was opened for commentary I will just let my objection stand to deprecating GALION and modifying the MPLNET entry in OSCAR to read MPLNET (GALION).
jdwild
commented
3 years ago Just in case my note was not clear. NDACC and GALION are completely distinct from each other. There should be no GALIONNDACC or NDACC (GALION) entry in OSCAR. This would imply one is a subset of the other, or one is managed by the other. Neither is true. Some sites should show an NDACC affiliation, and some sites should show a GALION affiliation. Sometimes these will intersect, sometimes they will not. As Judd indicates, NDACC is not coordinated by GALION, though the lidar sites may have an affiliation with both.
ejwelton
commented
3 years ago The original reason given for deprecating GALION was that it "is a mere coordination mechanism within the GAW program and not a program in itself." The more recently provided reason is "According to WMO (Oksana), there is no written MoU or LoA that would authorize a code list entry GALION in the program/network codelist." As I've mentioned, there are many entries in the Program Affiliation code list that do not meet these criteria either. How is the GAW Aerosol Lidar Observation Network (GALION) any different than the following entries which are also in the code list: GAW, GAW Contributing Networks, GAW Global, GAW Local, GAW Other Elements, GAW Regional, GAW-PFR, GCOS, also GCW, GOS, have similar series of entries. There are others I could find too, but the code list is long. Most of these are not programs/networks on their own. Why would GALION be expected to have an MOU/LOA with WMO? GALION is a GAW creation. Does GAW have an MOU with itself? So my question is: will these other code list entries be deprecated as well? If so, then you must do whatever is required to clean up and maintain your own database per your requirements (requesting input from me is irrelevant). If this proposal is limited to deprecating GALION only, then my objection stands. Regardless, I don't want my network listed as MPLNET (GALION) in any database.
fstuerzl
commented
3 years ago @ejwelton, thanks for your input. Regarding your question: There is another issue that deals with exactly this: #214 Until now, there is no decision yet on whether to deprecate coordinating terms or not.
fstuerzl
commented
3 years ago Solution as discussed with @tarasova-wmo: GALION and GALIONNDACC should not be in the code list. In the branch, both entries are marked as "invalid", subentries are now under "GAW Contributing networks" (https://github.com/wmo-im/wmds/blob/Issue-%23189/tables_en/2-02.csv).
Important: It will still be possible to add "GALION" as a label in OSCAR/Surface (similar to the label "Centennial Observing Stations"). That means an observation that is affiliated with a programme coordinated by GALION (like e.g. ADNet) can get two seperate affiliations in OSCAR (in this case "ADNet" and the label "GALION"). The affiliation with the label "GALION" could be organised with an approval process.
tarasova-wmo
commented
3 years ago GALION label should be made available similar to "Centennial" with the approval of the label for individual stations of contributing and other networks as well as GAW Regional/Global/Local stations.
ejwelton
commented
3 years ago I do not know what a "label" means in terms of OSCAR metadata. Network affiliation within OSCAR is achieved with the programAffiliation code list. I thought this issue was about deprecating GALION from this list. What is a "label" and how does that relate to program affiliation? Please keep in mind that your system is very complicated and contributing networks must maintain our metadata entires. I have never heard the word label, and thus do not know how to use that in lieu of programAffiliation. Can you explain? If there is a way to maintain GALION as a search option then that is fine with me.
joergklausen
commented
3 years ago Thanks to all who participated in the discussion, I summarize as follows, with a nuance:
Proposal Update entry GALION to appear as a single entry directly under GAW that can be selected in addition to other affiliations and will be subject to approval in OSCAR/Surface. Invalidate the entry GALIONNDACC from code list 2-02. List programs currently under GALION directly under either GAW Contributing networks or GAW Other elements, depending on their official status in GAW.
Reason GALION (GAw LIdar Observations Network) is a network of lidar observations contributing to GAW as described in GAW Report 178. The various programs/networks contributing to GALION are not coordinated as such by GALION, hence there can be no hierarchy that puts GALION above these networks. GALION is a network of networks with a steering group, providing criteria for lidar operators who wish to affiliate. As such, GALION can be considered an affiliation requiring approval. To facilitate documentation of this affiliation in WIGOS metadata, a distinct element 'GALION' is justified. Such an element already exists in the present code list and is best preserved, so that it can be used also in machine-to-machine metadata exchange. No change is required there. Because of the hierarchy stipulated in the past, and the fact that parts of NDACC contribute to GALION, a separate code element GALIONNDACC was introduced in addition to the regular NDACC entry. NDACC as a program does not support the notion of a specific GALIONNDACC. If this hierarchical relationship is now abandoned in favor of a separate additional affiliation with GALION, this entry GALIONNDACC is obsolete and can be removed.
Consequences for GAWSIS-OSCAR/Surface GALION will be an endpoint directly under GAW that can be selected for a search and will retrieve all stations that are affiliated with GALION. Affiliation with GALION will be subject to approval by the GALION co-chair @ejwelton.
The proposed changes are implemented at https://github.com/wmo-im/wmds/commit/300142abe082f22dee7892c5b1172b91d1ea6ea5#diff-a066c50c20373fc15bcd759e58529c31867ae12caecaa7d36ad7627f5e689622 Previous changes dealing with removing GALION as a node and the removal of GALIONNDACC (for reference) at https://github.com/wmo-im/wmds/tree/Issue-%23189
@ejwelton @jdwild @tarasova-wmo Please review and confirm agreement with this branch so that we can hopefully conclude this discussion.
ejwelton
commented
3 years ago I concur with the final proposal. Its all ok with me. Thanks
tarasova-wmo
commented
3 years ago agree with the proposal
joergklausen
commented
3 years ago 2 of 3 stakeholders agree, no response from @jdwild yet, but assumed to also concur. Branch is confirmed.
jdwild
commented
3 years ago I also agree. Sorry I was not available yesterday to respond to this.
amilan17
commented
3 years ago @fstuerzl @joergklausen
I was having difficulty merging the original branch - please edit this new branch with the proposed amendments: https://github.com/wmo-im/wmds/tree/issue189-FT
apologies for the extra work....
fstuerzl
commented
3 years ago @amilan17, @joergklausen I edited the new branch, view differences: https://github.com/wmo-im/wmds/commit/87ecf7825160710e151f0ca518962b127adebd52
amilan17
commented
3 years ago @fstuerzl - 2-02 still has: GALION,\WIGOS\GAW\,GALION,GAW Aerosol Lidar Observation Network
fstuerzl
commented
3 years ago @amilan17, sorry, I have corrected the path now to \WIGOS\GAW\GALION".
amilan17
commented
3 years ago @fstuerzl
I'm still confused. Please see diff of 2-02 in this comparison of old issue and new issue. They don't match. And I don't think the new branch is consistent with Jörg's last proposal from 7 days ago.
Proposal
Update entry GALION to appear as a single entry directly under GAW that can be selected in addition to other affiliations and will be subject to approval in OSCAR/Surface. Invalidate the entry GALIONNDACC from code list 2-02. List programs currently under GALION directly under either GAW Contributing networks or GAW Other elements, depending on their official status in GAW.
@amilan17 now, you have me confused, too. Sorry, if I got anything wrong... The following changes are included in the new branch https://github.com/wmo-im/wmds/compare/issue189-FT:
Isn't this what @joergklausen's proposal intended?
amilan17
commented
3 years ago @joergklausen Can you please verify this branch?
joergklausen
commented
3 years ago Branch https://github.com/wmo-im/wmds/compare/issue189-FT positively validated, all looks good.
fstuerzl
commented
3 years ago @joergklausen, @amilan I think, I found a mistake in this branch. In #187 CIS-LiNet is under GAW Other elements, and not under GAW Contributing networks. I will correct it in the new branch now.
@joergklausen, could you please verify the branch again?
Proposal
Deprecate existing entries GALION and GALIONNDACC.
Reason
GALION is a mere coordination mechanism within the GAW program and not a program in itself. Hence, the existing entries GALION and GALIONNDACC must be deprecated. The proposal is already agreed with the chief of the GAW program, Oksana Tarasova, as well as GALION co-chair @ejwelton.
Branch
https://github.com/wmo-im/wmds/tree/issue189-FT don't use: https://github.com/wmo-im/wmds/tree/Issue-%23189