Logic in case_st_archive doesn't allow multi-instance cases with mpas components

[billsacks]

The logic in case_st_archive.py: _archive_history_files does not handle the possibility that we could have an mpas component (which has a different pattern for history file names) in a multi-instance run. @kdraeder changed this logic in #1916 but the fundamental problem still remains. In that PR, this is noted explicitly:

            if ninst_string:
               if compname.find('mpas') == 0:
                  # Not correct, but MPAS' multi-instance name format is unknown.
                  newsuffix = compname + '.*' + suffix

[mvertens]

I think we need to refactor the short term archiver to not have hard-wired suffix assumptions in the code - and that this needs to be data driven.

On Wed, Sep 27, 2017 at 3:48 PM, Bill Sacks notifications@github.com wrote:

The logic in case_st_archive.py: _archive_history_files does not handle the possibility that we could have an mpas component (which has a different pattern for history file names) in a multi-instance run. @kdraeder https://github.com/kdraeder changed this logic in #1916 https://github.com/ESMCI/cime/pull/1916 but the fundamental problem still remains. In that PR, this is noted explicitly:

        if ninst_string:
           if compname.find('mpas') == 0:
              # Not correct, but MPAS' multi-instance name format is unknown.
              newsuffix = compname + '.*' + suffix

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ESMCI/cime/issues/1931, or mute the thread https://github.com/notifications/unsubscribe-auth/AHlxE_388PZ8WxqmX4CtNzawpqyLhNEoks5smsLCgaJpZM4PmceB .

[jedwards4b]

Do we have any requirements for component file name formats? If so they should be clearly stated in the documentation.

[mvertens]

I don't think this is a documentation issue. I think if the short term archiver xml files were actually distributed - then it would be up to each component to do this. As a first step - I think we need to move the current hard-wired code into the existing xml file and remove the hard wiring and then proceed to distribute it to the component directories.

On Thu, Sep 28, 2017 at 7:53 AM, jedwards4b notifications@github.com wrote:

Do we have any requirements for component file name formats? If so they should be clearly stated in the documentation.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ESMCI/cime/issues/1931#issuecomment-332843182, or mute the thread https://github.com/notifications/unsubscribe-auth/AHlxE2AFbot-OAyQIY0OojwdfjNqMi8Kks5sm6TLgaJpZM4PmceB .

[billsacks]

I seem to remember @rljacob saying on a recent cime call that mpas was open to changing its file naming conventions to match what's done for the other components, in which case this issue would go away. @rljacob am I remembering that right?

[mvertens]

I don't think expecting component models to necessarily always change the file naming convention is the right long term solution. Some minimum standard needs to be kept - but the current implementation needs to be made more general and data driven rather than hard-wired code assumptions.

On Thu, Sep 28, 2017 at 7:59 AM, Bill Sacks notifications@github.com wrote:

I seem to remember @rljacob https://github.com/rljacob saying on a recent cime call that mpas was open to changing its file naming conventions to match what's done for the other components, in which case this issue would go away. @rljacob https://github.com/rljacob am I remembering that right?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ESMCI/cime/issues/1931#issuecomment-332844978, or mute the thread https://github.com/notifications/unsubscribe-auth/AHlxExXgBW5NagDsrpUTwILXvKNKsYIAks5sm6YwgaJpZM4PmceB .

[rljacob]

Yes MPAS is open to that.

[mfdeakin-sandia]

It's not clear to me what the next step on this is?

[jedwards4b]

The short term archiver should use the regex expressions as defined in config_archive.xml as file name keys - is the regex for mpas currently correct in config_archives.xml? I'm working on adding tests for st_archive so I can include this in my branch.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.