2024_07: BUG: Resolve SDM-DOC CI/CD action problems

jamesmrollins commented 3 months ago

For Action: @tejaspesquare @matomasz @jamesmrollins FYI: @lzim @lijenn @ljmoody

What Happens

We cannot fire actions that automatically update the SDM-Docs on the MTL-DEMO website because we cannot merge DEV/TEST and MASTER branches as necessary to fire the update actions.
We cannot merge because the v 3.7 Sim UI code was updated directly from the DEV team's Azure/DEVOPS application. Therefore, if we were to merge branches, the existing code would be replaced with out-of-version code.

What Should Happen

When the model file is uploaded to the DEV branch in lzim/mtl an action fires that creates the supporting SDM-Doc HTML file.
The HTML file is uploaded to MTL-DEMO, matched against a CSS and deployed to the DEMO website.
When DEV is merged with TEST and TEST with Master, the actions should update the DEMO website projects accordingly.
We need to bring all code repositories up to date so the merging actions can fire.
We need to test the merging actions to ensure the resulting SDM-Doc HTML files are correct.

Affect on SMEs

None.
We are "hand updating" code in the Epicenter projects, so they are up-to-date and should not pose a problem for normal operations.
We have "hand Updated" the HMTL in the SDM-Doc in the the MTL DEMO website, so these will continue to support research aims.

Estimated Resolution: 7/5

jamesmrollins commented 2 months ago

Sim UI Standup 7/1

[ ] @tejaspesquare will inquire with Divyesh to determine the current status of this fix.
[ ] Will document in this card.

tejaspesquare commented 2 months ago

@jamesmrollins,
We are currently in the process of familiarizing ourselves with the implementation of CICD practices. Our goal is to establish the necessary CICD workflow by the end of this week.

jamesmrollins commented 2 months ago

@tejaspesquare Can you provide an update for this please?

tejaspesquare commented 2 months ago

Hi @jamesmrollins,

The script we have been developing is nearly finished and is almost fully functional. Currently, we are encountering an authorization issue when attempting to move a file from lzim/mtl to jamesmrollins/MTL-DEMO. However, in order to verify the functionality, we tested the same script to move a sample file from jamesmrollins/MTL-DEMO to jamesmrollins/sim_ui, and it worked as anticipated.

jamesmrollins commented 2 months ago

@tejaspesquare

Please provide a weekending update to this issue. Thank you..

tejaspesquare commented 2 months ago

Hello @jamesmrollins,

Despite our efforts to follow steps gathered from different sources, we were unable to achieve success. Every Jobs we attempted ended in failure.

tejaspesquare commented 2 months ago

Hello @jamesmrollins ,

We have not yet achieved success. All jobs getting failed.

lijenn commented 2 months ago

Discussed 7/19 Team Praxis Call:

[ ] @matomasz will check with @tejaspesquare on the status update of this card.

matomasz commented 2 months ago

Discussed 7/22 Standup with Dev:

Someone on DEV team will be working on this and can have an update by tomorrow

matthewtomo commented 2 months ago

Discussed 7/25 Standup with Dev @matthewtomo @tejaspesquare

It appears that this remaining CI/CD issue is failing due to an authentication problem.
@lzim, Would you be able to double check and verify for me that the lzim/mtl repo has the same permission settings as the lzim/teampsd repo based on what we tried on Tuesday: https://github.com/lzim/teampsd/issues/3222#issuecomment-2246301176

fyi: @jamesmrollins

lzim commented 1 month ago

@matomasz

[x] I just checked mtl repo and the Actions settings are set up the same way we reviewed together on Tues

@lzim, Would you be able to double check and verify for me that the lzim/mtl repo has the same permission settings as the lzim/teampsd repo based on what we tried on Tuesday:

matthewtomo commented 1 month ago

@lzim Thank you! We will continue searching for other possible sources of the issue.

fyi: @tejaspesquare

matthewtomo commented 1 month ago

@jamesmrollins I have a few troubleshooting steps that might help resolve our CI/CD pipeline issue. In case the token has expired, it might be easier to just generate a new one as follows.

Troubleshooting

Navigate to https://github.com/settings/tokens/new and generate a new token after providing a name, expiration date, and selecting the repo permission box as shown below (Feel free to set an actual expiration date for security purposes):
After that, it will give the option to copy the token:
Then, if you go to jamesmrollins/MTL-DEMO > Settings > Secrets and variables > Actions we should be able to add it as the secret value that the GitHub Action is looking for "API_TOKEN_GITHUB" (If the secret already exists, we can update its value instead):

Additional Issue

While looking into the failed GitHub Action, I also noticed a potential future issue.
The GitHub Action is defined in the file lzim/mtl/.github/workflows/SyncFilesToMTL-DEMO.yml , where it is attempting to move the file '/blue/mtl3.0_models/agg_i_v15_2022_11_01-A.html'. This file no longer exists. It has been replaced with the file '/blue/mtl3.0_models/agg_i_v15_2024_7_23-A.html' (Name reflects that the file was updated at a later date).

See named files in picture below.

I wanted to try to changing the filename, but it appears I do not have Push Access to lzim/mtl. I have opened a pull request to the Dev branch here: https://github.com/lzim/mtl/pull/470

fyi: @tejaspesquare

matthewtomo commented 1 month ago

Discussed DEV Standup 7/29/2024 @matthewtomo @tejaspesquare

Currently, we are waiting on confirmation from @jamesmrollins as to whether or not the above solution resolves the issue.
If not, we will continue looking into other causes.

fyi: @jamesmrollins @lijenn @ljmoody

matthewtomo commented 1 month ago

Discussed SDM-DOC CI/CD meeting @matthewtomo @jamesmrollins

Generated a new PAT for @jamesmrollins github account to use in the .github/workflows/SyncFilesToMTL-DEMO.yml GitHub Action.
We will need @lzim when she returns to add the token as a secret for the lzim/mtl repo.

fyi: @jamesmrollins @lijenn @ljmoody

lzim commented 1 month ago

@matthewtomo - Is this still needed? I'll add to agenda for #sim_ui_workflow tomorrow.

matthewtomo commented 1 month ago

Discussed sim_ui_workgroup 8/14/2024 @lzim @matthewtomo @lijenn @ljmoody

added new value to lzim/mtl secrets for API_TOKEN_GITHUB

fyi: @tejaspesquare @jamesmrollins

matthewtomo commented 1 month ago

Praxis Meeting week 3 2024_08 @matthewtomo @ljmoody @jamesmrollins

SDM-DOC GitHub Action is now working
[x] @jamesmrollins will make sure the Agg .xmile file is in the lzim/mtl repo
[x] @matthewtomo will remove the broken github action
[ ] @matthewtomo investigate and add .xmile files to the include list for successful action that copies files to jamesmrollins /mtl-demo

lzim / teampsd