Error during workflow registration

[yadudoc]

I'm running into an error during the workflow registration step listed in the FaaSr-tutorial repo (https://github.com/FaaSr/FaaSr-tutorial?tab=readme-ov-file#register-and-invoke-the-simple-workflow-with-github-actions)

Here's a snippet of the error message I'm seeing while attempting to run the tutorial on Posit cloud:

> faasr_tutorial$register_workflow()

── Registering workflow github actions for repo: My_GitHub_Account ───────────────────────────────────────────
❓Enter repository visibility[private/public]
public
✔ Create github local directories
✔ Create github env files
✔ Create github payload file
✔ Create github REAME file
✔ Create github workflow yml file
✔ Create github local repository                                                                  
✖ Error: Failed to create the repo                                                                
Error in faasr_register_workflow_git_remote_repo(token, response, private,  : 25% [1/4]   ETA:10s

FaaSr \ Registering workflow github actions ■■■■■■■■■■■■■■■■                  50% [2/4]   ETA: 4s

My first guess was that this might be an issue with my github PAT token. I went back and recreated the token with workflow and read:org scopes and loaded this up with credentials::set_github_pat(force_new=TRUE). That did not fix the issue. Please let me know if you have any recommendations for solving this. Thanks in advance!

This is part of my review efforts for your JOSS submission. Required for the functionality checklist item for openjournals/joss-reviews#7027

[renatof]

thank you again, @yadudoc - this is odd, we will try to replicate this problem on posit cloud and let you know what we recommend

[renatof]

hi @yadudoc - I just went through the posit tutorial again and verified that it worked for me following the tutorial page. I wonder if one of these could have happened:

1) perhaps you didn't update/save the faasr_env file with the new token? 2) perhaps the tutorial_simple.json wasn't configured/saved with a UserName that matched your GitHub account?

If it's not too much trouble for you, would it be possible for you to give it another go with a fresh Posit instance and the new token you created?

I sincerely appreciate your feedback and patience - thank you!

(ps: I haven't used credentials::set_github_pat(force_new=TRUE) before - not sure if there may be untested side-effects?)

[yadudoc]

Hi @renatof Thanks for your response. I redid the whole process and I believe I got one step farther, but the setup did not complete successfully. Here's the log message:

> usethis::use_git_config(user.name = "yadudoc", user.email = "yadudoc1729@gmail.com")
> credentials::set_github_pat()
If prompted for GitHub credentials, enter your PAT in the password field
Using GITHUB_PAT from Yadu Nand Babuji (credential helper: cache)
> faasr_tutorial <- faasr(json_path="tutorial_simple.json", env="faasr_env")

── Start FaaSr client tools ───────────────────────────────────────────────────────────────────────────────────────────
✔ Successfully get configuration from tutorial_simple.json and faasr_env
✔ Create the FaaSr directory: faasr_gh_local_repo
✔ Create the FaaSr directory: faasr_data
✔ Ready to use FaaSr client tools:
  • $register_workflow
  • $invoke_workflow
  • $set_workflow_timer
  • $unset_workflow_timer
> faasr_tutorial$register_workflow()

── Registering workflow github actions for repo: My_GitHub_Account ────────────────────────────────────────────────────
❓Enter repository visibility[private/public]
public
✔ Create github local directories
✔ Create github env files
✔ Create github payload file
✔ Create github REAME file
✔ Create github workflow yml file
✔ Create github local repository                                                                  
✔ Successfully create the repo                                                                    
✖ Error: Failed to update the remote repository                                                   
Error in faasr_register_workflow_github_actions(faasr, cred) :                50% [2/4]   ETA: 5s
FaaSr | Registering workflow github actions ■■■■■■■■■■■■■■■■                  50% [2/4]   ETA: 6s

I'm pretty sure my github credentials are correct, because I do see this newly created repo from when I ran the last command I've pasted above. (https://github.com/yadudoc/FaaSr-tutorial). Is this a perms issue? Does the token need perms beyond the perms I've attached here:

Any ideas on what might be going wrong?

[renatof]

Hi @yadudoc - thank you for the update. It looks like your token has the right permissions. I wonder if this is a side-effect of the first attempt; would you be able to delete the FaaSr-tutorial repo on github and retry (or configure a different repo name on "ActionRepoName": "FaaSr-tutorial") and retry? Thanks!

[yadudoc]

Hi @renatof, I tried changing the repo name and that didn't work either. I updated the ActionRepoName, re-registered the workflow and invoked it. I'm getting the same error as earlier:

> faasr_tutorial <- faasr(json_path="tutorial_simple.json", env="faasr_env")

── Start FaaSr client tools ───────────────────────────────────────────────────────────────────────────────────────────
✔ Successfully get configuration from tutorial_simple.json and faasr_env
✔ Ready to use FaaSr client tools:
  • $register_workflow
  • $invoke_workflow
  • $set_workflow_timer
  • $unset_workflow_timer
> faasr_tutorial$register_workflow()

── Registering workflow github actions for repo: My_GitHub_Account ────────────────────────────────────────────────────
❓Enter repository visibility[private/public]
public
✔ Create github local directories
✔ Create github env files
✔ Create github payload file
✔ Create github REAME file
✔ Create github workflow yml file
✔ Create github local repository                                                                  
✔ Successfully create the repo                                                                    
✖ Error: Failed to update the remote repository                                                   
Error in faasr_register_workflow_github_actions(faasr, cred) :                25% [1/4]   ETA: 9s
FaaSr \ Registering workflow github actions ■■■■■■■■■■■■■■■■                  50% [2/4]   ETA: 4s

[renatof]

Hello @yadudoc and thank you so much for your patience! @spark0510 and I went through the whole tutorial again from scratch independently, and we were not able to reproduce the problem.

One thing we did notice is that the tutorial documentation did not explicitly state to choose the choice of a “Classic” token - that was a failure on my part, and I apologize for the lack of clarity; the tutorial has since been updated.

Maybe this could be the root cause of the problem? We have not yet tested with tokens other than classic tokens, and the error message you show happens when "git push" fails.

If you would be so kind to try again from scratch, I would suggest:

1. On Github, create a new short-lived classic token with workflow and read:org scopes
2. On Github, delete your FaaSr-tutorial repo
3. Start a new Posit cloud instance and set it up with FaaSr as per tutorial docs
4. Enter the token above in faasr_env and save it
5. Enter your GitHub user name in tutorial_simple,json and save it

Let us know if this works? Thank you again very much for your time and feedback, we really appreciate it!

[yadudoc]

@renatof All of my latest tests were done with the classic token, so I don't think that is the issue. Do you know if the package drop logs somewhere that might have info to dig into?

[renatof]

hi @yadudoc - thank you again for your help reviewing!

We do drop logs for actions in the S3 bucket, but the error you're seeing is at registration time before the actions run, so before any action logs are created.

The error message indicates that the repository failed to update, which means that the Github API request to create the repository was successful but the git CLI command used to push updates to the repo failed (this is LOC 440 in https://github.com/FaaSr/FaaSr-package/blob/main/R/faasr_client_api_github-actions.R).

This seems to me it would be an indication of an problem with the token, though it's odd that the repo was created but not updated. We haven't been able to reproduce the problem yet.

Looking at your token permissions screenshot in the thread above, just wanted to check if the gray boxes ticked automatically by github upon selection of "workflow" also looked the same for you?

Thanks again!

[yadudoc]

@renatof I made progress after redoing the steps from scratch. The workflow invocation worked, but when checking for outputs, it appears to have failed. Here's a snippet from the shell:

☰ Successfully registered all github actions
> faasr_tutorial$invoke_workflow()
✔ faasr_register_workflow_github_invoke: GitHub Action: Successfully invoked:start
> mc_ls("play/faasr/tutorial")
Error in `processx::run(binary, args, ...)`:
! System command 'mc' failed
---
Exit status: 1
Stderr: <empty>
---
Type .Last.error to see the more details.
> mc_ls("play/faasr/tutorial")
Error in `processx::run(binary, args, ...)`:
! System command 'mc' failed
---
Exit status: 1
Stderr: <empty>
---
Type .Last.error to see the more details.
> mc_cat("play/faasr/tutorial/sample1.csv")
Error in `processx::run(binary, args, ...)`:
! System command 'mc' failed
---
Exit status: 1
Stderr:
mc: <ERROR> Unable to read from `play/faasr/tutorial/sample1.csv`. Bucket `faasr` does not exist.
---
Type .Last.error to see the more details.

[renatof]

Hi @yadudoc - glad to hear you got to this point, and thank you so much for trying again!

I think what happened is that the S3 faasr bucket we use in this tutorial is automatically deleted every so often. It's a free testing environment provided by minio; we create the test faasr bucket in the tutorial script but it may have been deleted by the time you checked. If you can try to invoke the workflow again, but first create the bucket to ensure it's in the play server: mc_mb('play/faasr') faasr_tutorial$invoke_workflow() mc_ls("play/faasr/tutorial")

[yadudoc]

It worked! Some steps took a bit of delay before the outputs popped up, I would recommend mentioning that in the tutorial.

Here's what I see now:

> mc_mb('play/faasr')
Bucket created successfully `play/faasr`.

> faasr_tutorial$invoke_workflow()
✔ faasr_register_workflow_github_invoke: GitHub Action: Successfully invoked:start
> mc_ls("play/faasr/tutorial")
data frame with 0 columns and 0 rows
> mc_ls("play/faasr/tutorial")
data frame with 0 columns and 0 rows
> mc_ls("play/faasr/tutorial")
data frame with 0 columns and 0 rows
> mc_cat("play/faasr/tutorial/sample1.csv")
Error in `processx::run(binary, args, ...)`:
! System command 'mc' failed
---
Exit status: 1
Stderr:
mc: <ERROR> Unable to read from `play/faasr/tutorial/sample1.csv`. Object does not exist.
---
Type .Last.error to see the more details.
> mc_ls("play/faasr/tutorial")
[1] "sample1.csv" "sample2.csv"
> mc_cat("play/faasr/tutorial/sample1.csv")
[1] "\"v1\",\"v2\",\"v3\"\n1,1,1\n2,4,8\n3,9,27\n4,16,64\n5,25,125\n6,36,216\n7,49,343\n8,64,512\n9,81,729\n10,100,1000\n"