Deploy via Git using rsconnect

[fh-mthomson]

Currently, we're using the connectapi package, specifically deploy_repo() to initialize programmatic deployments to Connect.

However, there is an increasing amount of active development in rsconnect that would motivate me to use only this package over connectapi for most interactions with the Connect API server.

Would it be possible support deploying via Git via the rsconnect::DeployApp() function family? I'm envisioning:

rsconnect::deployApp(..., repository = "X", branch = "Y", subdirectory = "Z")

[hadley]

Can you please explain what deploying via git means in this context?

[aronatkins]

Deploying via git (as described in https://github.com/rstudio/connectapi/issues/196) means:

1. creating a content item.
2. configure environment variables for that content item.
3. configuring a Git location for that content item.
4. performing a deployment operation for that content item.

Because the content item is Git-backed, the deploy fetches code from the Git location and creates a bundle from those contents; the manifest and all code is expected to already be present in the Git location.

They can perform this workflow using connectapi today, but would need to perform the connectapi::deploy_repo() steps outside that function call.

The writeManifest() function contributes to this workflow, since a valid manifest needs to be in the Git repository, but that needs to exist in the remote repository before beginning the workflow. Neither rsconnect nor connectapi perform the Git commit/push operations that would be required.

[fh-mthomson]

The current workflow is:

1. Create manifest (rsconnect::writeManifest()) and commit / push
2. Deploy via Git (connectapi::deploy_repo())

I'm curious whether this can be condensed to

1. Same as above
2. rsconnect::deployApp()

Specifically, using rsconnect::deployApp() would have the following benefits:

• Better user-facing messaging during the deployment process (connectapi::deploy_repo() tends to be silent)
• Already includes support for envVars
• Coalesces user workflows to a single package (rsconnect)

[hadley]

It seems like an ideal workflow would be to call writeManifest() then push to git (or pause so user can do it) then trigger the deploy. Or do we assume that you only need to deployGit() once because after that connect will automatically pick up changes as you push?

[aronatkins]

You need to configure content to track a Git location only once. If that content is configured to poll, no additional deploy actions are needed. If the content does not poll, a deploy action will pull from the repository at the time of that request.

@fh-mthomson - after the content has been created and configured, what is your "active development" workflow?

[fh-mthomson]

Exactly!

• Initial deployment (one time)

  • rsconnect::writeManifest()
  • Create PR to land to main (usually we have peer review)
  • Once landed, rsconnect::deployGit() tells Connect where to look for a manifest.json (and source files) from which to create/deploy an initial bundle

• Subsequent active development (where git polls for any incremental changes):

  • Make changes to .Rmd (or plumber.R) files
  • Create a PR against main
  • Merge after peer review
  • Connect git poll picks these changes up, and re-generates / deploys the updated bundle accordingly.

Also, less common, if we need to update the manifest.json manually (e.g., changing the dependencies), then manually:

1. Update renv.lock, renv::restore()
2. Create updated manifest: rsconnect::writeManifest()
3. Push new manifest to main which then gets picked up automatically by Connect (via git poll)
4. Connect run a new packrat Restore and then renders the updated content

[aronatkins]

I think for now, the best course is:

1. use rsconnect::writeManifest() to construct an appropriate manifest.json. This gives you the dependency-detection improvements, which are the pieces of rsconnect::deployApp that are appropriate to your workflow.
2. use connectapi:: deploy_repo() to create a content item associated with a named Git repository.
3. use the connectapi Content functions to set environment variables.

You have filed https://github.com/rstudio/connectapi/issues/196 to set environment variables when creating that Git-backed item; that's a great suggestion. Even with that change, you will need a separate workflow using connectapi to manage environment variables afterwards.

The connectapi Content object should have an environment_set function. That's not appearing in the current documentation, unfortunately. I did a sweep recently to produce documentation for more of the surface area, but that hasn't been published, yet. I'll see if I can make that happen.

[fh-mthomson]

Sounds reasonable for an interim approach! I'll look further into the existing env var functionality in connectapi, as well.

Thank you @aronatkins @hadley for the great functionality (in both rsconnect and connectapi).

[hadley]

So maybe this function would be more like setupGitDeployment(). @aronatkins do you think we should implement this here, or leave it in the realm of connectapi?

[aronatkins]

There is a pending improvement to rsconnect-python that will support creating Git-backed content: https://github.com/rstudio/rsconnect-python/pull/501

Let's leave it in connectapi for now. Admittedly, this could have landed in either package. rsconnect it typically more opinionated "workflow", while connectapi is more direct API usage.

Closing this issue assuming that the connectapi solution is sufficient, but we're willing to revisit if it's not appropriate.

CC @mmarchetti

[fh-mthomson]

While the connectapi solution is sufficient (i.e., I can do what I'd like with a combo of connectapi and rsconnect), my motivation for opening each this issue and https://github.com/rstudio/connectapi/issues/196 was to solicit a more intuitive, longer-term path where users could use one package for all deployment workflows (manual, git-backed, setting env vars, etc).

Increasingly, we've preferred rsconnect over connectapi due to the former having (1) more active development (2) more opinionated workflows that (3) closely integrate with Workbench's Publish UI and (4) the majority of "end users" very rarely need (or want) to interact with the API endpoints for simple deployment purposes.

To be clear, (1) is not a critique against connectapi but a testament to y'all making many, rapid enhancements to rsconnect that have benefitted our users. Thank you!

Expanding upon (3), I'd eventually love for users to be able to use click-button Publish via Git from Workbench. E.g., click Publish, specify a repo / branch, and, optionally, select local Env vars to upload securely with the bundle (https://github.com/rstudio/rstudio/issues/13032), then click "deploy" for immediate interactive feedback (pinging the Connect API from Workbench).

In contrast, programmatic deployment via connectapi tends to lend to a slower feedback loop that requires toggling between applications (Workbench <> Connect) e.g., to find the relevant deployment logs when things go awry. Related: https://github.com/rstudio/rsconnect/issues/725

Thanks again for the great packages and considering the options to move forward!

[hadley]

FWIW I agree with the vision that most folks should only ever need to know about rsconnect, and since Git backed deployment is pretty common, it'd be nice to include a helper here.

[fh-mthomson]

@aronatkins @hadley would you kindly be open to re-opening this issue?

In the interim, I'm finding myself rewriting aspects of connectapi to avoid introducing current conflicts with dbplyr (https://github.com/rstudio/connectapi/issues/177)