Feature Request: Workspace-scoped project settings

[denysvitali]

Is your feature request related to a problem? Please describe.

For some reason, setting up the project settings on a workspace, applies them to all the workspaces. This could either be a good thing (no need to do it manually), but also a bad thing (you can't have branch=workspace for a project).

Describe the solution you'd like Ideally, one should be able to poll from git using different settings for each workspace. With this addition, it should be possible to have a GitOps repository where you store (divided in multiple branches) different configurations for Waypoint, so that Waypoint can poll them.

Describe alternatives you've considered Creating a project for each environment

Explain any additional use-cases

• Different git target for different workspaces

Additional context

The view / settings are the same for each workspace, causing the git poll to be performed only on the default workspace and thus defeating the purpose of having workspaces in the first place.

[briancain]

Hey there @denysvitali ! Thanks for the issue. I'll try to give some context to some of the info in there just to fill out the gaps for the issue 😄

For some reason, setting up the project settings on a workspace, applies them to all the workspaces.

After looking into this, it seems to be a limitation of the current API. There's no way to apply settings to a certain workspace, and so when you save settings it gets applied to everything. We should fix this! It looks like the first step is making UpsertProjectRequest take a workspace parameter.

Ideally, one should be able to poll from git using different settings for each workspace.

The view / settings are the same for each workspace, causing the git poll to be performed only on the default workspace and thus defeating the purpose of having workspaces in the first place.

This is also a limitation of Waypoint today. Currently (as of release 0.4.0) git polling for a project only works on the default namespace. 😞 This is something we're going to have to fix. Both because projects should be able to poll across all workspaces independently, and because more features are going to start taking advantage of our internal polling queuer and should work across everything. Thanks for calling this out! 👍🏻

[denysvitali]

Thanks for the detailed answer Brian!

Since this is a little bit of a showstopper for our use case (deploying multiple environments from a single GitOps repository) I'll try to tackle the issue myself if you don't plan to do it soon - thanks for the pointers! 😊

[briancain]

@denysvitali - I would recommend waiting a bit before trying to tackle the issue. After chatting with the team this morning about this issue, the reasoning behind why the initial implementation of workspace settings is so simple was to ensure whatever solution we come up with for multi-workspace settings, promotion workflows, etc, will work without any major architecture refactoring. You are running into these limitations now, which is why things are behaving like you mentioned 😅

Personally, I think we should have a proper answer for this problem sooner rather than later because I see it as an important feature of Waypoint to support these multi-workspace and promotion workflows. I'll leave this issue open for now to track this and hopefully soon we as a team will have a plan to address this limitation in an upcoming release 🙏🏻 We appreciate all of the issues and context you have been providing for us for Waypoint! Thanks again.

[denysvitali]

I'm starting to tackle this issue by creating a ScopedProjectSettings map in the Project. The idea is to have a default implementation, and then some variations of that default based on the workspace.

I'm up for comments and suggestions :)

[denysvitali]

Friendly ping @briancain , a PR has been opened already since quite some time now 👍

[briancain]

Hey there @denysvitali! Sorry it has been so long since we got back to you about your pull request. Thank you for your patience there :pray:

The reason why we have not yet given that pull request its review is because we have been working on the groundwork for getting Waypoint in a position to handle workspace settings properly. Since we're not ready for this feature in Waypoint yet, we didn't want to merge an option that would possibly clash with our workspaces improvements, creating some tech debt we'd need to clean up and handle once we got to addressing workspace settings.

In the next release of 0.6.0, we've been working to nail down how workspaces work internally across every piece of Waypoint. You can see the initial work going in 0.6.0 here in these pull requests from @mitchellh:

• Project, app, workspace, and label scoping for Runner Config
• Scoped config in waypoint.hcl
• Expand config var scoping capabilities: by workspace, label selector, runner by project/app, etc.

Some CLI quality of life improvements around workspaces from @catsby:

• Add workspace CLI command, list/inspect subcommands
• Add context set subcommand

Next, we're planning on addressing how to set workspace settings (i.e. this issue), especially around polling different git branches across different workspaces in our next major release in 0.7.0. When we are ready, we'd love to share our thoughts on how we plan to implement this setting here to get some early feedback prior to its implementation if that's something you would like to contribute on.

Thanks again for hanging in there with us while we sort this out!

[denysvitali]

Whoa @briancain, I probably lived under a rock for the last 2 months. This is amazing!

Thank you! Can't wait to see these improvements! 🎉

---

Oh, and btw, unrelated, but today I held a public presentation about Hashicorp Waypoint - you can find the slides here: https://waypoint-slides.k8s.best/

[briancain]

@denysvitali wow! Love the slides 😄 Looks like you wrote a Waypoint app to deploy your slides? What a cool idea, love it!

[denysvitali]

https://github.com/denysvitali/deployments-made-easy-slides

(And I'll stop here with the OT, sorry 😅)

[briancain]

No, honestly, more please! 😆 Maybe send it to our team email or share on discuss, we love stuff like this! Thank you for sharing 🎉

[nsurleraux-railnova]

Is there any news on this? I would like to deploy 2 versions of the same app from 2 different branches (master for prod environment, dev for dev environment)

[nsurleraux-railnova]

Quick update: I tried to Git-workflow based on a single branch for several environment with Github Releases as trigger for the production but it is not enough in special cases (old code is on master, deployed on dev, but not ready for production, I have no choice but to deploy this code if I want to apply a bugfix on prod).

I really enjoy the philosophy of Waypoint and tried my best to workaround missing features, but it will be complicated to consider Waypoint mature until this feature is implemented.

@xiaolin-ninja @briancain Any ETA on this ?

[acaloiaro]

Please correct me if I get any of this wrong so that this issue correctly reflects the current state of things.

An attempt was made by @denysvitali in https://github.com/hashicorp/waypoint/pull/1815 to address the issue.

@briancain clarified that the team is thinking about some possible solutions and would be laying the groundwork for this feature in October 2021.

To respond to your comment above

When we are ready, we'd love to share our thoughts on how we plan to implement this setting here to get some early feedback prior to its implementation if that's something you would like to contribute on.

I would love to provide feedback on any proposed implementations as this feature is fairly critical to branch-based dev/staging/prod deployment workflows.

@evanphx followed up with a response on Denys' PR in March 2022, saying that the team is thinking about a slightly different solution to the problem here: https://github.com/hashicorp/waypoint/pull/1815#issuecomment-1057225177

---

For what it's worth, if the behavior for workspace-scoped project settings mimicked config variable scope, I'd be quite pleased.

[briancain]

Hey all -

Thanks so much for being patient with us on this feature request/issue. We are sorry that we haven’t updated you all in a while with our conclusion for going forward supporting this feature. We want to say we agree that this is a critical feature for Waypoints maturity as a tool, and because it’s so critical, we wanted to ensure that our solution was future proof before introducing a setting or option that we’d regret later down the line as we make major configuration and workflow changes to Waypoint as an early tool.

At the time of this issue being initially reported, we had not fully captured our intentions for how Workspaces should behave in Waypoint. Since last year, we’ve made a lot of major improvements to Workspaces in Waypoint that gets the project in a better position to handle workspace-scoped settings. However, our team did investigate an interim solution for Waypoint to handle workspace-scoped git poll settings without requiring the settings be stored on disk. The interim solution added more technical debt that we’d be comfortable with, and decided to hold off until Waypoint completes its implementation of custom Pipelines for projects.

This feature request has not been forgotten by the team. We believe that we will be ready to support this feature as we implement Waypoint Pipelines this year and intend to support multi-workspace git polling. Given that Pipelines will be a major step forward in how users will configure their application deployments in Waypoint, we think using this new structure will be the appropriate place for configuring git poll settings per-workspace via a git ref. You can see the beginnings of the Pipelines work by Mitchell a bit ago: https://github.com/hashicorp/waypoint/pull/3283 and it currently lives (not 100% functional) in the actively developed feature branch f-pipelines.

All of that said, we thank you for being passionate about Waypoint as it gains its sea legs as an early product. We value your opinions on this and would love to continue hearing feedback from you. Thank you so much.

[acaloiaro]

Hi @briancain, now that pipelines have been made available in preview, could you help us understand where multi-workspace polling stands?

I noticed that waypoint project inspect now shows a Workspaces field. However, the list of workspaces seem to be a global list of workspaces, and not tied to the current "project" in a discernible way. E.g. I see one of my workspace listed that is only associated with one different project. The context of the Workspaces field on a project's inspect output implies that Workspaces is a list of workspaces for which the current project has custom configuration (e.g. workspace-specific git polling configuration).

From this I gather that workspace-scoped project configuration is not yet ready, but is in the works.

Thanks for any updates!