Secure Worker routing

[gentlementlegen]

We used to only have GitHub Actions, which can be secured using GitHub Authentication system. Now that we have workers, they are currently open endpoints, which implies security risks as anyone can access them and potentially trigger them outside of our control.

This is why we should require authentication to reach these endpoints as well. After looking at what Cloudflare offers, I stumbled upon https://developers.cloudflare.com/cloudflare-one/ which provides authentication system Cloudflare wide.

I need to do some more research but from what I understand, we can provide routes we would like to protect, by only allowing users, IPs, domain to be able to reach it. Before doing any request, the caller should ask to exchange some credentials for a token, used for subsequent calls. Then we would just have to check for that token within our Workers, and deny the request if none is present.

The challenge is for 3rd parties, that would have to provide their endpoint for acquiring the token so we can call their services. Maybe that's where the IP whitelisting could come in handy.

Likewise, the invocation of these workers should be secured on the kernel side, to properly handle their callback, or, directly pickup the result of the fetch call to trigger next plugins in the chain.

[0x4007]

@whilefoo suggested creating workflow IDs and storing them in KV during a workflow execution.

Workflow meaning a chain of plugins for a specific response.

This workflow ID should be private and consequently can be used as the credential when the plugin is communicating with the kernel.

A specific workflow ID would be associated with specific installation ID + repository so it should inherit those credentials from within the kernel only.

[gentlementlegen]

That was in the scenario of having callbacks as the Kernel can check these values to see if the callback is legitimate. But on the Worker side there is no way to know if that ID is valid, except if the Worker pokes back the Kernel to check for validity which would make lots of round trips. And we don't need to do callbacks, we could directly pick up the result from the API call itself as the time elapsed will be < 1 second most likely all the time.

By using Zero Trust, Cloudflare creates proper tokens with granularity allowing only specific callers, by tunneling the request. We definitely need to protect the routes from random callers, so it seemed to me that it was the most direct way. Open to any suggestions!

[whilefoo]

Cloudflare zero trust is used for UI auth, not M2M auth as I understand. @gentlementlegen did you find a way to do it?

IP whitelist wouldn't work well because the kernel is deployed as a Cloudflare Worker so you'd need to whitelist all Cloudflare IPs which means any worker deployed on Cloudflare can access it.

What about using a private/public key pair which we already have for the Github App? When we make a request to the plugin we provide a signature signed by the private key and the plugin can verify this request came from the kernel by verifying the signature with our public key

[gentlementlegen]

New discovery is that Cloudflare workers can't call other Cloudflare workers, as it is unsupported (see error 1042). We will need 2 Cloudflare account probably, one with the kernel and one with the plugins if we want to be able to communicate with the current setup.

Also yes a signature would be sufficient I believe, I will try to set this up shortly.

---

Tested with a 2 Cloudflare accounts setup, works fine.

[whilefoo]

That's a bummer. @gentlementlegen If you don't mind (and if you haven't started), I can get started on this

[gentlementlegen]

@whilefoo Sure please do! Keep in mind that you will need 2 Cloudflare accounts with the Kernel in one and plugins in the other to have the whole workflow working.

[rndquu]

New discovery is that Cloudflare workers can't call other Cloudflare workers, as it is unsupported (see error 1042). We will need 2 Cloudflare account probably, one with the kernel and one with the plugins if we want to be able to communicate with the current setup.

Also yes a signature would be sufficient I believe, I will try to set this up shortly.

Tested with a 2 Cloudflare accounts setup, works fine.

As far as I understand 2 Cloudflare workers can call each other only if they exist in separate domains / cloudflare zones since Cloudflare workers from a single domain run in a single thread. There are service bindings but I'm not sure if they are applicable since they require setting service bindings for all of the core (i.e. "official") plugins like conversation rewards, assistive pricing, etc...

[ubiquibot[bot]]

! No price label has been set. Skipping permit generation.

[ubiquity-os-main[bot]]

[ 6.198 WXDAI ]

@0x4007

Contributions Overview

View	Contribution	Count	Reward
Issue	Comment	1	6.09
Review	Comment	1	0.108

Conversation Incentives

Comment	Formatting	Relevance	Reward
@whilefoo suggested creating workflow IDs and storing them in KV…	7 p: count: 70 score: 1	0.87	6.09
Glad to see that you are pushing code again.	0.9 p: count: 9 score: 1	0.12	0.108

[ 64.404 WXDAI ]

@gentlementlegen

Contributions Overview

View	Contribution	Count	Reward
Issue	Specification	1	20.564
Issue	Comment	3	35.434
Review	Comment	4	8.406

Conversation Incentives

Comment	Formatting	Relevance	Reward
We used to only have GitHub Actions, which can be secured using …	21.2 p: count: 211 score: 1 code: count: 1 score: 1	0.97	20.564
That was in the scenario of having callbacks as the Kernel can c…	26.8 p: count: 134 score: 1	0.755	20.234
New discovery is that Cloudflare workers can't call other Cloudf…	14.8 p: count: 73 score: 1 a: count: 1 score: 1	0.65	9.62
@whilefoo Sure please do! Keep in mind that you will need 2 Clou…	6 p: count: 30 score: 1	0.93	5.58
Is this referenced somewhere else? If not I believe this can als…	1.8 p: count: 17 score: 1 code: count: 1 score: 1	0.755	1.359
```suggestion export async function dispatchWorker(targetUrl: st…	2.4 p: count: 14 score: 1 code: count: 10 score: 1	0.655	1.572
@whilefoo Thank you for this. Will you take care of implementing…	1.8 p: count: 18 score: 1	0.625	1.125
I believe that https://github.com/ubiquibot/comment-incentives a…	5.8 p: count: 58 score: 1	0.75	4.35

[ 6.352 WXDAI ]

@whilefoo

Contributions Overview

View	Contribution	Count	Reward
Issue	Comment	2	0
Review	Comment	3	6.352

Conversation Incentives

Comment	Formatting	Relevance	Reward
Cloudflare zero trust is used for UI auth, not M2M auth as I und…	0 p: count: 106 score: 1	0.765	-
That's a bummer. @gentlementlegen If you don't mind (and if you…	0 p: count: 19 score: 1	0.68	-
Resolves #43	0 p: count: 2 score: 1	0.1	-
I checked and it's not referenced anywhere so I'll remove it	4.4 p: count: 11 score: 1	0.3	1.32
Yes I have implemented it in `command-query-user` because I need…	14.8 p: count: 36 score: 1 code: count: 1 score: 1	0.34	5.032

[ 5.134 WXDAI ]

@rndquu

Contributions Overview

View	Contribution	Count	Reward
Issue	Comment	1	5.134

Conversation Incentives

Comment	Formatting	Relevance	Reward
As far as I understand 2 Cloudflare workers can call each other …	6.8 p: count: 66 score: 1 a: count: 2 score: 1	0.755	5.134

[gentlementlegen]

@0x4007 It's true that the new bot is capable of evaluating comments without price labels. It is not something we want? If so, will change that behavior.

[0x4007]

@0x4007 It's true that the new bot is capable of evaluating comments without price labels. It is not something we want? If so, will change that behavior.

Requires price label to be a funded task. I suppose this should be configurable, with the default being off.

Also just realized that one run of ChatGPT 4o costs the same as 10x ChatGPT 3.5. Given that we are currently doing a 10x sample of the comments using 3.5 for the relevance scoring, lets just do a single run of 4o now instead.