Suggest to split core from all handlers

[danbars]

Hi,

The list of handlers is becoming long. I suggest an enhancement that handlers will not (necessarily) be part of this core repo. Instead, anyone can create a repo called cloudworker-proxy-handler-xxx , and you'll maintain a just list of known handlers. This is common practice in many core libraries that I see. Technically it is possible today without any change to your code, I just suggest that this will become an "official" (i.e. documented) way to contribute handlers. The benefits are smaller core size, easier maintenance (separation of concerns), and easier contribution. WDYT?

[markusahlstrand]

HI @danbars,

Sounds like a good idea. Do you think we should separate the repo or keep it as a monorepo and publish the handers separately? And should there be a couple of "core" handlers that are available in the main package?

[danbars]

I think there could be some handlers which are part of the core. Going over the list I can name: cache, kv-storage, basic-auth, loadbalancer, origin, rate-limit, response, transform, cors

The reason is that they feel more closely related to cloudflare workers rather than depending on specific 3rd party. Basic-auth is in the list just so anyone can get a working demo without any additional dependency. We could also add jwt and oauth but I think that requirements may vary a lot depending on the oauth provider or the jwt verification requirement (for example, comparing claims or audience). Eventually there may be oauth handlers specific to providers that will cover the verifications needed by that provider, and will get as options the needed parameter for that provider (e.g. google project id)

Personally I don't like mono-repos, but that's just personal preference. So I think what you feel is easier for maintenance.

The developer experience should be imo to npm install and then import the handlers that they use.

[markusahlstrand]

Maybe we could do a 2.0 release where the handlers are separated out? Might be a few other breaking changes that we could add at the same time..

One benefit with having the handlers as separate repo's is that it would be less centralised and it might be easier for others to publish their own handlers?

[danbars]

Definitely agree about that benefit. 2.0 sounds like a good idea. Do you have other breaking changes that you were holding?

2 more things that I was thinking about:

1. Logger interface that is passed to each handler as param, so they can call logger.debug() etc in a standard way. The default implementation for it can be console.log which is good enough for development. And other implementations can be provided as plugins (google stackdriver, logly, etc)
2. Hooks mechanism. This is more a pattern than actual code. I'd like to encourage handlers implementations to allow passing hook function as options. For example, onBeforeCallingOrigin function in the origin handler, that will allow me modifying the url or headers when calling origin

[markusahlstrand]

Good point with the loggers. I had one version in the beginning where you could create "named logger" and reference them in handlers. I removed this and instead went with the version of adding the loggers as rules, which felt more flexible. Maybe we could create some hybrid where they are configures using rules but still available in the context somehow..

I'm considering if we should use typescript for a version 2. I got bit of a love/hate relationship with typescript and are not sure if it would make sense to take that dependency?

I think the hooks can be achieved by ordering the rules correctly, but it's not always that easy to work with. The initial idea was to have an UI that could be used to setup all the rules and make the proxy available as a cloudflare app.

The major change in the v2 version would for sure be to change how the handlers are included so that the packaged proxy only includes the handlers that are actually used.

[markusahlstrand]

Think I'll start to break out some middleware to their own packages like you suggested. Considering to write them as normal koa-middlewares which hopefully should work with the proxy as well. It should also work to use with cloudworker router if it's a more custom code worker.

Considering migrating the middlewares to typescript.. Any downside with this?

[danbars]

Only thing about typescript is that you'll have to make sure you have proper build step. Not really a downside, just something to consider so whoever is consuming it will not have to compile it themselves

[tomlarkworthy]

You don't need to split the repository, but have the handlers imported with module syntax so the bundler can eliminate source code not used when deploying. It's quite burdensome looking up npm module names.

[markusahlstrand]

I'm thinking that this project needs a bit of refresh.. Would love to find some time to set this up with:

• Typescript.
• hono.dev instead of a customer router
• Add some storage. Maybe KV-storage? Only issue is the delay in the updates
• Add a UI for admin of router. My current favourite is react-admin
• Custom domain support so you can run it for domains not hosted on cloudflare

Think it should be pretty quick, but need to find a bit of time.

[danbars]

Not trying to demotivate you or anything, but since this project seemed to be abandoned, I moved to using itty-router now. It is well maintained and has a large community.

[markusahlstrand]

I gave the itty-router a try but for me I think the scope is a bit to small. I do think the hono.dev seems to be doing everything the cloudworker-router has done so think I'll switch to using that one instead. Think the proxy project would be interesting to rebuild. The idea was to do something more something more similar to Kong or the api gateway on amazon, so something built on top of a router.

[danbars]

An API gateway is a missing piece in the Cloudflare ecosystem. Leveraging "services" capability, a good API gateway can be used to trigger handlers on different services. Maybe something that accepts an openApi schema, handles input validation, api keys, throttling, permissions validation etc, and then triggers the handler itself - this could be really useful.

On Thu, Aug 31, 2023 at 11:49 PM Markus Ahlstrand @.***> wrote:

I gave the itty-router a try but for me I think the scope is a bit to small. I do think the hono.dev seems to be doing everything the cloudworker-router has done so think I'll switch to using that one instead. Think the proxy project would be interesting to rebuild. The idea was to do something more something more similar to Kong or the api gateway on amazon, so something built on top of a router.

— Reply to this email directly, view it on GitHub https://github.com/markusahlstrand/cloudworker-proxy/issues/72#issuecomment-1701761796, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOVDO5R7QYYHHE7FN4VAITXYD2FNANCNFSM4NO2FK2Q . You are receiving this because you were mentioned.Message ID: @.***>

[tomlarkworthy]

Oh cool its alive. I updated the toolchain for node 18, I can merge all that back here if you want (I was jsut hacking so its not done that nicely) https://github.com/tomlarkworthy/cloudworker-proxy Hono, module imports, de-nodify it by using CF local development and webcrypto etc. and it will be gleaming. There is much less need for node workarounds like dollarshaveclub (which is abandoned) these days. We definitely need something like this, envoy and stuff good but not edge capable, the performance of CF is astounding but nothing really leverages it.

[markusahlstrand]

Yes, sure would be great if you want to merge back. Would be great to have more people working actively on this :)

[markusahlstrand]

An API gateway is a missing piece in the Cloudflare ecosystem. Leveraging "services" capability, a good API gateway can be used to trigger handlers on different services. Maybe something that accepts an openApi schema, handles input validation, api keys, throttling, permissions validation etc, and then triggers the handler itself - this could be really useful. … On Thu, Aug 31, 2023 at 11:49 PM Markus Ahlstrand @.> wrote: I gave the itty-router a try but for me I think the scope is a bit to small. I do think the hono.dev seems to be doing everything the cloudworker-router has done so think I'll switch to using that one instead. Think the proxy project would be interesting to rebuild. The idea was to do something more something more similar to Kong or the api gateway on amazon, so something built on top of a router. — Reply to this email directly, view it on GitHub <#72 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOVDO5R7QYYHHE7FN4VAITXYD2FNANCNFSM4NO2FK2Q . You are receiving this because you were mentioned.Message ID: @.>

We're using TSOA a lot now, automatically generating oas for docs and validation and I really like that way of working. I'm also tinkering with Cloudflare for Saas for the auth-project I'm working on and think we could re-use a lot of parts from there. Most work is currently being done here, but I'll try to merge it back soon: https://github.com/sesamyab/auth

[tomlarkworthy]

yeah before we get too ahead with features, we should remove the rotted dependancies and get onto idiomatic local modern cloudflare dev, including using the module export style of connecting the proxy to the infra (instead of the 'fetch' event listener). The module way is nicer IMHO coz the cloudflare context is passed through as an ordinary function argument and its much easier to type (in the Typescript sense) and do bindings.

[tomlarkworthy]

WDYT? https://github.com/markusahlstrand/cloudworker-proxy/pull/120 Please advise me on your preferred ways-of-working if this PR is not how you would do it.

[markusahlstrand]

Think this is a good first step. I'll see if I can take a stab at doing the first steps of the renovation later today like migrating over to typescript and modernize the structure a bit.

[markusahlstrand]

I created a ticket for the V2 version of this library and will push a PR later today. Would be great with some feedback :)

The ticket is here #121

[markusahlstrand]

After starting to add hono support I realized that a lot of the handlers we had before are available as hono-middlewares. I think I'll scrap the cloudworker-router support and just use the hono-format for the handlers. https://hono.dev/middleware/builtin/basic-auth