search

iansmith / parigot

Develop as a monolith, deploy as microservices.
https://parigot.info
BSD 3-Clause "New" or "Revised" License
12 stars 1 forks source link

The log problem: how to structure it and other future builtins #1

Closed iansmith closed 1 year ago

iansmith commented 1 year ago

The Problem

We have a logging infrastructure that is called Pctx (and pctx and PCtx and this array of names probably indicate something is wrong). Pctx was originally envisioned as a replacement for context.Context but language neutral so it could be used in languages like python or rust. You can read more about the three primary features of Pctx here.

Although all these goals are all laudable, I think the logging part should be removed from the Pctx world and converted to a system provided service. This would be more "regular" with the RPC-based model of parigot and would allow users to formulate drop in replacements. This service would be the first truly parigot- provided service and it has revealed some naming problems.

Currently, we have multiple "projects" that we coordinate via the go.work file at the repo root. The ones that are important for this discussion are:

You will notice that each of these has a go.mod at the nominal root of the project like sys/atlanta.base/go.mod.

Note that something seems to be broken in api right now and that go.mod has gone missing but I'll likely put it back soon.

The api defines the services provided by paragot (such as the problematic logging service related to this ticket), lib/atlanta.base/go is the client side library that users of parigot should link into their go programs. sys/atlanta.base is the implementation of the code that runs inside wastime and provides the system call interface--which should be language independent.

The problem hinges on 3 key issues:

  1. When you write a .proto file you cannot import files from a different project (in the sense of those from the previous paragraph) with a path like ../../api/atlanta.base/... so the implementation of the api must be in the api project. This is poor if you are trying to have the api directory/project just be .proto files that user code can reference from any language and have various implementations in the lib project/directory.
  2. The entire point of the go.work feature of go is to work with multiple repositories at the same time. You can do go build github.com/iansmith/parigot/api/... to build the API project, for example. However, the fact that all these directories/projects are actually part of the same repo (github.com/iansmith/parigot) makes me feel like I should not be using this feature of go in this way. It seems like it is intended more for totally different repositories, like perhaps github.com/iansmith/parigot and github.com/iansmith/parigot-sample-programs.
  3. We need language neutrality in some of these cases (api, to some extent sys) but not all.

So the problem is that we to do some kind of a reorganization of how the repository is structured so that we can provide sample implementations of services and not prevent users from creating their own implementations of the services as well. The fact that logging is one that many users may want to replace with their own version has brought this to a head now.

iansmith commented 1 year ago

Fixed this with the new generics code. Also, now both the kernel and split mode services use the same codepath.