search

typelevel / feral

Feral cats are homeless, feral functions are serverless
Apache License 2.0
168 stars 41 forks source link

natchez integration and lambda init #233

Open skirsdeda opened 2 years ago

skirsdeda commented 2 years ago

There's a problem when using tracing in combo with resources that are cached (in init or setup part of execution), which is even included in one of the examples: https://github.com/typelevel/feral/blob/main/examples/src/main/scala/feral/examples/KinesisLambda.scala#L52-L53.

The problem:

  1. Actual root span cannot be used in init because it will never be closed. This is even worse if you reuse Trace[IO] instance inside request handler code, because all spans there would become children of this never-ending root span. And some natchez backend implementations (like XRay) only send root span together with all children when root span is finished. So you get no recorded spans as a result.
  2. If you use fake root span to solve the first problem (smth like this: https://github.com/tpolecat/natchez/issues/566#issue-1278510665), then spans produced during init (like DB connection pool initialization with skunk) are separate from spans produced in the first request (handler code) of that lambda instance.

This means that TracedHandler implementation is useless if you need to have Trace[IO] during init. In case of skunk, you would want to see SQL query spans inside request spans and for that you need to reuse same instance of Trace[IO] for all requests.

How it should work:

It should be possible to build a single Trace[IO] instance for the lifetime of lambda instance. Spans produced during initialization would get recorded as children of root span for the first ever request to that lambda instance.

I hope I managed to describe this clearly 😅. Would be great to work this out and get that example fixed so that it would show how to use skunk and tracing with feral.

armanbilge commented 2 years ago

Thanks for writing this up! Indeed, there are still some things to work out with Tracing :)

If I understand correctly, I think you've run into a version of problem (2) as described in https://github.com/typelevel/otel4s/issues/11#issuecomment-1159464303. See also bayou for some exploratory work at improving the situation.

In the short term, instead of using IO-based tracing, perhaps Kleisli based tracing would work here?

armanbilge commented 2 years ago

@skirsdeda did you have any more luck with this? Any ideas or feedback you have would be very valuable, thanks!

skirsdeda commented 2 years ago

@armanbilge I am working on it. I think I am somewhere around half way through at this point. Should have smth to show next week.