Document and commit to decision around edxapp middleware spans

robrap commented 2 months ago

During the New Relic work, we temporarily disabled spans for middleware. @timmc-edx mentioned he'd like to leave it this way.

AC:

[ ] Create ADR for disabling (or enabling) middleware instrumentation in edxapp.
[ ] Update comments around disabling, or ticket re-enabling, depending on decision.
[ ] Communicate the decision to other teams.

Notes/Questions:

Middleware instrumentation is currently disabled in edxapp, and likely enabled in other services, so we can still screenshot the differences.
What do we lose without the middleware spans when working on or debugging middleware issues or errors?
What are the workarounds if we leave this disabled? Could it makes sense documenting temporarily enabling in an environment if we are working on a middleware issue?

robrap commented 2 months ago

@timmc-edx: This is another one that probably should be assigned to you, so you have the chance to make the case for leaving this disabled. Thoughts?

timmc-edx commented 2 months ago

I don't know that it needs to be assigned to me—I don't have strong feelings around it, and I think anyone can make the decision. I'll just link this from 692 for now.

robrap commented 2 months ago

Thanks @timmc-edx. I unassigned you. That said - I thought you had an opinion somewhere to leave this disabled. If you do have any thoughts, you can document them in a comment on this ticket.

timmc-edx commented 2 weeks ago

Added commentary to https://2u-internal.atlassian.net/wiki/spaces/ENG/pages/1008500894/Datadog+configuration+and+setup+details

timmc-edx commented 11 hours ago

Communicated in observability Slack channel: https://twou.slack.com/archives/C048L3Y9RDG/p1730233003973139

I'm not sure we need an ADR for this, with the wiki docs in place. If we don't, then this issue is probably complete.

edx / edx-arch-experiments

Document and commit to decision around edxapp middleware spans #778