data61 / anonlink-entity-service

Privacy Preserving Record Linkage Service
Apache License 2.0
26 stars 8 forks source link

Improve opentracing #499

Open hardbyte opened 4 years ago

hardbyte commented 4 years ago

Misc issues when trialing jaeger tracing.

1) I've seen the warning "invalid parent span" for the following tasks at various times (not always):

image

2) Aggregate comparisons task doesn't include project or run id information:

image

3) Uploading CLKS don't seem to be linked to the worker task that handles the uploaded data:

image

hardbyte commented 4 years ago

We might not be initializing the tracer correctly due to the WSGI forking - https://github.com/jaegertracing/jaeger-client-python#wsgi-multi-processing-fork2

hardbyte commented 4 years ago

More nice to haves:

hardbyte commented 4 years ago

During debugging I note that the parent_span of our tasks have an uber-trace-id

wilko77 commented 4 years ago

opentracing-instrumentation natively supports celery since release 3.1. That might help...

hardbyte commented 3 years ago

I've been using opentelemetry, exporting to jaeger on another project. These are the dependencies I'm using:

jaeger-client = "^4.3.0"
opentelemetry-api = "^0.17b0"
opentelemetry-sdk = "^0.17b0"
opentelemetry-instrumentation = "^0.17b0"
opentelemetry-instrumentation-flask = "^0.17b0"
opentelemetry-instrumentation-celery = "^0.17b0"
opentelemetry-instrumentation-sqlalchemy = "^0.17b0"
opentelemetry-exporter-jaeger = "^0.17b0"
opentelemetry-instrumentation-redis = "^0.17b0"
opentelemetry-instrumentation-psycopg2 = "^0.17b0"
opentelemetry-instrumentation-requests = "^0.17b0"