search

pulumi / pulumi-azure-native

Azure Native Provider
Apache License 2.0
125 stars 34 forks source link

Provide a simple mechanism to log all HTTP requests/responses #886

Open mikhailshilkov opened 3 years ago

mikhailshilkov commented 3 years ago

Troubleshooting often requires logging the HTTP payloads that are sent to and from the Azure API. We can log those with extended debug options. However, the output is very large and noisy, which makes it hard to be analyzed by end users.

Should we provide a separate option to produce a log file that is just HTTP and nothing else? It could be an extra config value passed to the native provider.

t0yv0 commented 3 years ago

OR label the output with a keyword so one can grep the large noisy log to filter to this log?

viveklak commented 3 years ago

As long as we can standardize this across providers (at least native providers), I am very much in support of either approach.

t0yv0 commented 3 years ago

Some more notes here. The AppDash traces I've been working with currently have spans for all Service calls and gRPC calls. These spans carry log annotations. I am working on a utility to do app dash trace file -> csv (or plain text) conversion so it's easy to extract these. I'm also working to propagate traces so you could use OpenTracing API to add spans in providers. That feels like it could be a reasonable carrier for this information:

  1. I can run stats on the CSV to find e.g. total time taken by the HTTP requests to Azure or total payload size
  2. I can query and find specific invocations (by response status or latency) and look at their payloads
  3. AppDash UI is not the best but is available right away to visualize the request sequence
viveklak commented 3 years ago

While extending tracing to provider internals is hugely valuable, the specific issue here I believe is more around the ability to introspect on requests and response bodies to help debug issues. This should complement the work on tracing - for instance annotating debug log entries with trace/span ids. But I think we need an easy to flip switch for users to dump the request bodies without all the other cruft independently.

t0yv0 commented 3 years ago

That' fair. It's somewhat unusual that our traces currently carry request bodies for gRPC etc calls. I am guessing this is all operating under the assumption that pulumi traces are "small enough" so we can cram it in. This might not continue to hold true.

t0yv0 commented 3 years ago

And I see your UX point. Thanks.

viveklak commented 3 years ago

That' fair. It's somewhat unusual that our traces currently carry request bodies for gRPC etc calls. I am guessing this is all operating under the assumption that pulumi traces are "small enough" so we can cram it in. This might not continue to hold true.

Ah I forgot that request bodies are in the traces. That is indeed unusual.