search

microsoft / perfview

PerfView is a CPU and memory performance-analysis tool
http://channel9.msdn.com/Series/PerfView-Tutorial
MIT License
4.1k stars 698 forks source link

Allow Stream as input (vs filepath) #1709

Open brandonh-msft opened 1 year ago

brandonh-msft commented 1 year ago

When doing processing in batch/bulk, it would be more advantageous to allow the user to pass a System.IO.Stream into new ETWTraceEventSource() vs having to download the file to disk and pass the filename in.

We are attempting to do processing in the cloud via an Azure Function and so downloading to disk is blocking this scenario.

pharring commented 1 year ago

This is a fair request, but unfortunately there's no good alternative for ETW files. The root of the problem is that the Win32 API for opening ETW files, OpenTrace takes a file path.

@brianrob and I have explored alternatives. For example, I've heard that there is (or soon will be) a new Windows API we can use, but it probably won't be available in Azure Functions for a while. We're also aware of another team doing custom processing of the ETL file format (so it can be done within a Linux container, for example). That would be fragile and unsupported, so unlikely to make it into this open source project.

I'm in the same boat, BTW. A service I operate processes ETW files (from Azure Blob storage) in Azure Functions too. Contact me internally if you like and we can trade ideas.

pharring commented 10 months ago

This request has come up again, but this time for .netperf/.nettrace files. I believe this is more achievable since EventPipeEventSource has a constructor that takes a Stream.

brianrob commented 10 months ago

This request has come up again, but this time for .netperf/.nettrace files. I believe this is more achievable since EventPipeEventSource has a constructor that takes a Stream.

Agreed for EventPipeEventSource, though we'd need to do some due diligence to see if there are any streaming-specific paths and address them, since this constructor expects to have the data live-streamed (e.g. do we disable any features because we're streaming).

cc: @noahfalk, @davmason

noahfalk commented 10 months ago

e.g. do we disable any features because we're streaming).

As I recall EventPipeEventSource has the same behavior regardless whether the Stream is backed by a file or a stream. One spot that might show a little variation is that either the constructor is going to block until the header is read or some of the properties that return stream info (like the process ID) aren't going to have a value until that header is received. There may also be some perf differences where the file-backed code will read ahead and cache but when stream-backed it avoids doing that.

brianrob commented 10 months ago

Thanks @noahfalk. @pharring it sounds like you might already be able to use the existing EventPipeEventSource constructor that takes a Stream.