search

ROCm / ROCm-CompilerSupport

The compiler support repository provides various Lightning Compiler related services.
45 stars 31 forks source link

Debug intermediaries mode #3

Closed stuartarchibald closed 5 years ago

stuartarchibald commented 5 years ago

Given that this library is currently buffering input/output through real files on the file system, it'd be useful to have a mode (perhaps set by an environment variable) whereby the intermediate files generated and consumed by the API are left in their /tmp for later inspection and debugging.

scott-linder commented 5 years ago

I think this is a reasonable ask, although I believe @t-tye mentioned a desire to avoid inspecting environment variables in the library. Would it be OK to have this as an option at library compile time?

We are also planning to remove all filesystem access from the library by default, but this option could then be changed to save temporaries to the filesystem even if the work is done in-memory.

stuartarchibald commented 5 years ago

By: Would it be OK to have this as an option at library compile time?

Do you mean, at the time the comgr library is itself compiled, or at compile time within the library, as in, when using an action kind that is a compilation variant?

If the former, having code that is a direct consumer of the library (whilst having actually tried to build from source and conda package it, not with success as yet) I'm of the view it is preferable to be able to set this at run time. Should users run into trouble with the Numba ROC target (or indeed some other consumer target), being able to ask users to switch on a debug mode via some method would make accessible the input data for the bug/problem they hit so it can be easily uploaded for analysis.

If the latter, then yes, great, thanks!

We are also planning to remove all filesystem access from the library by default, but this option could then be changed to save temporaries to the filesystem even if the work is done in-memory.

This is a really useful thing to have, it's been implemented in a couple of variations in the Numba ROC target. Most useful is having a command line implementation that dumps to files, and then also library/in-memory based implementations that do the same, and then being able to compare the outputs to validate the library is behaving as the command line.

scott-linder commented 5 years ago

This is implemented in f331bad by reading an environment var named AMD_COMGR_SAVE_TEMPS