Open siju-samuel opened 6 years ago
Todos:
Proposed API design: '_tvm.contrib.graphruntime' extend with two interfaces '_debugrun' and _'injectvalue'. '_debugrun' can execute a specific node only if all the inputs are ready. Stepper will be invoked with '_invokestepper' from '_tvm.tools.debug.wrapper.uiframework' based on the user run option. '_invokestepper' in '_tvm.tools.debug.wrapper.uiwrapper' create 'DebugStepper' class (in _tvm.tools.debug.ui.uistepper) for Stepper UI and handlers. '_tvm.tools.debug.runtime.debugruntime' uses _tvm.contrib.graphruntime to create below stepper interfaces:
A wrapper interfaces layer will be created in _tvm.tools.debug.wrapper.uiwrapper for the above interfaces. Based on 'DebugStepper' user events, stepper runtime interfaces will be called through '_tvm.tools.debug.wrapper.uiwrapper'
OBJECTIVE Support a debugging tool for TVM's computation graphs which helps to access internal graph structures, ops, input and output values at TVM runtime.
In TVM's current computation-graph framework, computation after graph construction happens as part of Python function(
graphruntime.run
). Basic Python debugging tools such as pdb cannot be used to debug graphruntime.run because TVM's graph execution happens in the underlying C++ layer. C++ debugging tools such as gdb are not ideal either, because of their inability to recognise and organise the stack frames and variables in a way relevant to TVM's operations, tensors and other graph constructs.Runtime debug will fulfil the below objectives.
TODOs
Proposed API Changes
tvm.contrib.graph_runtime.create
add a new Boolean flagdebug
to make the runtime debug-gable, this API will be exposed to user to enable or disable debug functionality. Inclass GraphModule
two membersdebug
anddbgobj
are added.debug
flag will store whether the debug for this is enabled or not anddbgobj
holds the object ofdebugruntime
(including the ui framework)tvm.contrib.graph_runtime.set_inputs
is modified to pass the inputs data set from script to thedebugruntime
if thedebug
flag is enabled.tvm.contrib.graph_runtime.run
is modified to invoke the_debug_cli_run
which will bring up the ncurses framework. ncurses framework will wait for actual user-input for the run operation. once user gives the input, will invoke theruntime.GraphRuntime.DebugRun()
ingraph_runtime.cc
if user select to run with debug. Otherwise usualruntime.GraphRuntime.Run()
ingraph_runtime.cc
is invoked. 'DebugRun' can execute a specific node only if all the inputs are ready.c_runtime_api.h
is modified to add new struct to hold the output information.tvm.contrib.graph_runtime.set_debug_buffers
this new api is introduced to collect the run output of each node. InGraphRuntime
a new fieldstd::vector<TVMDbgTensor*> debug_buffers_;
is introduced to store the pointers of output buffers.After each operation execution is completed
runtime.GraphRuntime.DebugRun()
the output is copied to the debug buffer and the outputs are dumped to a temporary directory. UI framework will read this outputs from the temporary directory and will show in the display.tvm.contrib.graph_runtime.inject_value
used to inject a node tensor value during the execution Stepper functionality is supported to run each node by node. Stepper will be invoked with 'invoke_stepper' from 'tvm.tools.debug.wrapper.ui_framework' based on the user run option.invoke_stepper
intvm.tools.debug.wrapper.ui_wrapper
createDebugStepper
class (intvm.tools.debug.ui.ui_stepper
) for Stepper UI and handlers.tvm.tools.debug.runtime.debug_runtime
usestvm.contrib.graph_runtime
to create below stepper interfaces:A wrapper interfaces layer will be created in
tvm.tools.debug.wrapper.ui_wrapper
for the above interfaces. Based onDebugStepper
user events, stepper runtime interfaces will be called throughtvm.tools.debug.wrapper.ui_wrapper
TVMDBG profiler can be used for profiling the model based on TVM kernels. The objective is to provide the execution time of each graph node and map its source in the TVM kernels. This can be used to identify the time consuming nodes and analyse its kernel source. This helps identify the areas to be analyse more to optimise.