jrmadsen
commented
4 years ago What is an easy way to replicate what opencv is doing with CV_TRACE_REGION_NEXT?
Well, normally, adding a new component for another marker API is pretty straight-forward, e.g. the ones that already exist for Caliper, TAU, VTune, NVTX, etc. but the way OpenCV implements it makes it more difficult than normal.
- The underlying object OpenCV uses (
cv::utils::trace::details::Region) has an extremely restricted interface:Region(const LocationStaticStorage& location)and that is it. - Constructing
LocationStaticStorageis quite complicated. SeeCV__TRACE_DEFINE_LOCATION_andCV__TRACE_DEFINE_LOCATION_FN.
Currently, doing this will require a heap allocation until I add a new bundle variant for handling components with this sort of non-default-ctor restriction.
Basically the component would look like this:
TIMEMORY_DECLARE_COMPONENT(opencv_trace)
#if !defined(USE_OPENCV)
TIMEMORY_DEFINE_CONCRETE_TRAIT(is_available, component::opencv_trace, false_type)
#endif
#if defined(USE_OPENCV)
namespace tim
{
namespace component
{
struct opencv_trace : public base<opencv_trace, void>
{
using region_t = CV_TRACE_NS::details::Region;
opencv_trace() = default;
~opencv_trace() = default;
opencv_trace(const opencv_trace&) = delete;
opencv_trace(opencv_trace&&) = default;
opencv_trace& operator=(const opencv_trace&) = delete;
opencv_trace& operator=(opencv_trace&&) = default;
static std::string label() { return "opencv_trace"; }
static std::string description() { return "Provides OpenCV tracing API"; }
void start(const LocationStaticStorage& _loc)
{
if(!m_data)
m_data.reset(new region_t{ _loc });
}
void stop() { m_data.reset(); }
private:
std::unique_ptr<region_t> m_data = {};
};
}
}
#endifAnd now you can pass LocationStaticStorage to the start(...) function of the bundle. But you'll need something to reproduce this construction of LocationStaticStorage, like this:
namespace // ensure instantiation is unique to each compiled file
{
template <size_t, char> // template on line number and first char of name
const auto&
get_cv_trace_define_location_fn(const char _name[], const char _file[], int _line,
int _flags)
{
// from CV__TRACE_DEFINE_LOCATION_
static CV_TRACE_NS::details::Region::LocationExtraData** _data = nullptr;
static const CV_TRACE_NS::details::Region::LocationStaticStorage _inst =
= { &(_data), _name, _file, _line, _flags };
return _inst;
}
} // namespace
#define CV__TRACE_INLINE_LOCATION_FN(name, flags) \
get_cv_trace_define_location_fn<__LINE__, name[0]>( \
name, CV_TRACE_FILENAME, __LINE__, \
(flags | CV_TRACE_NS::details::REGION_FLAG_FUNCTION))and finally, usage would resemble this (which could be hidden behind a macro:
tim::component_tuple<opencv_trace> _obj{ "opencv_trace_example" };
_obj.start(CV__TRACE_INLINE_LOCATION_FN("opencv_trace_example", 0));Another question related to how to transition from the opencv tracing macros to timemory: opencv makes a distinction between tracing a function and tracing a region. Is such a distinction helpful?
If you look at the#define CV__TRACE_REGION(...) vs. #define CV__TRACE_OPENCV_FUNCTION() in core/utils/trace.hpp you will see that a CV__TRACE_OPENCV_FUNCTION is essentially just CV__TRACE_REGION(__FUNCSIG__).
How do I use timemory to count user defined events?
There is a templated tim::component::data_tracker<Tp, Tag> where Tp is the data type you want to track and Tag is just an arbitrary struct to differentiate component X that tracks ints and component Y that tracks ints. You basically put it in a bundle and call the store(...) function.
It can be quite useful to create a dedicated auto_tuple bundle for the just the components which track data bc you can append to the history in a single line bc the auto_* bundles call start() and stop() at construction/destruction.
This is probably best demonstrated with an example.
using namespace tim::component;
// component_tuple requires explicit start, stop is optional (will call stop if started when it gets destroyed)
using bundle_t = tim::component_tuple<opencv_trace, data_tracker_integer>;
// auto_tuple automatically calls start and stop
using tracker_t = tim::auto_tuple<data_tracker_integer>;
// auto_* bundles will write to stdout when destroyed
tim::settings::destructor_report() = true;
bundle_t _obj{ "opencv_test" };
_obj.start();
for(int i = 0; i < 10; ++i)
{
long ans = fibonacci(10) + fibonacci(10 + (i % 3));
// store a single iteration
tracker_t{ TIMEMORY_JOIN("", _obj.key(), "#", i % 3) }.store(ans);
// join macro is like pythons "#".join(...)
// accumulate into parent bundle and demonstrate using lambda
// specifying how to update variable
_obj.store([](long cur, long upd) { return cur + upd; }, ans);
}The result:
>>> opencv_test#0 : 110 data_integer [laps: 1]
>>> opencv_test#1 : 144 data_integer [laps: 1]
>>> opencv_test#2 : 199 data_integer [laps: 1]
>>> opencv_test#0 : 110 data_integer [laps: 1]
>>> opencv_test#1 : 144 data_integer [laps: 1]
>>> opencv_test#2 : 199 data_integer [laps: 1]
>>> opencv_test#0 : 110 data_integer [laps: 1]
>>> opencv_test#1 : 144 data_integer [laps: 1]
>>> opencv_test#2 : 199 data_integer [laps: 1]
>>> opencv_test#0 : 110 data_integer [laps: 1]
[data_integer]|0> Outputting 'timemory-ex-derived-output/data_integer.json'...
[data_integer]|0> Outputting 'timemory-ex-derived-output/data_integer.tree.json'...
[data_integer]|0> Outputting 'timemory-ex-derived-output/data_integer.txt'...
|---------------------------------------------------------------------------------------------------------------------------------------------------------|
| STORES SIGNED INTEGER DATA W.R.T. CALL-GRAPH |
|---------------------------------------------------------------------------------------------------------------------------------------------------------|
| LABEL | COUNT | DEPTH | METRIC | UNITS | SUM | MEAN | MIN | MAX | STDDEV | % SELF |
|---------------------|------------|------------|--------------|------------|------------|------------|------------|------------|------------|------------|
| >>> opencv_test | 1 | 0 | data_integer | | 1469 | 1469 | 1469 | 1469 | 0 | 0 |
| >>> |_opencv_test#0 | 4 | 1 | data_integer | | 440 | 110 | 110 | 110 | 0 | 100 |
| >>> |_opencv_test#1 | 3 | 1 | data_integer | | 432 | 144 | 144 | 144 | 0 | 100 |
| >>> |_opencv_test#2 | 3 | 1 | data_integer | | 597 | 199 | 199 | 199 | 0 | 100 |
|---------------------------------------------------------------------------------------------------------------------------------------------------------|Can I use timemory to to profile how timing changes over the duration of a long running process?
So I am currently working on getting a support for timemory in a python project called Hatchet which essentially takes the *.tree.json output files and converts them into a Pandas dataframe for extended analysis. Also, in the longer term, we will be developing a framework for performance unit testing which will automate historical comparisons but, at present, the roll-your-own-solution would be to just import multiple jsons into Python and do a comparison there. The regular .json files (w/o .tree) provide the results in a flat JSON array so it is relatively easy to traverse (the json trees require recursion to process).
Also, you can specify TIMEMORY_INPUT_PATH=/path/to/some/folder/of/results + TIMEMORY_DIFF_OUTPUT=ON in the environment (or the corresponding tim::settings::input_path(), etc.) and when the results are finalized, timemory will search that input folder, try to find a corresponding input, and then generate additional *.diff.* files in the output folder with the difference between any matches it can find.
For my usage, I need to run multiple processes simulateneously , and I need to have each process have a unique output path.
One options is to prefix the files with some key (such as the PID):
tim::settings::output_prefix() = std::to_string(tim::process::get_id()) + "-";Another options is to specify the TIMEMORY_TIME_OUTPUT=ON in the env (tim::settings::time_output()) and that will create subdirectories which are time-stamped. This time-stamp gets fixed for the process the first time settings::get_global_output_prefix() is called and time-output is enabled. Also, there is a time_format() in settings which allows you to customize the time-stamping format using the strftime format.
pwm1234-sri
Now that I have a working windows build of timemory, I have a few usage questions.
CV_TRACE_REGION_NEXT? See Tracing macro list for details, but it is simply a way to easily use regions for different steps.So far I have only used the library api functions
timemory_push_regionandtimemory_pop_region. But I believe this is more easily done using the timemory c++ api, but I am not sure how to do this or where to look for guidance.Another question related to how to transition from the opencv tracing macros to timemory: opencv makes a distinction between tracing a function and tracing a region. Is such a distinction helpful? If so, how do I make this distinction with timemory? (If it matters, my primary backend profiling/analysis tool is VTune.)
How do I use timemory to count user defined events? For example, with an image processing application, I would like to count the number of images read, written, processed, displayed, copied, etc. Can timemory help me keep track of these events?
Can I use timemory to to profile how timing changes over the duration of a long running process? Again for an image processing application that runs for 7 days, how do I get a timeseries of profiling measurements so I can see how performance changes over time? For example, do image reads take longer on day 7 hour 24 than they did on day 1 hour 1?
Documentation suggestion:
For my usage, I need to run multiple processes simulateneously , and I need to have each process have a unique output path. Of course, with timemory this is possible, but it took me a little while to figure out how. Here is my summary code snippet. Is there a place in the documentation to put a tip like this? I would like to contribute if it is useful and I know where to put it.
Thanks again for your help!