How do i benchmark or measure the time with the python-ffi?

[mzguntalan]

First of all, thank you for the language/compiler. I enjoy coding in it!

In python, there is a timeit utility and it's pretty handy. I would like to know the fair way of measuring the run speed interacting through python.

I used the python-ffi compiled using cuda. I pass the futhark arrays instead of numpy (RAM) so that it doesn't measure the copying.

My question

1. Does futhark, cache results? Because i run the code with timeit, with the same values.
2. When it returns a value through the ffi, is the calculation actually done? JAX in python, needs a .block_until_ready() since the calls are asynchronous.

Thank you!

[athas]

With the caveat that I didn't write futhark-ffi originally and don't remember all of how it works...

1. No, Futhark does not memoize duplicate computation.
2. Most Futhark entry points are asynchronous (although depending on what they do, most of the work may still be done before they return). Skimming the futhark-ffi code, it does not look like it wraps entry points in any synchronisation.

It looks like futhark-ffi does not actually expose the futhark_context_sync() function from the C FFI, although it uses it internally when mapping Futhark arrays to Numpy arrays.

[mzguntalan]

Thank you! I was able to find futhark_context_sync(self.ctx) in the Futhark python object.