Open picca opened 3 years ago
@picca: I'd need to know more details about how onException
is used. Do you have a link to a sample repository or minimal code example
However, note that a large number of allocations does not necessarily mean having a large heap footprint. The only way to know that for sure is to collect a heap profile by following the instructions here:
Hello,
here the repository of the code https://repo.or.cz/hkl.git/tree/6c35373943fb58c92e28f60a53b819cca900304f:/contrib/haskell
the function I am speaking about is located in this file
This required also a non hackages distributed hdf5 binding located here.
https://github.com/picca/hs-hdf5
in here you can find the h5e_try function, which could maybe interact with the onException
If you need more informations, I can provide them.
Cheers
@picca: I probably won't have time to dig further into this, but my advice is to collect the heap profile by following the instructions I linked to earlier. That will give a clearer picture of whether or not the program is running in constant space or not, and if it is not it will also highlight which parts of the code are holding onto memory
Hello, I discovered an error in my C code which increased a lot the memory usage.
I would like your advice about the exception handling.
framesHklP h5d det
contain in fact a bunch of
bracket ...
bracket ...
bracket ....
The exception I want to deal with is during the acquisition of the ressources. If something went wrong, I would like to skip the current ressource and process the next resource generated via each job
.
I am not sure how to do this.
@picca: I believe one way to handle that is to wrap each resource acquisition with a catch
statement that will ignore the exception if you don't want the exception to be fatal.
In other words, if you have something like:
example = do
bracket acquire0 release0 $ \resource0 -> …
bracket acquire1 release1 $ \resource1 -> …
…
Then you can wrap each resource block in something like this:
ignore :: IO () -> IO ()
ignore io = io `catch` handler
where
handler :: SomeException -> IO ()
handler _ = return ()
example = do
ignore $ bracket acquire0 release0 $ \resource0 -> …
ignore $ bracket acquire1 release1 $ \resource1 -> …
…
Thanks a lot for your answer.
@picca: You're welcome! 🙂
Hello, I have one more question. suppose that I read a value from my file and depending of this value to not yield a value.
Should I encode this logic with if then else
or is there a special value which can be send downstream to say, skip this ?
@picca: You can do something like:
x <- readSomeValue
if someFunctionOf x
then yield x
else return ()
This works because return ()
does nothing (i.e. emits no value)
Hello, I am writing a program which read its data from an hdf5 file. for each images of this file it compute soimething and add it into an IORef wchi is is in fact a C objects managed via FFI
the interesting part of the code is here
Pipe Safe is used to managed the resources, (open and close files), In my case I open 10 times the same file (for test) My program use a lot's of memory and it seems that most of this memory is in the onException method.
here part of the profilling informations. all the hkl_xxx are FFI functions (time consuming)
So I would like your help in order to investigate this issue.
This program should use a constant amount of memory independently of the number of images red. But this is not the case, the more image, the bigger the onException is.
thanks for considering