search

SIMEXP / psom

pipeline system for octave and matlab
http://psom.simexp-lab.org
Other
24 stars 13 forks source link

Job execution stopped but no Matlab error #134

Open brosscle opened 5 years ago

brosscle commented 5 years ago

Hello,

When I try to execute a pipeline with hundreds of jobs, each processing the same basic operation (clip an image), around 250 jobs run perfectly, and the function then returns to the initial Matlab code, without execute the remaining jobs. The lines returned in my matlab console are : Caught "std::exception" Exception message is: fl:filesystemPathNotFound

Please note that it is not a Maltab error that would interrupt the initial script, it just stop the execution without stopping the workers. Then my cores are still working and my only way to escape is to reboot my computer or to trigger the command killall MATLAB

Do you have any idea about the cause of this issue? I believe it is either PSOM or my local machine that causes it.

Thank you for any advice !

pbellec commented 5 years ago

hi @brosscle

Are you using PSOM 1 or the latest release?

We have stopped testing PSOM on Matlab after the 1.X version, and even the 1.X would need to be tested with recent versions...

The error does not look familiar. Because it deals with path you may want to make sure you specify full paths to the pipeline, rather than relative, e.g. '/home/toto/pipeline' and not 'pipeline'

brosscle commented 5 years ago

Hi @pbellec, thanks a lot for your quick answer.

Unfortunately yes, we use the latest release of PSOM. We developed a GUI that automatically generate jobs thanks to a database containing metadata for each scan, and the paths are absolutes. I will investigate the features of my computer, as I don't find any relevant answers on the internet.

I will keep you informed if I discover the issue. Thanks again !

pbellec commented 5 years ago

You may want to try with https://github.com/SIMEXP/psom/releases/tag/v1.0.2 The API is almost identical, although some details of the logs are different, and the execution engine is completely different.

There is clearly some work to be done for PSOM 2 to run on Matlab. If you want to try and work on this issue, I would be happy to provide feedback and incorporate changes back to the library.