search

astrorama / SourceXtractorPlusPlus

SourceXtractor++, the next generation SExtractor
https://astrorama.github.io/SourceXtractorPlusPlus/
GNU Lesser General Public License v3.0
72 stars 9 forks source link

ONNX does not run with slurm #482

Open mkuemmel opened 2 years ago

mkuemmel commented 2 years ago

Since we are using the ONNX runtime for the ML detection we suffer from a problem between ONNX and slurm: https://github.com/microsoft/onnxruntime/issues/10736

Some comments from Alejandro:

It seems to be C++ code

http://www.xavierdupre.fr/app/onnxruntime/helpsphinx/api_summary.html#onnxruntime.SessionOptions.intra_op_num_threads

It would have to be added here I guess
https://github.com/astrorama/SourceXtractorPlusPlus/blob/2823f0c939a5464d4793994c71367c24b2aa32db/SEImplementation/src/lib/Common/OnnxModel.cpp#L23

Not sure how to expose that to the user of sourcextractor++, though, because there are a *lot* of options.
mkuemmel commented 1 year ago

I stumbled again over this one, meaning I was wondering why a job did not start until I remembered this issue.

Shall we fix this one variable to 1 to have a solution for slurm and see how far we come?

Slurm is quite commonly used, and there seems to be no issue with other framewrorks such as SGE (used at SDC-DE).