search

ilastik / ilastik

ilastik-shell, applets, and workflows to string them together.
https://ilastik.org
Other
344 stars 130 forks source link

Incomplete CPU usage during pixel classification export #1304

Open imagejan opened 8 years ago

imagejan commented 8 years ago

While exporting probability maps from a pixel classification workflow, CPU usage is only ~38%, see attached screenshot:

taskmanager_while_exporting_probmaps

On earlier versions (e.g. ilastik-1.1.5), this used to be nearly 100% CPU usage. Tested on Windows 7 with 16 cores.

akreshuk commented 8 years ago

@imagejan, are you exporting to tiff? I've been testing last week and I have observed the same behaviour when exporting to tiff, not to hdf5.

imagejan commented 8 years ago

@akreshuk thanks for the quick reply!

No, this is exporting to h5 to a network location mounted on 10Gbit/s network. I wondered if this might actually be an I/O issue, but I'm able to export to .h5 from Fiji with the HDF5 plugin to the same location in (almost) no time, so I don't think I'm bandwidth limited.

stuarteberg commented 8 years ago

@imagejan, Can you give me some details about the input file? Specifically, the file type, the shape, and the axes (t-xy v.s z-xy, etc.). If you're able to provide a sample input file and .ilp project file, that would be even better, but I'll make do with what you can tell me.

imagejan commented 8 years ago

@stuarteberg sorry for the delayed follow-up. I was reading a tiff stack (tzyx) that was written from Fiji/ImageJ. I figured that the speed was increased when I copied the file into the project instead of using a relative link. However, we also saw some performance issues when running in batch processing and in headless mode. But I'll have to check back the exact conditions and carefully compare to the 1.1.5 version where we (presumably) didn't have these issues. I'll report back with more information and an example project as soon as I find the time.

Reuschse commented 4 years ago

I am having the same issue. CPU usage stays constant at round 20%. I trained a couple of stacks (xyz) with autocontext and am trying to batch process this now, while saving the output as hdf5.

I am running this in Ilastik 1.3.3.post2 on a Windows machine with Windows 10, 64GB RAM and Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz, 2095 Mhz, 8 Core(s), 16 Logical Processer(s).

Are there known issues or a way to increase CPU to 100% to speed things up?