PermissionError: [WinError 32]

[deeplearining-sheen]

Hi,

I run as following part, and then I got error message: PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'batches\32891163.case_00010.pickle'

Pls. let me know how to solve it.

Library import

from miscnn.evaluation.cross_validation import cross_validation

Run cross-validation function

cross_validation(validation_samples, model, k_fold=3, epochs=5, iterations=2, evaluation_path="evaluation1", draw_figures=True, save_models=True, run_detailed_evaluation=True, callbacks=[cb_lr])

Epoch 1/5 2/2 [==============================] - ETA: 0s - loss: 2.4823 - dice_soft: 0.1726 - dice_crossentropy: 1.7110 Epoch 00001: val_loss improved from inf to 2.52166, saving model to evaluation1\fold_0\model.hdf5 2/2 [==============================] - 17s 9s/step - loss: 2.4823 - dice_soft: 0.1726 - dice_crossentropy: 1.7110 - val_loss: 2.5217 - val_dice_soft: 0.1605 - val_dice_crossentropy: 1.7466 - lr: 1.0000e-05 Epoch 2/5 2/2 [==============================] - ETA: 0s - loss: 2.4626 - dice_soft: 0.1791 - dice_crossentropy: 1.7857 Epoch 00002: val_loss improved from 2.52166 to 2.51943, saving model to evaluation1\fold_0\model.hdf5 2/2 [==============================] - 16s 8s/step - loss: 2.4626 - dice_soft: 0.1791 - dice_crossentropy: 1.7857 - val_loss: 2.5194 - val_dice_soft: 0.1609 - val_dice_crossentropy: 1.7434 - lr: 1.0000e-05 Epoch 3/5 2/2 [==============================] - ETA: 0s - loss: 2.4510 - dice_soft: 0.1830 - dice_crossentropy: 1.9342 Epoch 00003: val_loss did not improve from 2.51943 2/2 [==============================] - 13s 7s/step - loss: 2.4510 - dice_soft: 0.1830 - dice_crossentropy: 1.9342 - val_loss: 2.5364 - val_dice_soft: 0.1554 - val_dice_crossentropy: 1.6986 - lr: 1.0000e-05 Epoch 4/5 2/2 [==============================] - ETA: 0s - loss: 2.4698 - dice_soft: 0.1767 - dice_crossentropy: 2.3842 Epoch 00004: val_loss improved from 2.51943 to 2.51251, saving model to evaluation1\fold_0\model.hdf5 2/2 [==============================] - 17s 9s/step - loss: 2.4698 - dice_soft: 0.1767 - dice_crossentropy: 2.3842 - val_loss: 2.5125 - val_dice_soft: 0.1609 - val_dice_crossentropy: 1.7242 - lr: 1.0000e-05 Epoch 5/5 2/2 [==============================] - ETA: 0s - loss: 2.4784 - dice_soft: 0.1739 - dice_crossentropy: 1.8819 Epoch 00005: val_loss did not improve from 2.51251 2/2 [==============================] - 20s 10s/step - loss: 2.4784 - dice_soft: 0.1739 - dice_crossentropy: 1.8819 - val_loss: 2.5216 - val_dice_soft: 0.1581 - val_dice_crossentropy: 1.6996 - lr: 1.0000e-05

PermissionError Traceback (most recent call last)

in () 4 cross_validation(validation_samples, model, k_fold=3, epochs=5, iterations=2, 5 evaluation_path="evaluation1", draw_figures=True, save_models=True, ----> 6 run_detailed_evaluation=True, callbacks=[cb_lr]) C:\Users\User\anaconda3\envs\sheen-miscnn\lib\site-packages\miscnn-1.0-py3.6.egg\miscnn\evaluation\cross_validation.py in cross_validation(sample_list, model, k_fold, epochs, iterations, evaluation_path, draw_figures, run_detailed_evaluation, callbacks, save_models, return_output) 82 # Run training & validation 83 history = model.evaluate(training, validation, epochs=epochs, ---> 84 iterations=iterations, callbacks=cb_list) 85 # Backup current history dictionary 86 if return_output : validation_results.append(history.history) C:\Users\User\anaconda3\envs\sheen-miscnn\lib\site-packages\miscnn-1.0-py3.6.egg\miscnn\neural_network\model.py in evaluate(self, training_samples, validation_samples, epochs, iterations, callbacks) 206 # Clean up temporary files if necessary 207 if self.preprocessor.prepare_batches or self.preprocessor.prepare_subfunctions: --> 208 self.preprocessor.data_io.batch_cleanup() 209 # Return the training & validation history 210 return history C:\Users\User\anaconda3\envs\sheen-miscnn\lib\site-packages\miscnn-1.0-py3.6.egg\miscnn\data_loading\data_io.py in batch_cleanup(self, pointer) 173 # IF file matches seed pattern -> delete it 174 if pointer == None and match(str(self.seed) + "\.*", file) is not None: --> 175 os.remove(os.path.join(self.batch_path, file)) 176 # Delete complete batch directory 177 if self.delete_batchDir: PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'batches\\32891163.case_00010.pickle'

[muellerdo]

Hey @deeplearining-sheen,

mhm, interesting bug. Looks like there is still an active process on a temporary preprocessed sample file, which throws an exception when cleaning up the temporary files after finishing the first cross-validation fold.

MIScnn shouldn't has an open process on these files at this point.

What IO interface are you using?

Do you have a third party software running on the data? Like synching with Gdrive or Dropbox?

Cheers, Dominik