[BUG] CNMF-E correlation and PNR batch process not working

[EricThomson]

Describe the bug When trying to get correlation/pnr inspection plot in cnmf-e pipeline, it doesn't work (I get an error)

To Reproduce Steps to reproduce the behavior, for example:

1. caiman toolbox -> cnmfe -> inspect correlation and pnr --> add to batch
2. within the batch manager window try to run the item (start at selection).
3. It runs and then turns red and I get the following stack trace:

   
   status: 0
   output_info: Traceback (most recent call last):
   File "c:\users\lab_user\miniconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\batch_run_modules\CNMFE.py", line 143, in run
   os.remove(mf)
   PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'F:/mesmerize_projects/mesmerize_testing/batches/motion_correction_testing\\memmap-2f70cc3a-a35f-4338-a240-b531f3b1d23d_d1_600_d2_537_d3_1_order_C_frames_999_.mmap'

output: 2f70cc3a-a35f-4338-a240-b531f3b1d23d output_files: ['2f70cc3a-a35f-4338-a240-b531f3b1d23d_pnr.pikl', '2f70cc3a-a35f-4338-a240-b531f3b1d23d_cn_filter.pikl'] Y.shape: [999, 600, 537] processing_time: 1.0310599684715271

**Expected behavior**
corr/pnr plot to come up like in caiman jupyter and mesmerize docs.

**Copy-paste or take a screenshot of the terminal that Mesmerize is running in**
![image](https://user-images.githubusercontent.com/6643322/119026783-8f362500-b973-11eb-8a68-8b651485dafa.png)

**Operating System & specs (CPU, RAM etc.). Please complete the following information:**
Windows 10

**Details about your Mesmerize install**
caiman installed with mamba, mesmerize with pip.

[kushalkolar]

@EricThomson Thanks again for providing all the details!

Strange, I think this might be something peculiar with how windows handles processes & permissions since I've never seen this on linux (we've ran hundreds of batches). This is just a cleanup step where it cleans up the memmap files when its done.

I've tested Mesmerize somewhat on windows, but not as much as on linux.

For now you can put a try except block around the for loop on line 142 in: mesmerize/viewer/modules/batch_run_modules/CNMFE.py

a simple sleep(5) or something before the for loop might work too

Meanwhile I'll try to reproduce this on a windows machine.

[EricThomson]

That worked. I wrapped the for loop with this:

        sleep(5)
        try:
            for mf in glob(batch_dir + '/memmap-*'):
                os.remove(mf)
        except PermissionError as pe:
            print(f"Sorry can't clean up your mess: Permission error: {pe}")
        except Exception as e:
            print(f"Can't clean up: {e}")

This time it worked -- it hit the PermissionError as before but continued on without any other mishaps:

Sorry can't clean up your mess: Permission error: [WinError 32] The process cannot access the file because it is being used by another process: 'F:/mesmerize_projects/mesmerize_testing/batches/motion_correction_testing\memmap-60fe4e80-0a39-43fc-a0e1-67c66f72aa11_d1_600_d2_537_d3_1_order_C_frames999.mmap'

I'll leave this issue open so you can mess around with it.

[kushalkolar]

I was able to reproduce it, I'm not exactly sure what process would be using the memap file since the dview instance is closed, I'll put the try except "fix" in the next release. Thanks for reporting it! I guess you'll just have to periodically remove memmap files from your batch dir or workdir every once in a while until I can figure out a proper fix.