cortex-lab / Suite2P

Tools for processing 2P recordings
Other
121 stars 66 forks source link

Error: copy tiff failed, retrying--->> #105

Closed johannasigl closed 6 years ago

johannasigl commented 6 years ago

Dear all, I get this error during registration:

"Set 1, tiff 36 done in time 5752.12" "copy tiff failed, retrying...>>".

Any idea of what might be the problem? Cheers!

carsen-stringer commented 6 years ago

We've set this up so that if the copying fails, it will try copying the tiff again after a wait period (in case you briefly lose connection to a server which stores the data). This isn't necessarily an error if your data is indeed being copied over a network connection. Is that the case?

johannasigl commented 6 years ago

Actually its a local hard drive.. I have had the same issue several times with this recording session, will try again. But its on "retrying" for more than 12h now.

carsen-stringer commented 6 years ago

Yeah it will keep retrying forever. The copy command is a command that your operating system executes. Check that the paths are correct to all your tiffs, and that you have permission to copy them. The paths that Suite2P finds to your tiffs are in ops.fsroot . Check that these are correct.

You can also check if there's a corrupted tiff (although this likely is not the problem). You can open tiffs 37-41 in imageJ and see if they have any issues.

hummuscience commented 6 years ago

I am having the same problem. tiff file is a .tiff stack of 13k images taken with MetaFlour, loaded into ImageJ and exported a single 4g .tiff stack (metaflour does not output tiff files).

The root directory is on the local hard-drive of a virtual machine (the virtual c:/ of the windows virtual machine). I tried it previously with a network drive that was mounted into the same windows virtual machine but that also did not work with the same error.

Any idea what can be checked?

johannasigl commented 6 years ago

For me it was one faulty tiff stack out of my 100 recordings that I had to remove. I still don't know what was wrong but it couldn't copy this one.

carsen-stringer commented 6 years ago

You can try running it without copying locally, it just may be slower. In your master_file, set ops0.temp_tiff = []; and it shouldn't attempt to copy it.

You can also try to send the copy command through matlab outside of Suite2P and debug it copyfile('othertiff','temptifflocation')

it's in a try, catch loop right now so you won't see the error message while running suite2P

hummuscience commented 6 years ago

This is what I am getting as error now after turning off local copying:

Warning: TIFF library warning - 'TIFFReadDirectory:  Unknown field with tag 50838
(0xc696) encountered.' 
> In Tiff (line 651)
  In nFramesTiff (line 9)
  In reg2P (line 163)
  In run_pipeline (line 38)
  In master_file_example (line 101) 
Warning: TIFF library warning - 'TIFFReadDirectory:  Unknown field with tag 50839
(0xc697) encountered.' 
> In Tiff (line 651)
  In nFramesTiff (line 9)
  In reg2P (line 163)
  In run_pipeline (line 38)
  In master_file_example (line 101) 
carsen-stringer commented 6 years ago

Is this a warning or an error? What is the format of your images before you convert them to tiffs? I don't know if anyone else is using suite2p in the way you've described (creating tiffs using imageJ). I know that tiffs created using matlab will work fine in suite2p, and we have a script to do this in the repository.

Another alternative is that this is because your tiffs are larger than 4gb and sometimes matlab has issues with these. As a first pass could you try splitting your data into two tiffs?

hummuscience commented 6 years ago

These seem to be errors, Suite2P continues running after them. The format is Metafluor-spefific but ImageJ can read it via bioformats and convert it to tiff.

Turning off the copying and reducing the size (skipped a frame, down to 5Hz and 1.9Gb total size) does the trick.

carsen-stringer commented 6 years ago

you can also split the data into two tiffs if you don't want to lose temporal resolution. suite2p will look for all tiffs in the specified directory.