search

LBEM-CH / focus

High Throughput Electron Microscopy Image Processing Software
http://www.focus-em.org
GNU General Public License v2.0
35 stars 14 forks source link

Problems in finding images upon reopening focus #226

Open JMJB opened 6 years ago

JMJB commented 6 years ago

Dear Focus-Team, we notoriously have the problem in finding the projects image data. In more detail: 1) all works fine when we set up a new project. 2) When we then reopen the project zero images are found although folders and images exist.

What could be the cause of this problem?

Kind regards Jean-Marc

JMJB commented 6 years ago

Dear Focus-Team, we are still facing the same problem.

Kind regards

aseedb commented 6 years ago

Hi, We found the origin of the problem. The projects were saved on a CIFS share, which does not support symbolic links. The raw data can stay on the CIFS share but the project folder must be in a linux partition. Is there any option to disable symbolic links in focus? Is it possible to set the output directory on a share that does not support symbolic links?

sthennin commented 6 years ago

Hi, this is a difficult problem. FOCUS uses symbolic links in various situations. For example, we used the ending .mrc for 2D images and .mrcs for stacks of 2D images. However, some programs only can work with files ending with .mrc, but still should run on the stacks. So, we create a symbolic link from the .mrcs to a file called "dummy.mrc". We then call that program with the file name "dummy.mrc". When that program then opens "dummy.mrc", it in fact opens the .mrcs. If your file system doesn't allow symbolic links, then in all such situations, the files should be renamed or moved first, then used, and then renamed back. That would be work, but is doable. The search in the scripts folder of the FOCUS source code, using fgrep -R "ln -s" * reports 54 instances, where the scripts create symbolic links.

However, a bigger problem is the handing of the "GLOBAL" parameters: FOCUS uses some parameters that are local to the image, such as the "imagename" or the "defocus" for that image. These parameters are stored in the image folder in a file called "2dx_image.cfg". But other parameters are global to that project, such as the protein name (e.g., "ABCG2") or the choice of parameters that you want to use for the call of "MotionCor2", such as "- Patch 5 5". These parameters are stored in a file called "2dx_merge.cfg", which resides in the "merge" directory of that project. The file structure then is: /auto/IMAGE001/2dx_image.cfg ===> local parameters /auto/2dx_master.cfg -> ../2dx_master.cfg ===> symbolic link /2dx_master.cfg -> merge/2dx_merge.cfg ===> symbolic link to global parameters /merge/2dx_merge.cfg ===> global parameters

If a script or program then needs any parameter, then FOCUS first searches in the /2dx_image.cfg for entries. If these are not found, then it searches in /../2dx_master.cfg for the remaining entries. Since that ../2dx_master.cfg points always to the correct 2dx_merge.cfg file, it will always find the correct parameter settings.

If your file system now doesn't allow symbolic links, then the entire parameter handling in FOCUS has to be re-written. That would be a big mess, where we would have to work deep in the C++ code.

You could replace the links /auto/2dx_master.cfg with copies of the file /merge/2dx_merge.cfg but then each time you or a program change a global parameter, you would have to copy that 2dx_merge.cfg file onto 2dx_master.cfg, which is not practical.

Since nobody else seems to have had that problem so far: Could you not somehow work with FOCUS on a linux partition?

Henning.

aseedb commented 6 years ago

Hi Henning, Thank you for your detailed reply! Don’t worry about that, we will buy some more disk for the Linux system. We can still sniff images from the FEI image server but instead of writing output on it, we'll do it on the Linux drive. Benoît