imfuse: finding good parameters for focus stacking

[holgerd001]

This issue is opened to test, discuss, improve, ... the results of imfuse. Many parameters are possible, many parameters can have arguments and some parameters can be combined reasonable and other not. Some hints can be found here: https://github.com/mviereck/microscopy-tools/tree/master/fusesets

Suggestion for a workflow:

1. take photos
2. align them (e.g. align_image_stack)
3. fuse them (imfuse)
4. convert tif to jpg (magick) (optional)
5. transfer exif data (exiftool) (if exif data is in the aligned images, imfuse do that for you)

[holgerd001]

In the past I have used enfuse and got more or less good results with enfuse -o result.tif --exposure-weight=0 --saturation-weight=0 --contrast-weight=1 --hard-mask --gray-projector=l-star input_????.tif

In the hope imfuse will be faster and/or provide better results I started working with imfuse.

Until now I haven't found a parameter set for imfuse that produces a result as good as enfuse do. Most results have halos. So I started this issue and continue experimenting with different parameters. Within the next days I'll send some results.

Any suggestion witch parameters should be combined or witch not?

Greetings Holger

[mviereck]

Most results have halos. Any suggestion witch parameters should be combined or witch not?

Your moth example has a black background. Likely the halos appear there? To avoid halos in black background, it helps to add a weighted saturation mask, automatically with --saturation=w or manually with e.g. --saturation=w15. Example:

imfuse --log --saturation=w15  IMAGES

• To get an impression of the effect of the saturation mask, run imfuse --saturation IMAGES.
• The result will have clear colors without halos, but with a lack of detail in the motive itself.
• Halo effects mostly appear in low contrast areas.
• Combining e.g. --log with --saturation=w15 will apply the saturation mask only in areas where the contrast strength of --log is below 15%.
  • Just writing --saturation=w will set a default weight value fitting the first specified mask option.

Further examples with --saturation in https://github.com/mviereck/microscopy-tools/blob/master/fusesets/macro-soft-less-sat . These more complex examples also add options less,close,soft.

• less and close help to strengthen objects that are a bit weak and are partially painted over by objects that are in fact behind them.
• soft generates a transparent result in low contrast areas so the background image shines through.
• The default background with soft is enfuse, an enfuse result. For dark images it makes sense to try dark backgrounds, too, e.g. --background=min or --background=saturation

In the hope imfuse will be faster and/or provide better results I started working with imfuse.

Likely imfuse will forever be slower than enfuse. sigh enfuse already gives pretty results. imfuse wants to improve these points:

• enfuse gives very soft results. Especially with microscopic captures the result is too weak. imfuse wants to create more sharp and detailed results. In fact, I am no satisfied with any available stacking tool if it comes to microscopic captures. I'd call imfuse the best one for this job. A good one is picolay.
• Macroscopic objects can be a bit too soft with enfuse. imfuse wants to add more detail.
• Thin objects in foreground can be overwritten by strong objects in background, e.g. hairs. imfuse wants to preserve those details. (Options less and close).

In areas with strong contrast imfuse gives pretty nice results with most provided mask options. Currently I work on further solutions for low contrast areas. Current solutions are described above; I hope to find even better (and easier) solutions.

[mviereck]

An attempt to write a guidance that might result in a wiki article:

Remove artefacts in low contrast areas

Currently there are basically two ways to handle low contrast areas with artefacts like halo and noise:

• First way: --soft mode with close to transparent low-contrast areas. A background image shines through.
  • Default background for --soft is --background=enfuse.
    • Check the look of different backgrounds e.g. with imfuse --background=max IMAGES. As no mask is specified, only the background is generated and fully visible. Try backgrounds min, max, mean, enfuse saturation.
  • Example: imfuse --diffstat --soft=25 --background=min
  • A high value for --soft up to 100 makes the mask very transparent. Low values make it more opaque. Senseful values depend on the mask option and can be in a range between 1 (e.g. --sobel) and 75 (e.g. --morphology).
• Second way: Mixing a contrast mask like --diffstat with a weighted image mask.
  • Example: imfuse --diffstat --saturation=w15.
    • w15 darkens the saturation mask to 15% of its strength. It is mixed with the --diffstat mask. The saturation mask will only take noteable effect where the contrast strength of the --diffstat mask is below 15%. This way the saturation mask hits the noisy area without harming the well contrasted area.
• Combined way:
  • Example: imfuse --diffstat=soft25 --saturation=w15 IMAGES
    • Instead of global option --soft the soft mode is specified for the --diffstat mask only. This ensures that the saturation mask takes full effect without being made transparent by --soft.

Strenghten weak foreground objects

Some objects in foreground like thin hairs might have less contrast significance than the main object behind. In that case they are not completely drawn and might look transparent, broken, or disappear at all.

• The option to fix this is --less.
  • --less=n% draws weaker objects if they are stronger than n% than the unsharp range since the last strong contrast. Senseful values for n% differ depending on the chosen mask option.
    • Example: imfuse --diffstat --less=5 IMAGES.
  • In most cases it makes sense to combine --less with --close. --close fills gaps between nearby contrast lines and marks them as strong contrast, too.
    • Example: imfuse --diffstat --less=5 --close=2x2 IMAGES
  • --less and --close can be specified for single masks instead of globally.
    • Example: imfuse --diffstat=less5,close2x2 --saturation=w15 IMAGES

General recommendations:

• Install image viewer geeqie and always run with options -vV to see what is going on.
• Use option --cache while trying several parameter sets. This allows imfuse to re-use some files. Delete cache.imfuse manually once you are done.
• Big images can be resized to be smaller to have a faster test set.
• Recommended masks for macroscopic stackshots: --log --dog --wavelet --diffstat=r2 --freichen=15
• Recommended masks for microscopic stackshots: --diffstat=r5 --freichen --sobel --morphology --comet --compass

[holgerd001]

Combining e.g. --log with --saturation=w15 will apply the saturation mask only in areas where the contrast strength of --log is below 15%.

* Just writing `--saturation=w` will set a default weight value fitting the first specified mask option.

To get an overview about --log and --saturation here is what I have done:

1. Taken photos

2. align_image_stack -v -C -c80 -m --use-given-order -a aligned_c80_C_morder

3. enfuse -o xD2040938_aligned_c80_C_m_order_grayl-star.tif --exposure-weight=0 --saturation-weight=0 --contrast-weight=1 --hard-mask --gray-projector=l-star Now I have an image "xD2040938_aligned_c80_C_m_order_grayl-star.tif" to compare with.

4. resize the aligned images to be faster for file in aligned_c80_C_m_order_00??.tif; do convert $file -resize 20% pc20-$file; done

5. using imfuse several times with variations of --log and ----saturation e.g. imfuse -o imfu_log_satw65_pc20.tif --log --saturation=w65 pc20-aligned_c80_C_m_order_00??.tif

6. save bandwith and mem: mogrify -format jpg imfu_*.tif

imfu_log_pc20.tif

imfu_log_satw5_pc20.tif

imfu_log_satw10_pc20_c80_C_m.tif

imfu_log_satw15_pc20_c80_C_m.tif

imfu_log_satw25_pc20.tif

imfu_log_satw35_pc20.tif

imfu_log_satw45_pc20.tif

imfu_log_satw55_pc20.tif

imfu_log_satw65_pc20.tif

imfu_sat_pc20.tif

Made by emfuse (xD2040938_aligned_c80_C_m_order_grayl-star_pc20.jpg):

Most of the images generated by imfuse have black pixels in the light part right and left from the eyes of the fly. If I use the non resized aligned images as input the number of these black pixels decrease (imfu_log_satw15_aligned_c80_C_m.jpg).

Nevertheless the result of emfuse looks more 'natural'.

Next I use less, close and soft.

[holgerd001]

1. take photos

A short explanation about the way I take photos.

Inspired by 'traumflieger' I build up a lens to get more than a magnification of 1:1. It's made of a camera + adapter (mft-Minolta) + bellows + adapter (Minolta-M42) + adapter (M42-RMS) + microscope lens (Kern Plan 4/0.10 160/0.17)

[mviereck]

Thank you for showing your camera setup! Quite interesting. I have an unused microscope, maybe I'll build something similar, too, with one of the microscop objectives. Do you manually adjust the steps with the wheels?

Thank you for sharing the imfuse results. They are quite unsatisfying, --saturation=w does not help as expected. With --soft you should get much better results.

What does the shortcut c80_C_m mean? Btw, if you don't specify an output name, imfuse creates a name that includes all chosen options and automatically set default values.

[holgerd001]

Do you manually adjust the steps with the wheels?

Yes. And this can be a problem, because it's not very sensitive. A hundredth rotation of the adjusting knob can change a wide range of depth of field (DOF). Actual I need a macro slide or something like that. Perhaps a positioning slide like turning lathes have.

What does the shortcut c80_C_m mean?

That are the parameters align_image_stack has created the output files with. -c80 -C -m -m Optimize field of view for all images, except for first. Useful for aligning focus stacks with slightly different magnification. -C Auto crop the image to the area covered by all images -c num Number of control points (per grid) to create between adjacent images (default: 8)

Btw, if you don't specify an output name, imfuse creates a name that includes all chosen options and automatically set default values.

That's what I do too.

I have an unused microscope, maybe I'll build something similar, too, with one of the microscop objectives. Here are some hints:

• have a look at traumflieger.de
• 4x magnification is enough (at first)
• it should be a lens for a limited microscope tube. Here 160mm. Afaik you need an additional lens, if you take an infinity lens (objective).
• you can save one adapter, if you take a M42 ballows
• you can buy a M42-RMS or M42 M26F adapter at traumflieger.de. If you need a different adapter or just simply want to test if it works, you can make an adapter by yourself.

Take a small part of plywood. Saw a circle that is a little bit to big to fit into the M42. Drill a hole that is a little bit to small for your thread of the lens. Use abrasive paper for the acuteness. The thread of the lens and of the M42 part is much harder then the wood. So the thread is self-cutting. The internal side should be black. It's not a high precision adapter, but it works.

[mviereck]

Thank you the hints!

And this can be a problem, because it's not very sensitive. A hundredth rotation of the adjusting knob can change a wide range of depth of field (DOF).

It might help if you add a gear wheel and rotate it with a gear spiral. (Not sure if I translated right: Zahnrad und Schnecke, beides von fischertechnik). Here a picture of my microscope; the spiral is driven by a stepper motor, the gear wheel is glued to the microscope fine tuning wheel. The glue is Heißkleber, strong enough but easy to remove. Turning the spiral by hand instead with the stepper motor also allows very fine rotation of the gear.

Or just one big wheel glued on the small knob of your slide would already allow finer rotation by hand, without using a spiral.

Actual I need a macro slide or something like that.

You could also use a cheap old microscope as macro slide. If the microscope moves its table, you can remove the microscope tube and use your current setup. Alternatively you could somehow mount the camera onto the tube. Will be tricky to find the right distance to get a sharp projection onto the chip.

[holgerd001]

It might help if you add a gear wheel and rotate it with a gear spiral. ...

Thanks for this hints. I think about something similar. Mounting two runners (Schubladenschienen) parallel on a wooden board. An other board where the camera gets mounted on) becomes mounted on the runners. Than a threaded rod beside to move top board. But in the moment it's to cold to work in the workshop.

You could also use a cheap old microscope as macro slide.

That's an interesting idea. The microscope I have needed to be cut (by saw). And the microscope stage is to small to mount a camera on. The insects I take photos from are needled. I use a 'third hand' to keep it in position. It's difficult to mount the 'third hand' on the microscope stage. But I'll think about.

[mviereck]

I've added two new options that reduce noise: --cutblur and --finalblur. I am still trying out their effects.

Your stack above might give a good result with:

imfuse --dog=s0.3 --soft=25 --less=6 --cutblur=0.5

imfuse --dog=s0.3 --finalblur=w10,s1 --less=6 --cutblur=0.5

imfuse --log=s0.3  --finalblur=w10,s1

[mviereck]

I've made a change in the mask generation that impacts --soft, --less and --finalblur. They need much higher values than before, but are now the same across all mask options. I am still checking out, but as a start I recommend --soft=100, --less=60, --finalbur=w75,s1. (Or, as mask arguments, like --log=soft100,less60.)

[mviereck]

I've made several changes the last few days, and meanwhile some commits had new introduced bugs; I should have checked better before uploading. I hope you didn't had issues because of that. Now all seems to work well again. For your image above, I assume this setup will do a good job:

imfuse --wavelet --soft --less --close 

With current default values, this is the same as:

imfuse --wavelet=p10 --soft=100 --less=60 --close=2x2 --background=enfuse

[holgerd001]

imfuse --wavelet --soft --less --close

Yes, impressiv. As I compare it with the result from enfuse --gray-projector=l-star I see:

• higher contrast, more crisp
• a little bit pixelated in the light area left of the left eye and left of the mouth
• more details
• different bokeh

And I noticed, that I have to improve the lighting. At the moment I need two small LED lamps with curtain in front of to make the light softer. But I assume it's not soft enough.

[mviereck]

Yes, impressiv.

I am happy to hear this. :-)

a little bit pixelated in the light area left of the left eye and left of the mouth

Likely that can be fixed. --cutblur should help, but makes the image softer overall. (With --cutblur you can omit --close). --finalblur is worth a try, but less effective along with --soft. Higher values for --soft might help, too. (Yet I am looking at the --soft code again, found likely an issue that can cause some false results or detail loss. So the value range might change again.)

Edit: pixelated areas can also be a result of .jpg quality loss, e.g. while resizing the image. I always convert to .tif first before doing any other image processing.

And I noticed, that I have to improve the lighting. At the moment I need two small LED lamps with curtain in front of to make the light softer. But I assume it's not soft enough.

You might get slightly better light distribution with option --fakehdr that makes dark areas brighter and bright areas darker. Of course, no computer technique can achieve or outweight the quality of a good capture.

Some setups can be found in www.focusstackingforum . They often use something like a semitransparent white tube around the object with light sources outsight of the tube. That results in diffuse light inside.

[holgerd001]

Higher values for --soft might help, too.

It seems --soft is 100 by default. I got imfuse.wavelet=p10_soft100_less60_close2x2_bg=enfuse.fc1e5b.tif using the defaults (imfuse --wavelet --soft --less --close).

[mviereck]

It seems --soft is 100 by default.

You can set values above 100, too.

[holgerd001]

You can set values above 100, too.

Ok, I thought that means percentage and 100 means 'all'. I tried 150 but the differential is marginal.

[mviereck]

Ok, I thought that means percentage and 100 means 'all'.

Percent value is right, but here allowing values above 100% and below 0%

I've changed the argument handling of --soft yet. It is hard to explain in words, so I've opened a new ticket #9 that shows an example table.

[mviereck]

I am working on tutorials and examples in the wiki: https://github.com/mviereck/microscopy-tools/wiki