Adding Focus mechanism for on-line focus measurements

[beniroquai]

We recently discussed the ability to measure the focus of a system using IR + Quadrant diodes. I had this ESP32 camera flying around and found out that you can do image processing on the chip. (See ./ESP32/ESP32_CAM_FindcenterBeam/ESP32_CAM_FindcenterBeam.ino/

the idea is to point an IR Laser e.g. scrapped from a CD drive and relate the deflection to the focus position

Formular should be:

dz=0.5*Nx*d_pix/sin(alpha)

The ESP32 can detect the Infrared laser. Now we "only" need to fit a guassian to the laser position:

the value will be sent to the Raspberry Pi to control the PWM signal for the lens.

[beniroquai]

And it seems to be working - I didn't expect that the ESP32 microprocessor is capable of doing so much cool image processing. Unfortunately, the IR laser burned. Anyway, that's what the masked detection of the laser looks like as a function of Z:

@theosanderson do you think that is a bit too much for this? In theory, you have real-time focus ability without any image processing on the Pi.

[beniroquai]

Here with red laser, MTP and ESP32 camera:

[theosanderson]

Wow super cool!

So from my application point of view this is probably overkill and I might not invest the extra time but again really really don't focus on my application! :)

[RainerHeintzmann]

Kann das Diagram auf dem Mobiltelefon nicht sehen. Du beleuchtest maximal schräg? Kann man das Signal auf der Camera noch besser fokussiert bekommen?

---

Rainer Heintzmann Institute of Physical Chemistry, Friedrich-Schiller-Universität Jena, Helmholtzweg 4, 07743 Jena, Germany Leibniz Institute of Photonic Technology, Albert-Einstein Str. 9, 07745 Jena, Germany Tel.: +49 (0) 3641 206 431

Am 01.12.2020 um 20:55 schrieb Benedict Diederich notifications@github.com:

And it seems to be working - I didn't expect that the ESP32 microprocessor is capable of doing so much cool image processing. Unfortunately, the IR laser burned. Anyway, that's what the masked detection of the laser looks like as a function of Z:

@theosanderson do you think that is a bit too much for this? In theory, you have real-time focus ability without any image processing on the Pi.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[beniroquai]

So, some good news. It works ;-)

The recent commit the ESP32 can successfully detect the focus position of the MTP:

My Max Val: 61733 at pos: 18
My Max Val: 61207 at pos: 10
My Max Val: 61454 at pos: 10
My Max Val: 58079 at pos: 10
My Max Val: 57146 at pos: 10
My Max Val: 57391 at pos: 10
My Max Val: 56601 at pos: 10
My Max Val: 60351 at pos: 10
My Max Val: 64783 at pos: 13
My Max Val: 64687 at pos: 14
My Max Val: 63944 at pos: 14
My Max Val: 64300 at pos: 15
My Max Val: 63152 at pos: 16
My Max Val: 62332 at pos: 15
My Max Val: 62465 at pos: 18
My Max Val: 61247 at pos: 20
My Max Val: 61239 at pos: 21
My Max Val: 61229 at pos: 21
My Max Val: 61256 at pos: 22
My Max Val: 61254 at pos: 21
My Max Val: 61300 at pos: 21
My Max Val: 61264 at pos: 20
My Max Val: 61298 at pos: 20
My Max Val: 61188 at pos: 20
My Max Val: 61195 at pos: 21
My Max Val: 60812 at pos: 21
My Max Val: 60741 at pos: 21
My Max Val: 60614 at pos: 21
My Max Val: 59833 at pos: 21
My Max Val: 59595 at pos: 21
My Max Val: 57634 at pos: 21
My Max Val: 57427 at pos: 21
My Max Val: 58830 at pos: 21
My Max Val: 59734 at pos: 21
My Max Val: 61239 at pos: 20
My Max Val: 61533 at pos: 17
My Max Val: 61999 at pos: 14
My Max Val: 61576 at pos: 20
My Max Val: 60984 at pos: 11
My Max Val: 61243 at pos: 11
My Max Val: 63088 at pos: 38
My Max Val: 62603 at pos: 72
My Max Val: 63043 at pos: 90
My Max Val: 62802 at pos: 113
My Max Val: 61361 at pos: 125
My Max Val: 59043 at pos: 144
My Max Val: 57472 at pos: 157
My Max Val: 53924 at pos: 176
My Max Val: 52047 at pos: 181
My Max Val: 50569 at pos: 186
My Max Val: 47344 at pos: 198
My Max Val: 45333 at pos: 213
My Max Val: 44010 at pos: 228
My Max Val: 42608 at pos: 0
My Max Val: 37786 at pos: 0
My Max Val: 42524 at pos: 84
My Max Val: 43008 at pos: 79
My Max Val: 43562 at pos: 84
My Max Val: 44284 at pos: 81
My Max Val: 44840 at pos: 72
My Max Val: 45272 at pos: 75
My Max Val: 46009 at pos: 90
My Max Val: 47118 at pos: 81
My Max Val: 48319 at pos: 89
My Max Val: 49433 at pos: 83
My Max Val: 50684 at pos: 82
My Max Val: 52685 at pos: 239
My Max Val: 54733 at pos: 239
My Max Val: 57159 at pos: 239
My Max Val: 58410 at pos: 228
My Max Val: 58938 at pos: 219
My Max Val: 59761 at pos: 210
My Max Val: 60365 at pos: 201
My Max Val: 60940 at pos: 193
My Max Val: 61921 at pos: 182
My Max Val: 62849 at pos: 172
My Max Val: 63105 at pos: 163
My Max Val: 63316 at pos: 155
My Max Val: 64397 at pos: 145
My Max Val: 64792 at pos: 132
My Max Val: 65376 at pos: 111
My Max Val: 65268 at pos: 102
My Max Val: 65426 at pos: 88
My Max Val: 64707 at pos: 83
My Max Val: 64062 at pos: 69
My Max Val: 62643 at pos: 58
My Max Val: 62149 at pos: 49
My Max Val: 60761 at pos: 39
My Max Val: 60391 at pos: 29
My Max Val: 59245 at pos: 18
My Max Val: 58179 at pos: 10
My Max Val: 56163 at pos: 10
My Max Val: 54835 at pos: 10
My Max Val: 54249 at pos: 10
My Max Val: 54582 at pos: 10
My Max Val: 54301 at pos: 10
My Max Val: 55568 at pos: 10
My Max Val: 55750 at pos: 10
My Max Val: 59028 at pos: 10
My Max Val: 62649 at pos: 10
My Max Val: 63476 at pos: 38
My Max Val: 64254 at pos: 59
My Max Val: 64834 at pos: 86
My Max Val: 64042 at pos: 101
My Max Val: 63053 at pos: 114
My Max Val: 61571 at pos: 122
My Max Val: 60573 at pos: 128
My Max Val: 58821 at pos: 137
My Max Val: 58264 at pos: 138
My Max Val: 57890 at pos: 136
My Max Val: 58857 at pos: 130
My Max Val: 59981 at pos: 113
My Max Val: 59432 at pos: 100
My Max Val: 59193 at pos: 86
My Max Val: 57466 at pos: 71
My Max Val: 55982 at pos: 61
My Max Val: 54726 at pos: 47
My Max Val: 54193 at pos: 30
My Max Val: 54156 at pos: 11
My Max Val: 53204 at pos: 10

It sums all Values along one direction, applies a guassian kernel to smooth the signal and finds the maximum. Brut-force:

This is a live-demo:

No we somehow need to have a nice routine to combine the sensor output (Serial?) from the ESP32 with the focus position of the Raspberry Pi. Any ideas what would be the easiest? Once the focus position is found using e.g. max contrast it directly relates to the focus position from the ESP32.

[beniroquai]

I wanted to improve the focus by having the camera fixed and the laser moving along with the objective lens. Not so smart. Better: Move everything. Laser, Lens, Camera. Then you just need to keep the reflected beam always at the same spot on the camera and you will be in-focus all the time. Simple Focus upgrade for every long-working distance lens:

This came in handy. It just weights some 2g. (Sorry, it'S amazone..)

[beniroquai]

Looks promising: I manually moved the microwellplate up and down with greatest precision ;-)

The key was to add a aluminium foil + pinhole in front of the laser + red gel filter in front of the camera (deep red, LEE filters).

Now it looks like that:

I will try writing some basic code to stay in focus for every scan position. Let's see how well this goes..

Also cool: The esp32 camera from M5Stack has an LED Pin which can direclty be used for the LED strips. So no additional Arduino anymore.

All wrapped up:

[beniroquai]

Good news, a simple feedback mechanism holds the focus accros the plate. Focusdetection range could be something around +/- 1mm .

Some result:

[oeway]

Good news, a simple feedback mechanism holds the focus accros the plate. Focusdetection range could be something around +/- 1mm .

@beniroquai Great work! Could you explain what we are looking at in the gif above?

[beniroquai]

Hey, good question ;-)

It is a xy-scan of some HeLa cells on a MTP well plate:

The microscope goes through the well plate step by step, checks focus, readjusts and takes an image. It's not updated for speed yet, but I'm surprised that it works so well to be honest.

A "hands-on-video" which shows the setup is this one:

The Raspberry Pi controls the Microscope and receives commands by the focus sensor.