Use with a stupid camera

Hello. I wonder if it is very hard to add a stupid camera mode. I mean use the image captured by any capture device in the system like a webcam or analog capture device and just translate pixels into microns using a fixed ratio. I mean that we have just an analog microscope with hand-rotated gauges, just added a camera there for better convenience and capturing pictures.

Hi. As far as I understand, you want to use a webcam (or similar devices) as a substitute for a "real camera" , and then to be able to control this webcam together with other devices via YouScope. In principal, adding such a feature is possible, and I long ago also thought about it to simplify debugging. One possibility would be to add a new camera device type, which is essentially nothing else than a wrapper around the TWAIN API (https://en.wikipedia.org/wiki/TWAIN ). Every device which supports the TWAIN standard (many webcams, but also scanners) could then be used as a substitute for a "real camera". This task can be roughly split up into several steps: (i) Writing a functions which discovers (via the TWAIN API) all currently installed TWAIN devices; (ii) Writing a function which takes the name/identifier of a TWAIN device as an input, and connects to this device; (iii) Writing a function which makes an image with a connected TWAIN device, and returns the image e.g. as an array of bytes; (iv) Mapping this functionality to YouScope. I don't know how experienced you are in programming, specifically, in C/C++. In principal, steps i-iii have nothing to do with YouScope, and I would have no advantage trying to accomplish them only because I know YS. For step iv, I have a clear advantage. Thus I could offer to do step iv if you or somebody else would take over step i-iii and send me the source code (C/C++). I have currently a lot to do, and would thus not know when I would find time if I would have to do everything on myself...sorry.

Hi. As far as I understand, you want to use a webcam (or similar devices) as a substitute for a "real camera" , and then to be able to control this webcam together with other devices via YouScope

Not quite, it's pretty uncontrolable, it just sends the picture. I want the following workflow: 1 I put something of known size on the glass. 2 I draw an interval over the image with a mouse and input its size. 3 The software remembers the ratio. We call it calibration. 4 Then I can draw intervals over an image and measure their sizes. 5 There should be multiple ratios remembered, since the device is driven manually. For example the minimum zoom and the maximum zoom level. 6 For the continious magnifying the software should estimate the zoom by matching the contents of the neibouring frames to each other, for example with SIFT, or other descriptors and compute the affine transform (there exists an example how to do it in OpenCV).

(i) Writing a functions which discovers (via the TWAIN API) all currently installed TWAIN devices; (ii) Writing a function which takes the name/identifier of a TWAIN device as an input, and connects to this device; (iii) Writing a function which makes an image with a connected TWAIN device, and returns the image e.g. as an array of bytes;

I wonder if OpenCV highgui module has it. I have used it with a webcam some time ago.

The simplest thing which I forgot: There is already a TWAIN driver in YS/MM! Just follow the documentation provided at https://micro-manager.org/wiki/TwainCamera . If you are lucky, it might work (if you are asked for a camera ID, a good guess is to take "0", but you might also simply try all numbers from 0-9...) OpenCV also seems to support it, but I didn't try: https://docs.opencv.org/3.4.1/d8/dfe/classcv_1_1VideoCapture.html . The example at the bottom of the page looks like they internally use TWAIN.

Considering your workflow: Are you sure that for what you want to do a microscope control software like YS (or any other) is actually the right kind of application? Essentially, your workflow rather sounds like a custom GUI around some custom image processing task with the additional feature to get an image from a camera (either continuously or when you press a button). That this camera is actually connected to a completely manual microscope seems to be a rather minor detail. The point is, there exist specialized software to do microscope control (e.g. YS), and specialized software to do image processing (e.g. ImageJ). Even though YS offers some elementary functionality to look at images, zoom & draw some lines, this is definitely not YS main purpose. Similar to this elementary support of image processing in YS, most specialized image processing software has some elementary support for capturing images from a camera, scanner, or similar. It seems that such elementary support would be totally sufficient for you since the rest of the microscope is anyway completely manually controlled, if I understand completely.

So, without wanting to convince you not to use YS, you might for this specific task consider to use

ImageJ: https://imagej.nih.gov/ij/plugins/webcam-capture/index.html
Matlab: https://www.mathworks.com/help/supportpkg/usbwebcams/ug/snapshot.html
Python using OpenCV or similar: https://stackoverflow.com/questions/11094481/capturing-a-single-image-from-my-webcam-in-java-or-python
C++ using OpenCV
Java/Jython: https://stackoverflow.com/questions/276292/capturing-image-from-webcam-in-java

Essentially, all of this software allows you to take images from a Webcam or Scanner, do some image processing and to build a GUI around it. YS is mainly written in Java, so if you would want you could write a plugin for YS (I can help with the YS side), but a standalone application would probably work as well. For the latter, feel free to "steal" the YS code to display an image, zoom and draw lines (it's open source): https://github.com/langmo/youscope/blob/master/youscope-addon/src/org/youscope/uielements/ImagePanel.java Note that Java offers no support for unsigned numbers, which is a bit inconvenient for programs with a focus on image processing. You could also write the GUI in Java and the processing in Matlab (it's easy to talk to Java from Matlab), or use the combination Java/Jython (Jython=Python which can easily talk to Java). C++ is IMO too low level for your task...

In case I misunderstood what you want to do, please tell me!

You understand me pretty well. The software I need is very basic, I could write it myself, but I guess it is a part of any microscope control software (any microscope software needs to transform pixels into units and all the microscope software I have seen has the tool "draw an interval over the image and get its length"). So I guessed that your software has something similar. So I just don't want to reinvent a wheel.

The simplest thing which I forgot: There is already a TWAIN driver in YS/MM! Just follow the documentation provided at https://micro-manager.org/wiki/TwainCamera .

Thank you for the info, I'll try it.

Well, it depends on the details of your workflow: 1) you can indeed draw lines in YS. Just click on an image (e.g. in the LiveStream) and drag the mouse. It will display the length of the line, in pixels. 2) If you want, I can easily add a field where you can type in a conversion factor pixels->mm, and, given that you inserted a conversion factor, display all lengths in mm. This is then essentially the same as in nearly all other microscope control software, where you also somewhere manually have to define this conversion factor, in one or the other way (typically hidden in a way such that the active conversion factor is nearly never the correct one). 3) I can easily add a functionality where you drag a line, define its length in mm, and which then calculates the conversion factor for you. Afterwards, all length will be displayed in mm using this factor. 4) Instead of one conversion factor, I can display a list of conversion factors (e.g. for the minimum and maximum zoom). I can implement it such that you can then click on one of these factors, and that this factor then becomes "active" (all lengths will be displayed in mm, using this factor. If you want, you can also define a name for each factor, making it easier to handle them.

You seem to have the ability to continuously change the magnification. A discrete list of factors would thus probably not do the trick. Your proposed algorithm to do the "automatic continuous magnification factor determination while zooming" is the troublesome part. Yes, one can probably do something like that. No, it will probably not do the trick. There will be some small error when automatically determining the zoom based on comparing the current frame with the previous one and estimating the affine transformation between them. This error will add up, that is, after zooming a bit around (or not zooming at all; the program doesn't know when you manually zoom, i.e. will have to always assume that you are currently zooming) the magnification factor will be completely off. Better would be to always compare the current frame with some reference frame at which you define the magnification factor. This would work if the differences in the zoom factor don't become too big. Some more buttons to click to define one or more reference frames. This could work in principal, given that you have enough and appropriate features in your images.

Now comes the point: While I am happy to implement 1-4 in YS because these might be useful functions in general, an algorithm to automatically determine the zoom level based on something as described above would have to be directly tailored to your specific task, and only for that (there would be a lot of decisions which have to be made and parameters which have to be optimized to make it working for your task). To make a long story short: I will not implement such an algorithm because this would be a lot of work without any benefit for me or any other user of YS. Sorry to state this rather directly.

However, of course you will not have to reinvent the wheel again if you would implement such an algorithm on your own. As I said, you are more than welcome to write an own plugin in YS based on the functionality (e.g. to draw lines, make images, programmatically zoom,...) which is already there, such that you would only have to implement the tasks specific to your problem. You are also more than welcome to simply "steal" (-> open source) the corresponding source code implementing this functionality, rewrite and adjust it as you want (i.e. in a stand-alone program). The corresponding part of YS to show images, draw lines etc is well documented, extendable and easily usable (when writing a plugin), respectively easily "stealable" (when writing a standalone program; see link in previous post). Independently if you write a plugin or only "steal" the code, I am more than happy to help you with writing/adjusting the sources. I will just not write the "automatic zoom adjustment" feature, that's your job.

You seem to have the ability to continuously change the magnification.

Yes, I do.

A discrete list of factors would thus probably not do the trick.

At least we have 2 fixed factors: the extreme positions of the knob.

To make a long story short: I will not implement such an algorithm because this would be a lot of work without any benefit for me or any other user of YS. Sorry to state this rather directly.

Completely OK.

langmo / youscope

Use with a stupid camera #7