search

ai-cfia / nachet-frontend

Frontend application for seed classification of images acquired from digital microscopes
MIT License
2 stars 2 forks source link

User Feature Request: Alter image capture to analyse full resolution #144

Closed TaranMeyer closed 1 week ago

TaranMeyer commented 2 weeks ago

Is your feature request related to a problem? Please describe. As currently constructed, the Nachet web app resizes the 1080p feed from the Tagarno camera to fit the view box on the app screen, then takes the capture which the model then analyzes from that smaller view box. As a result the images the app takes are only 971 x 584 pixels, roughly half of the original 1920 x 1080. This reduction in pixels relative to the training images reduces the number of pixels in each seed image for the model to analyze, makes the captured images less similar to the images used to train the model, and can be expected to reduce accuracy.

Describe the solution you'd like It would be better to take the capture for the model from a full-sized version of the camera feed, then reduce/render that to fit the box in the app for results display purposes only.

Additional context Joseffus seemed to have some good ideas about how this could be done, I hope he can address the issue more clearly than I have 😄

ChromaticPanic commented 2 weeks ago

The issue is that the microscope feed is bound to the canvas size which is proportional to the size of your browser window. Then the save function saves this scaled image.

If the need is urgent quickest way around this is to simply use a higher resolution monitor, 1440p or 4k would allow the browser and canvas to easily have more than 1920x1080 pixels.

There are a few options to fix this in code. We could try using a hidden high res canvas for saving images and sending for inference. Or imageCapture grabFrame() takePhoto() browser functions (experimental) https://webrtchacks.com/still-image-from-webcam-stream-approaches/

ChromaticPanic commented 1 week ago

I used OBS as a cam source to stream these images. We see the after image has 1920x1080 resolution. Now I don't know if the microscope source has higher than 1920x1080 resolution because we can use the maximum resolution provided by the image source. e.g. a 4K camera will have 3840x2160 pixels

This first one is before the fix I implemented testforcetrue-2024-5-7

This one is after #149 testforcetrue2-2024-5-7

ChromaticPanic commented 1 week ago

Looks like this also fixes #54 . I don't have a microscope to check 😅. But it looks like in the files I saved, the top one is cropped and the bottom one shows the full feed after I implemented the resolution change.

screenshot image

TaranMeyer commented 1 week ago

Oh this is awesome!!!! I will do some checking today!

... I guess I jumped the gun gain, it doesn't look like there's a change on the front end yet? 🤣

ChromaticPanic commented 1 week ago

Oh this is awesome!!!! I will do some checking today!

... I guess I jumped the gun gain, it doesn't look like there's a change on the front end yet? 🤣

I think your interface is still on 0.4.0 ? We are actually on 0.8.0 now, I think there may be deployment issues as well. I brought it up at today's scrum since the endpoint Liang is using is on 0.2.0 .

TaranMeyer commented 1 week ago

Yes, mine says 0.4.0 for sure. Is there a way I can fix that, or a different link I should be using? Refreshing the page doesn't seem to do it, though I suppose I haven't cleared my browser cache.

ChromaticPanic commented 1 week ago

Yes, mine says 0.4.0 for sure. Is there a way I can fix that, or a different link I should be using? Refreshing the page doesn't seem to do it, though I suppose I haven't cleared my browser cache.

It has to do with the deployment pipeline. I haven't looked that far into it but the devOps team should be able to figure it out easily.