search

waveform80 / picamera

A pure Python interface to the Raspberry Pi camera module
https://picamera.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
1.57k stars 355 forks source link

Fix inaccurate Live preview #23

Closed russb78 closed 10 years ago

russb78 commented 11 years ago

While it's flagged in the documentation, the suggested workaround of setting the camera to full resolution doesn't appear to get the live preview and taken image to match.

For context, I'm creating a stop motion animation package. The 'onion-skinning' effect is working perfectly with the Alpha preview laid over the top of the last image taken (via Pygame), but since the live preview isn't an accurate representation of the actual image, it's impossible to align them.

If anyone has got a suggestion on how to post-process the image via PIL / Pygame in the short-term, I'd really appreciate the input. Any information on if/when there might be a fix / workaround would be really appreciated.

Thanks for the library - it's otherwise fantastic!

waveform80 commented 11 years ago

Interesting - I've never tried actually overlaying the preview with a taken image - I'd just assumed the two were the same based on a quick eye-balling of a few captures, and comments in the forums that this was the way to compensate. Is there any chance you could attach a capture of the overlay so I can see exactly how they're mis-aligned? There's a slight possibility that the fix for #21 (which I'm working on today) might be applied similarly to previews and captures, but it rather depends on what sort of mis-alignment we're looking at (smaller, larger, offset?).

In the meantime, assuming the mis-alignment we're talking about is, say, smaller (the captured area is larger than the preview), the Image.crop method in the PIL library is probably the simplest to work around it (assuming you can figure out the crop-box required):

import io
import picamera
from PIL import image

with picamera.PiCamera() as camera:
    camera.resolution = camera.MAX_IMAGE_RESOLUTION
    stream = io.BytesIO()
    camera.capture(stream, format='jpeg')
    stream.seek(0)
    img = Image.open(stream)
    # Crop 10 pixels off the left and right, and 20 off the top and bottom
    cropped = img.crop((10, 20, 10, 20))
    cropped.save('foo.jpg')

(untested - this is just off the top of my head)

russb78 commented 11 years ago

Yes, the captured image is markedly larger than the preview - the final image is zoomed out compared to the preview (much more on the edges). Sorry, I don't have a preview to hand right now, but the demonstration images in your documentation are pretty accurate representations of the difference.

I've actually managed to get the preview and the capture at full resolution (using windowed mode for the preview) and the alpha preview and taken image then match! Sadly, at the average res for a Raspberry Pi screen (720p ish), you're only getting around a quarter of the actual image, so it's still not a good solution for my stop motion animation app, sadly.

I've tried exactly halving the resolution of the camera sensor / preview window to try and emulate the full-resolution match-up at a more usable resolution, but the same zoomed out mis-matching kicks in again. Frustrating!

The PIL method you've mentioned could work, thanks for that. It'll take a lot of eye-balling and guesswork, but it's certainly worth a try... I don't suppose you (or anyone) knows the maths surrounding the differences between preview / actual image? Any information might be useful (my maths is utterly abysmal).

I'll try and get an example of the overlay / taken image if possible.

Thanks again and keep up the great work.

waveform80 commented 11 years ago

Ahh, I probably need to make the docs clearer: the only time the preview matches the capture area at the moment is when the camera's resolution is set to maximum (2592x1944). At any resolution other than that, the preview is drawn from the 1920x1080 pixels at the centre of the camera's sensor (the "video area") and scaled to the requested resolution, while the capture uses the full area (scaled down to the requested resolution). That's what the bit at the end of the Preview vs Still Resolution is going on about. Unfortunately this just seems to be an artifact of the way the preview port of the camera works.

Come to think of it, though, there is one other occasion when the preview matches the capture area. Try using the "use_video_port=True" option on the capture methods; when the camera uses the video port for stills it should be selecting from the same region that the preview draws from. Unfortunately this comes with a bunch of caveats (no exif tags, certain effects disabled, and the images tend to look grainier).

I think those are your only two options right now - run at full res and scale images down with PIL/pygame afterward, or capture with use_video_port=True. If you need a hand with the maths just post the res you're capturing at and the res that you want - I'm sure I can bash together some simple code for it.

Having said all that, issue #21 may provide an answer to this. I've been working with the resizer and the camera's video port today and I think I've got it up to full resolution (albeit at a reduced framerate of 15fps). It's possible I can do the same for the preview system so it'll always match the capture area. I've haven't had the chance to experiment with this bit yet though.

Anyway, I'll leave this open for now but I suspect ultimately I'll be closing it as a dupe of #21 (assuming the resizer works with the preview the way I expect it will).

russb78 commented 11 years ago

Thanks for that. Having read that part of the docs again, it makes perfect sense.

Cheers for the tips - I'm going to get on that right now. Much appreciated!

russb78 commented 11 years ago

No dice - the 'use_video_port = True' option doesn't seem to affect the end results in any way with my code.

I'm guessing this is because I'm not using the stream method for capture (which is too grainy / low fidelity for my purpose)? Either that our I haven't got a grasp on how to use the option correctly.

So close, but yet so far - here's a gist of my current code, I'd really appreciate it if you'd take a look and see if the use_video_port option can be applied here (and how)...

https://gist.github.com/russb78/7412501

At the moment I'm stuck with forcing full resolution on the sensor, using a full screen preview (which is squished to fit on my screen with black borders either side), scaling a preview image to display behind the camera preview (when in Alpha preview mode). It works for creating the onion-skinning effect, but since the images as so huge there's massive lag. Also, the end results are in 4:3, which isn't ideal for uploading stop motion animations to YouTube :)

All my problems would be solved if the the preview used the full camera sensor, regardless of the resolution you set the camera to. Do you think something like this would require a full-on firmware fix?

I've got a deadline on this of the end of the week on this and I'm wondering if I'm barking up the wrong tree.

russb78 commented 11 years ago

And I've figured out use_video_port (and it works!!!)....

waveform80 commented 11 years ago

Ah, good stuff - I take it you've managed to get a capture the same size as the preview (presumably in some wide-screen aspect ratio) using use_video_port=True? If so, that's probably your best route forward right now. Admittedly the quality won't be as good as using the capture port (which uses the full resolution of the sensor and performs some interpolation to produce a better quality image), but I'm hopeful I can add that capability with the resizer component. I've skimmed through the gist you linked to - looks good, though I guess it's a little out of date if you've got the use_video_port bit working. If I can find the time this week I'll have a go at using it with the development copy of picamera to see if I can get the full-quality stuff working.

russb78 commented 11 years ago

Thanks for the advice and help. EDIT: To answer your question, yes the image capture and the preview match at any resolution using use_video_port=True'.

I've uploaded the topic of our discussion, a RasPi camera module stop motion animation application, to github:

https://github.com/russb78/pi-mation

Feel free to have a play around (and tips on my terrible coding would be welcome).

Thanks again - you've been really helpful.

waveform80 commented 10 years ago

Given the resizer now (mostly) works, there's a couple of possible ways of solving the disparity between capture and preview sizes. The documentation should be enhanced for 1.2 to properly describe both methods (use_video_port or set full-res + resize), then I think we can reasonably close this.