[BUG] Continuous AutoFocus fails to converge

[fabrii4]

Please only report one bug per issue!

Describe the bug When using continuous autofocus, the lens position mostly keeps oscillating and fails to settle. Only in some cases, depending on the framed scene, it succeeds to converge to a stable value.

To Reproduce

from picamera2 import Picamera2
from libcamera import Transform, controls
import cv2
import numpy as np
import matplotlib.pyplot as plt

#init fig to plot lens position
fig, ax = plt.subplots()
ax.axis([0, 500, 0, 10])
lenspos = np.zeros(500)
plot = ax.plot(lenspos)
fig.canvas.manager.set_window_title('LensPosition')
fig.canvas.manager.show() 

#init camera
cam = Picamera2() 
m=(2304, 1296) 
w=(384,216) 
window=(m[0]-w[0], m[1]-w[1], w[0]*2, w[1]*2)
config=cam.create_preview_configuration(main={"format": 'RGB888', "size": m})
cam.configure(config)
cam.set_controls({"FrameDurationLimits": (33333, 33333)})
cam.set_controls({"AfMode": controls.AfModeEnum.Continuous, 
                  "AfMetering": controls.AfMeteringEnum.Windows, 
                  "AfWindows": [window]})
cam.start()

while True:
    im = cam.capture_array("main")    
    metadata = cam.capture_metadata()
    cv2.rectangle(im, ((m[0]-w[0])//2,(m[1]-w[1])//2), ((m[0]+w[0])//2,(m[1]+w[1])//2), (0,0,255), 2)
    cv2.imshow("frame",im)

    key = cv2.waitKey(1)
    if key == ord("q"):
        exit()

    #plot lens position over time
    lenspos[:-1]=lenspos[1:]
    lenspos[-1] = metadata['LensPosition']
    plot[0].set_ydata(lenspos)
    fig.canvas.draw()
    fig.canvas.flush_events()

Expected behaviour The lens position should always settle to some optimal value.

Console Output, Screenshots

Hardware : Raspberry pi 5 Pi camera module 3 noir wide

[davidplowman]

Hi, and thanks for the report. The autofocus works by performing correlations between pixels on the sensor that get a slightly different view of scene. It's not terribly difficult for this process to get confused, either because the details in the scene look similar in slightly different places, or because there aren't any terribly strong details, or because there different parts of the scene are at different distances.

@njhollinghurst may be able to suggest some possible workarounds, or perhaps some debug that could be collected to try and see what's going on.

[njhollinghurst]

At first sight it looks like CDAF is working but PDAF is unstable.

In general, is there a certain type of scene that will make it focus successfully, compared to those for which it fails? Could you check also that the lens is clean and undamaged.

Could you try setting the environment variable LIBCAMERA_LOG_LEVELS=RPiAf:0 and report the logged output please, e.g. the first 50 lines when starting the camera, for typical"good" and "bad" cases? Thanks.

[fabrii4]

Hi,

Yes, it seems that areas with less details are more likely to focus (like the wooden surface I posted above or an empty wall). When I try to focus on some object or on a printed page it usually fails. The lens looks clean and undamaged.

Here are the logs for the good and bad cases:

GOOD [9:47:26.204360579] [59184] RPiAf af.cpp:723 setLensPosition: 1 [9:47:26.217123562] [59189] RPiAf af.cpp:760 setMode: 2 [9:47:26.217194449] [59189] RPiAf af.cpp:704 Window: 1920, 1080, 768, 432 [9:47:26.217204782] [59189] RPiAf af.cpp:676 setRange: 2 [9:47:26.217214597] [59189] RPiAf af.cpp:683 setSpeed: 0 [9:47:26.217362519] [59189] RPiAf af.cpp:231 switchMode: statsRegion: 0,0,4608,2592 [9:47:26.361152653] [59189] RPiAf af.cpp:317 Recompute Phase weights 16x12 [9:47:26.361174356] [59189] RPiAf af.cpp:499 SKIP [9:47:26.361178930] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:47:26.398003103] [59189] RPiAf af.cpp:355 Recompute Contrast weights 8x8 [9:47:26.398245319] [59189] RPiAf af.cpp:499 SKIP [9:47:26.398261967] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:47:26.428599827] [59189] RPiAf af.cpp:499 SKIP [9:47:26.428679862] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:47:26.461876962] [59189] RPiAf af.cpp:499 SKIP [9:47:26.461986348] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:47:26.494948824] [59189] RPiAf af.cpp:499 SKIP [9:47:26.494982231] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:47:26.528977570] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=654 phase=-34 conf=1 [9:47:26.561395190] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=651 phase=-14 conf=1 [9:47:26.594717380] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=654 phase=-19 conf=1 [9:47:26.628025699] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=653 phase=0 conf=1 [9:47:26.661910985] [59189] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=652 phase=-33 conf=1 [9:47:26.694709818] [59189] RPiAf af.cpp:638 1 sst2->3 stp0->4 ft1.00->0.00 fs1.00->0.00 cont=652 phase=-2 conf=1 [9:47:26.728122857] [59189] RPiAf af.cpp:638 1 sst3->3 stp4->3 ft0.00->0.00 fs0.00->0.00 cont=403 phase=-6 conf=2 [9:47:26.761542841] [59189] RPiAf af.cpp:638 1 sst3->3 stp3->2 ft0.00->0.00 fs0.00->0.00 cont=375 phase=0 conf=1 [9:47:26.794600426] [59189] RPiAf af.cpp:638 1 sst3->3 stp2->1 ft0.00->0.00 fs0.00->0.00 cont=373 phase=6 conf=1 [9:47:26.828384103] [59189] RPiAf af.cpp:638 1 sst3->3 stp1->0 ft0.00->0.00 fs0.00->0.00 cont=378 phase=-11 conf=2 [9:47:26.861246972] [59189] RPiAf af.cpp:638 1 sst3->3 stp0->4 ft0.00->2.00 fs0.00->2.00 cont=380 phase=-50 conf=1 [9:47:26.894593753] [59189] RPiAf af.cpp:638 1 sst3->3 stp4->3 ft2.00->2.00 fs2.00->2.00 cont=464 phase=-5 conf=2 [9:47:26.928335524] [59189] RPiAf af.cpp:638 1 sst3->3 stp3->2 ft2.00->2.00 fs2.00->2.00 cont=784 phase=532 conf=1 [9:47:26.961313186] [59189] RPiAf af.cpp:638 1 sst3->3 stp2->1 ft2.00->2.00 fs2.00->2.00 cont=968 phase=-57 conf=1 [9:47:26.994884350] [59189] RPiAf af.cpp:638 1 sst3->3 stp1->0 ft2.00->2.00 fs2.00->2.00 cont=993 phase=79 conf=1 [9:47:27.027878586] [59189] RPiAf af.cpp:638 1 sst3->3 stp0->4 ft2.00->4.00 fs2.00->4.00 cont=1003 phase=547 conf=1 [9:47:27.061147036] [59189] RPiAf af.cpp:638 1 sst3->3 stp4->3 ft4.00->4.00 fs4.00->4.00 cont=546 phase=-87 conf=3 [9:47:27.094587963] [59189] RPiAf af.cpp:638 1 sst3->3 stp3->2 ft4.00->4.00 fs4.00->4.00 cont=678 phase=-34 conf=3 [9:47:27.127997132] [59189] RPiAf af.cpp:638 1 sst3->3 stp2->1 ft4.00->4.00 fs4.00->4.00 cont=718 phase=-93 conf=1 [9:47:27.161856048] [59189] RPiAf af.cpp:638 1 sst3->3 stp1->0 ft4.00->4.00 fs4.00->4.00 cont=763 phase=-28 conf=1 [9:47:27.197305016] [59189] RPiAf af.cpp:638 1 sst3->3 stp0->4 ft4.00->6.00 fs4.00->6.00 cont=761 phase=108 conf=3 [9:47:27.229559085] [59189] RPiAf af.cpp:638 1 sst3->3 stp4->3 ft6.00->6.00 fs6.00->6.00 cont=448 phase=-36 conf=1 [9:47:27.261656751] [59189] RPiAf af.cpp:638 1 sst3->3 stp3->2 ft6.00->6.00 fs6.00->6.00 cont=256 phase=-44 conf=1 [9:47:27.294733762] [59189] RPiAf af.cpp:638 1 sst3->3 stp2->1 ft6.00->6.00 fs6.00->6.00 cont=270 phase=0 conf=3 [9:47:27.328396164] [59189] RPiAf af.cpp:638 1 sst3->3 stp1->0 ft6.00->6.00 fs6.00->6.00 cont=268 phase=-64 conf=3 [9:47:27.361951107] [59189] RPiAf af.cpp:451 FindPeak: 2.46298 [9:47:27.361994291] [59189] RPiAf af.cpp:638 1 sst3->4 stp0->4 ft6.00->3.46 fs6.00->3.46 cont=267 phase=-16 conf=3 [9:47:27.395243816] [59189] RPiAf af.cpp:638 1 sst4->4 stp4->3 ft3.46->3.46 fs3.46->3.46 cont=358 phase=-48 conf=1 [9:47:27.430775078] [59189] RPiAf af.cpp:638 1 sst4->4 stp3->2 ft3.46->3.46 fs3.46->3.46 cont=682 phase=54 conf=1 [9:47:27.463647575] [59189] RPiAf af.cpp:638 1 sst4->4 stp2->1 ft3.46->3.46 fs3.46->3.46 cont=905 phase=265 conf=1 [9:47:27.496641699] [59189] RPiAf af.cpp:638 1 sst4->4 stp1->0 ft3.46->3.46 fs3.46->3.46 cont=947 phase=122 conf=3 [9:47:27.528027235] [59189] RPiAf af.cpp:638 1 sst4->4 stp0->4 ft3.46->2.96 fs3.46->2.96 cont=961 phase=-31 conf=2 [9:47:27.562827775] [59189] RPiAf af.cpp:638 1 sst4->4 stp4->3 ft2.96->2.96 fs2.96->2.96 cont=980 phase=-8 conf=3 [9:47:27.595916933] [59189] RPiAf af.cpp:638 1 sst4->4 stp3->2 ft2.96->2.96 fs2.96->2.96 cont=1054 phase=41 conf=1 [9:47:27.628350998] [59189] RPiAf af.cpp:638 1 sst4->4 stp2->1 ft2.96->2.96 fs2.96->2.96 cont=1077 phase=-6 conf=1 [9:47:27.661444545] [59189] RPiAf af.cpp:638 1 sst4->4 stp1->0 ft2.96->2.96 fs2.96->2.96 cont=1081 phase=-20 conf=1 [9:47:27.699768660] [59189] RPiAf af.cpp:638 1 sst4->4 stp0->4 ft2.96->2.46 fs2.96->2.46 cont=1086 phase=64 conf=3 [9:47:27.728869033] [59189] RPiAf af.cpp:638 1 sst4->4 stp4->3 ft2.46->2.46 fs2.46->2.46 cont=1043 phase=373 conf=1 [9:47:27.763745423] [59189] RPiAf af.cpp:638 1 sst4->4 stp3->2 ft2.46->2.46 fs2.46->2.46 cont=1075 phase=293 conf=1 [9:47:27.794369664] [59189] RPiAf af.cpp:638 1 sst4->4 stp2->1 ft2.46->2.46 fs2.46->2.46 cont=1100 phase=39 conf=1 [9:47:27.828033196] [59189] RPiAf af.cpp:638 1 sst4->4 stp1->0 ft2.46->2.46 fs2.46->2.46 cont=1101 phase=-4 conf=1 [9:47:27.863519367] [59189] RPiAf af.cpp:638 1 sst4->4 stp0->4 ft2.46->1.96 fs2.46->1.96 cont=1102 phase=51 conf=1 [9:47:27.897090587] [59189] RPiAf af.cpp:638 1 sst4->4 stp4->3 ft1.96->1.96 fs1.96->1.96 cont=998 phase=-11 conf=1 [9:47:27.929467171] [59189] RPiAf af.cpp:638 1 sst4->4 stp3->2 ft1.96->1.96 fs1.96->1.96 cont=993 phase=608 conf=1 [9:47:27.962081860] [59189] RPiAf af.cpp:638 1 sst4->4 stp2->1 ft1.96->1.96 fs1.96->1.96 cont=1005 phase=-91 conf=1 [9:47:27.995729689] [59189] RPiAf af.cpp:638 1 sst4->4 stp1->0 ft1.96->1.96 fs1.96->1.96 cont=1011 phase=447 conf=1 [9:47:28.027641341] [59189] RPiAf af.cpp:638 1 sst4->4 stp0->4 ft1.96->1.46 fs1.96->1.46 cont=1006 phase=-29 conf=1 [9:47:28.061537127] [59189] RPiAf af.cpp:638 1 sst4->4 stp4->3 ft1.46->1.46 fs1.46->1.46 cont=817 phase=78 conf=1 [9:47:28.095940066] [59189] RPiAf af.cpp:638 1 sst4->4 stp3->2 ft1.46->1.46 fs1.46->1.46 cont=812 phase=-31 conf=1 [9:47:28.128452832] [59189] RPiAf af.cpp:638 1 sst4->4 stp2->1 ft1.46->1.46 fs1.46->1.46 cont=807 phase=84 conf=1 [9:47:28.160740122] [59189] RPiAf af.cpp:638 1 sst4->4 stp1->0 ft1.46->1.46 fs1.46->1.46 cont=822 phase=20 conf=1 [9:47:28.194642019] [59189] RPiAf af.cpp:451 FindPeak: 2.65131 [9:47:28.194694888] [59189] RPiAf af.cpp:638 1 sst4->5 stp0->4 ft1.46->2.65 fs1.46->2.65 cont=817 phase=22 conf=1

BAD [9:44:27.514120063] [58797] RPiAf af.cpp:723 setLensPosition: 1 [9:44:27.526862236] [58802] RPiAf af.cpp:760 setMode: 2 [9:44:27.526940160] [58802] RPiAf af.cpp:704 Window: 1920, 1080, 768, 432 [9:44:27.526955882] [58802] RPiAf af.cpp:676 setRange: 2 [9:44:27.526967159] [58802] RPiAf af.cpp:683 setSpeed: 0 [9:44:27.527133322] [58802] RPiAf af.cpp:231 switchMode: statsRegion: 0,0,4608,2592 [9:44:27.678992680] [58802] RPiAf af.cpp:317 Recompute Phase weights 16x12 [9:44:27.679014236] [58802] RPiAf af.cpp:499 SKIP [9:44:27.679018569] [58802] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:44:27.713180093] [58802] RPiAf af.cpp:355 Recompute Contrast weights 8x8 [9:44:27.713364681] [58802] RPiAf af.cpp:499 SKIP [9:44:27.713457142] [58802] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:44:27.745916782] [58802] RPiAf af.cpp:499 SKIP [9:44:27.745987706] [58802] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:44:27.779246696] [58802] RPiAf af.cpp:499 SKIP [9:44:27.779266529] [58802] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:44:27.812619647] [58802] RPiAf af.cpp:499 SKIP [9:44:27.812648943] [58802] RPiAf af.cpp:638 1 sst2->2 stp0->0 ft1.00->1.00 fs1.00->1.00 cont=0 phase=0 conf=0 [9:44:27.845875675] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft1.00->1.52 fs1.00->1.52 cont=2380 phase=-20 conf=50 [9:44:27.879186757] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft1.52->1.97 fs1.52->1.97 cont=2626 phase=-17 conf=47 [9:44:27.912514820] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft1.97->2.17 fs1.97->2.17 cont=3025 phase=-7 conf=46 [9:44:27.945816476] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.17->2.55 fs2.17->2.55 cont=3002 phase=-14 conf=45 [9:44:27.979241537] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.55->2.78 fs2.55->2.78 cont=3524 phase=-8 conf=47 [9:44:28.012532323] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.78->2.97 fs2.78->2.97 cont=3691 phase=-7 conf=48 [9:44:28.045834942] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.97->2.98 fs2.97->2.98 cont=3737 phase=-2 conf=47 [9:44:28.079104655] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.98->3.07 fs2.98->3.07 cont=3731 phase=-5 conf=46 [9:44:28.112781265] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.07->3.18 fs3.07->3.18 cont=3741 phase=-6 conf=45 [9:44:28.145719800] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.18->3.45 fs3.18->3.45 cont=3721 phase=-10 conf=46 [9:44:28.179049104] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.45->3.41 fs3.45->3.41 cont=3613 phase=4 conf=45 [9:44:28.212360908] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.41->3.41 fs3.41->3.41 cont=3627 phase=1 conf=44 [9:44:28.245703156] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.41->3.35 fs3.41->3.35 cont=3635 phase=5 conf=44 [9:44:28.279021164] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.35->3.35 fs3.35->3.35 cont=3656 phase=0 conf=44 [9:44:28.312365449] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.35->3.35 fs3.35->3.35 cont=3667 phase=0 conf=44 [9:44:28.345645179] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.35->3.36 fs3.35->3.36 cont=3667 phase=-3 conf=43 [9:44:28.379010908] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.36->3.36 fs3.36->3.36 cont=3646 phase=1 conf=46 [9:44:28.412734406] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.36->3.37 fs3.36->3.37 cont=3667 phase=-2 conf=44 [9:44:28.446975576] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.37->3.37 fs3.37->3.37 cont=3643 phase=-2 conf=45 [9:44:28.481391742] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.37->3.40 fs3.37->3.40 cont=3636 phase=-3 conf=45 [9:44:28.512651873] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.40->3.40 fs3.40->3.40 cont=3641 phase=-1 conf=48 [9:44:28.546054359] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.40->3.13 fs3.40->3.13 cont=3642 phase=10 conf=45 [9:44:28.579438457] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.13->3.13 fs3.13->3.13 cont=3662 phase=0 conf=44 [9:44:28.612526600] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.13->3.33 fs3.13->3.33 cont=3726 phase=-7 conf=44 [9:44:28.646292114] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.33->3.13 fs3.33->3.13 cont=3666 phase=7 conf=43 [9:44:28.679325999] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.13->3.14 fs3.13->3.14 cont=3672 phase=-2 conf=45 [9:44:28.714087286] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.14->3.14 fs3.14->3.14 cont=3712 phase=-2 conf=45 [9:44:28.747290722] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.14->3.14 fs3.14->3.14 cont=3718 phase=0 conf=43 [9:44:28.779494089] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.14->3.14 fs3.14->3.14 cont=3722 phase=0 conf=45 [9:44:28.813058831] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.14->3.20 fs3.14->3.20 cont=3732 phase=-5 conf=43 [9:44:28.847212984] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.20->3.20 fs3.20->3.20 cont=3706 phase=0 conf=44 [9:44:28.881000943] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.20->3.23 fs3.20->3.23 cont=3717 phase=-3 conf=44 [9:44:28.914219730] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.23->3.23 fs3.23->3.23 cont=3695 phase=1 conf=45 [9:44:28.947341798] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.23->3.23 fs3.23->3.23 cont=3688 phase=0 conf=45 [9:44:28.982904102] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.23->3.23 fs3.23->3.23 cont=3677 phase=1 conf=46 [9:44:29.012070488] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.23->3.29 fs3.23->3.29 cont=3692 phase=-5 conf=43 [9:44:29.047227191] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.29->3.29 fs3.29->3.29 cont=3667 phase=1 conf=46 [9:44:29.080325889] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.29->3.29 fs3.29->3.29 cont=3659 phase=2 conf=44 [9:44:29.114976697] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.29->3.00 fs3.29->3.00 cont=3679 phase=11 conf=41 [9:44:29.146883442] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.00->3.01 fs3.00->3.01 cont=3687 phase=-1 conf=45 [9:44:29.179774737] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.01->3.01 fs3.01->3.01 cont=3720 phase=0 conf=45 [9:44:29.212720105] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.01->2.99 fs3.01->2.99 cont=3741 phase=3 conf=45 [9:44:29.247089827] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.99->2.98 fs2.99->2.98 cont=3728 phase=2 conf=44 [9:44:29.279080718] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.98->2.99 fs2.98->2.99 cont=3743 phase=-2 conf=47 [9:44:29.312316876] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.99->3.02 fs2.99->3.02 cont=3755 phase=-4 conf=46 [9:44:29.346464363] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.02->3.07 fs3.02->3.07 cont=3744 phase=-4 conf=45 [9:44:29.378674211] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.07->3.06 fs3.07->3.06 cont=3734 phase=2 conf=47 [9:44:29.412867308] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.06->3.07 fs3.06->3.07 cont=3740 phase=-1 conf=43 [9:44:29.447260178] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.07->3.07 fs3.07->3.07 cont=3724 phase=0 conf=46 [9:44:29.478478661] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft3.07->2.91 fs3.07->2.91 cont=3730 phase=7 conf=47 [9:44:29.512707275] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.91->2.95 fs2.91->2.95 cont=3723 phase=-4 conf=46 [9:44:29.546180667] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.95->2.96 fs2.95->2.96 cont=3726 phase=-1 conf=44 [9:44:29.582218904] [58802] RPiAf af.cpp:638 2 sst2->2 stp0->0 ft2.96->2.96 fs2.96->2.96 cont=3729 phase=1 conf=46

[njhollinghurst]

Thanks. It looks like we may need to tweak the algorithm to be more stable with small AF windows and low "confidence" values.

[fabrii4]

Thank you. Just as a note, it does not seem that the AF window size matters, I observe the same behavior also with large AF windows or with Auto AfMetering.

[njhollinghurst]

"phase" and "conf" are the PDAF data coming from the camera sensor (not quite in their raw form -- this is after applying the window). In the "GOOD" example, phase makes some bizarre jumps; fortunately it's ignored. In the "BAD" example the confidence is high enough that it's no longer ignored; those sudden jumps of +/-10 might be what's making it bounce.

I have seen it become wobbly in the past, though this seems to be an unusually bad example. I'll have to try to reproduce this, and maybe revise the control algorithm.

Right now it runs in a continuous loop (as longs as "conf" is high enough) - it will never settle completely - but perhaps it should.

[davidplowman]

Is there any possibility of adjusting the "conf" value in the tuning file, at least as a workaround for the time being?

[njhollinghurst]

Yes. Can edit the tuning file /usr/share/libcamera/ipa/rpi/pisp/imx708_wide_noir.json (make a copy first!)

In the "rpi.af" section near the end of the file, try increasing the numbers for "conf_thresh" (which gates PDAF on a per-region granularity) and "conf_epsilon" (which is more of a global threshold). Perhaps "conf_epsilon": 24, "conf_thresh" : 32,.

Other parameters that affect stability are "rpi.af"/"speeds"/"normal"/"pdaf_gain" (reduce its magnitude) and "pdaf_squelch" (increase it). Those parameters are applied here: af.cpp:370

Please ensure also that the lighting is good and not glaring into the camera.

[fabrii4]

Thanks, I run a quick test with "conf_epsilon": 24, "conf_thresh" : 32, and indeed it seems to reduce the amplitude of the oscillations.

Right now it runs in a continuous loop (as longs as "conf" is high enough) - it will never settle completely - but perhaps it should.

Yes, it would be nice to have it to always settle completely after some timeout (like in the GOOD low "conf" case) in order to avoid differences in the captured frames.

Or alternatively, is it possible to get some focus indication like the FocusFOM metadata but only for the AfWindow region? In that way I could use AfPause to stop and restart autofocus when triggered.

[njhollinghurst]

FocusFOM is only reported on a (not very well-specified) central region right now; there doesn't seem to be a control defined to set a window. If scenes are mostly static, "Triggered Auto" mode might be a better fit than Continuous AF.

OTOH if you stick with Continuous AF, try increasing (e.g. doubling) "squelch". That will create more of a dead-band, though admittedly it is a soft rather than hard threshold and doesn't completely stop the lens moving.

[fabrii4]

But "Triggered Auto" mode requires a trigger, I need the camera to be able to refocus automatically.

Anyway, I will play with the parameters in the tuning file, the changes you suggest already make a big improvement.