Pylance keeps finding python errors in openCV-python code on vs code insider

[jquintanilla4]

Type: Bug

VSCode keeps finding pylance errors with the openCV part of my code. But this only happens in visual studio code insider, it doesn't happen in the regular version of vscode.

The following code brings up pylance errors for ir_image, norm_type, and depth_image. My code runs fine, but the errors remain.

while True:
    capture = k4a.get_capture()

    ir_image = capture.ir
    depth_image = capture.depth

    # Normalize IR image using percentile method
    ir_min = np.percentile(ir_image, 1) # Find the closest to darkest pixels
    ir_max = np.percentile(ir_image, 99) # Find the closest to brightest pixels
    ir_image_clipped = np.clip(ir_image, ir_min, ir_max) # Clip the image by remove brigter than ir max and darker than ir min
    ir_image_normalized = cv.normalize(ir_image_clipped, None, alpha=0, beta=255, norm_type=cv.NORM_MINMAX, dtype=cv.CV_8U)
    # we make the brigthtest 0 and the darkest 255, all the other pixel values are adjusted proportiaonally in between
    # Convert IR image into 3 channels using OpenCV image
    ir_image_3channel = cv.cvtColor(ir_image_normalized, cv.COLOR_GRAY2BGR)

    # Transform the infrared image to the color camera's coordinate space
    transformed_depth_image = transformation.depth_image_to_color_camera(depth_image, calibration=k4a_calibration, thread_safe=True)
    # Normlize the image
    depth_image_normalized = cv.normalize(transformed_depth_image, None, alpha=0, beta=255, norm_type=cv.NORM_MINMAX, dtype=cv.CV_8U)
    # Convert IR image into 3 channels using OpenCV image
    depth_image_3channel = cv.cvtColor(depth_image_normalized, cv.COLOR_GRAY2BGR)

errors for depth_image and ir_image are the following: Argument of type "ndarray[Unknown, Unknown] | None" cannot be assigned to parameter "a" of type "_ArrayLikeComplex_co | _ArrayLikeTD64_co | _ArrayLikeObject_co" in function "percentile" Type "ndarray[Unknown, Unknown] | None" cannot be assigned to type "_ArrayLikeComplex_co | _ArrayLikeTD64_co | _ArrayLikeObject_co" Type "None" cannot be assigned to type "_ArrayLikeComplex_co | _ArrayLikeTD64_co | _ArrayLikeObject_co" "array" is not present "len" is not present "getitem" is not present "contains" is not present "iter" is not present "reversed" is not present ...PylancereportGeneralTypeIssues

The error for norm_type is the following: No parameter named "norm_type"PylancereportGeneralTypeIssues

The expected result is not to have errors from pylance where there isn't any.

Extension version: 2023.5.20 VS Code version: Code - Insiders 1.79.0-insider (9084e081d4e89ed8ab67fce340d573c4e1378939, 2023-05-15T05:20:50.327Z) OS version: Windows_NT x64 10.0.19044 Modes:

System Info

|Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i9-10920X CPU @ 3.50GHz (24 x 3504)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|undefined| |Memory (System)|63.69GB (54.92GB free)| |Process Argv|. --crash-reporter-id e8a947eb-c0e1-46cc-ab8a-102f94e17be1| |Screen Reader|no| |VM|0%|

A/B Experiments

``` vsliv695:30137379 vsins829:30139715 vsliv368cf:30146710 vsreu685:30147344 python383cf:30185419 vspor879:30202332 vspor708:30202333 vspor363:30204092 vstes627:30244334 vslsvsres303:30308271 pythontb:30258533 pythonptprofiler:30281269 vshan820:30294714 pythondataviewer:30285072 vscod805:30301674 bridge0708:30335490 bridge0723:30353136 cmake_vspar411:30581797 vsaa593cf:30376535 pythonvs932:30404738 cppdebug:30492333 vsclangdf:30492506 c4g48928:30535728 dsvsc012:30540252 pynewext54:30618038 pylantcb52:30590116 pyind779:30611226 pythonsymbol12:30651887 showlangstatbar:30737417 azdwalk:30721579 pythonms35:30671666 19089226:30680468 57b77579:30687741 pythonfmttext:30716741 pythoncmvfstr:30726892 fixshowwlkth:30724385 hidesbindicator:30724476 pythongtdpath:30726887 ```

[erictraut]

Could you please provide a self-contained, minimal code sample that demonstrates the problem? The code sample above references a bunch of symbols that are not defined, such as k4a, np, cv, and transformation. We would need to take a guess as to what these symbols refer to. Some of them are obvious (like np), but the others are not.

Also, if your code sample references any third-party libraries, please provide the version of that library that you have installed.

[jquintanilla4]

Here's the full code, it's still a work in progress file. But yeah that's the full bit.

import cv2 as cv
from pyk4a import PyK4A, Config, ImageFormat, ColorResolution, FPS, DepthMode, transformation
import numpy as np
import open3d as o3d
import os

# Create directory to store the depth images and normals
os.makedirs('depth_images', exist_ok=True)
# Create directoy to store the infrared images
os.makedirs('ir_images', exist_ok=True)

cap_device = 1
cap_width = 1280
cap_height = 720
buffer_size = 1

# Initialize Azure Kinect
k4a = PyK4A(Config(
    color_resolution=ColorResolution.RES_720P,
    depth_mode=DepthMode.NFOV_2X2BINNED,
    camera_fps=FPS.FPS_30,
    synchronized_images_only=True,
    color_format=ImageFormat.COLOR_BGRA32))

# Function to create a point cloud from a depth image
def capture_point_cloud(depth_image):
    # Create a open3d Image object from the depth image
    depth_o3d = o3d.geometry.Image(depth_image)
    # Get the camera calibration from the Azure Kinect via open3d library
    calibration = o3d.camera.PinholeCameraIntrinsic(o3d.camera.PinholeCameraIntrinsicParameters.Kinect2DepthCameraDefault)
    # Create a point cloud from the depth image
    pcd = o3d.geometry.PointCloud.create_from_depth_image(depth_o3d, calibration, np.eye(4))
    # Flip the point cloud right side up and mirror it on the y-axis
    transformation_matrix = np.array([[-1, 0, 0, 0],[0, -1, 0, 0],[0, 0, 1, 0],[0, 0, 0, 1]])
    # Apply the transformation to the point cloud
    pcd.transform(transformation_matrix)

    return pcd

k4a.open()
k4a.start()

# Get the calibration and create a Transformation object
k4a_calibration = k4a.calibration

# Main loop
while True:
    capture = k4a.get_capture()

    ir_image = capture.ir
    depth_image = capture.depth

    # Normalize IR image using percentile method
    ir_min = np.percentile(ir_image, 1) # Find the closest to darkest pixels
    ir_max = np.percentile(ir_image, 99) # Find the closest to brightest pixels
    ir_image_clipped = np.clip(ir_image, ir_min, ir_max) # Clip the image by remove brigter than ir max and darker than ir min
    ir_image_normalized = cv.normalize(ir_image_clipped, None, alpha=0, beta=255, norm_type=cv.NORM_MINMAX, dtype=cv.CV_8U)
    # we make the brigthtest 0 and the darkest 255, all the other pixel values are adjusted proportiaonally in between
    # Convert IR image into 3 channels using OpenCV image
    ir_image_3channel = cv.cvtColor(ir_image_normalized, cv.COLOR_GRAY2BGR)

    # Transform the infrared image to the color camera's coordinate space
    transformed_depth_image = transformation.depth_image_to_color_camera(depth_image, calibration=k4a_calibration, thread_safe=True)
    # Normlize the image
    depth_image_normalized = cv.normalize(transformed_depth_image, None, alpha=0, beta=255, norm_type=cv.NORM_MINMAX, dtype=cv.CV_8U)
    # Convert IR image into 3 channels using OpenCV image
    depth_image_3channel = cv.cvtColor(depth_image_normalized, cv.COLOR_GRAY2BGR)

    # Save the transformed depth images
    cv.imwrite(f'depth_images/depth_image_{len(os.listdir("depth_images"))}.png', transformed_depth_image)

    # Display the image
    cv.imshow('IR Feed', ir_image_3channel)
    cv.imshow('Aligned Depth Image', depth_image_3channel)
    # cv.imshow('Color Image', color_image)

    # Exit the loop if needed
    if cv.waitKey(5) & 0xFF == ord('q'): # quit the program with the letter q
        try:
            k4a.stop()
            k4a.close()
        except Exception as e:
            print('Azure Kinect', e) # handles the error when the Azure Kinect is not connected
        break

# After exiting the loop, load each depth image, create a point cloude, and estimte normals
depth_image_files = os.listdir('depth_images')
for depth_image_file in depth_image_files:
    depth_image = cv.imread(f'depth_images/{depth_image_file}', cv.IMREAD_UNCHANGED)
    # create a point cloud from the depth image
    pcd = capture_point_cloud(depth_image)
    # Estimate normals
    pcd.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.1, max_nn=30))
    # Convert normals to numpy array for further processing
    normals = np.asarray(pcd.normals)
    # Save the normals
    np.save(f'depth_images/{depth_image_file[:-4]}_normals.npy', normals)

k4a.stop()
k4a.close()
cv.destroyAllWindows()

Here's the versions of everything Package Version

---

asttokens 2.2.1 attrs 23.1.0 backcall 0.2.0 click 8.1.3 colorama 0.4.6 comm 0.1.3 ConfigArgParse 1.5.3 contourpy 1.0.7 cycler 0.11.0 dash 2.9.3 dash-core-components 2.0.0 dash-html-components 2.0.0 dash-table 5.0.0 debugpy 1.6.7 decorator 5.1.1 executing 1.2.0 fastjsonschema 2.16.3 Flask 2.2.3 fonttools 4.39.3 ipykernel 6.22.0 ipython 8.12.0 ipywidgets 8.0.6 itsdangerous 2.1.2 jedi 0.18.2 Jinja2 3.1.2 jsonschema 4.17.3 jupyter_client 8.2.0 jupyter_core 5.3.0 jupyterlab-widgets 3.0.7 kiwisolver 1.4.4 MarkupSafe 2.1.2 matplotlib 3.7.1 matplotlib-inline 0.1.6 nbformat 5.7.0 nest-asyncio 1.5.6 numpy 1.24.3 open3d 0.17.0 opencv-python 4.7.0.72 packaging 23.1 parso 0.8.3 pickleshare 0.7.5 Pillow 9.5.0 pip 22.2.1 platformdirs 3.2.0 plotly 5.14.1 prompt-toolkit 3.0.38 psutil 5.9.5 pure-eval 0.2.2 Pygments 2.15.1 pyk4a 1.5.0 pyparsing 3.0.9 pyrsistent 0.19.3 python-dateutil 2.8.2 pywin32 306 pyzmq 25.0.2 setuptools 63.2.0 six 1.16.0 stack-data 0.6.2 tenacity 8.2.2 tornado 6.3.1 traitlets 5.9.0 wcwidth 0.2.6 Werkzeug 2.2.3 widgetsnbextension 4.0.7

[StellaHuang95]

Hi @jquintanilla4, I can repro this issue in both regular vscode and vscode insider. The inconsistent behavior you're observing might be due to inconsistent settings for reportGeneralTypeIssues or python.analysis.typeCheckingMode between the two versions.

The error message you're encountering indicates that the argument ir_image passed to the numpy.percentile function is of type ndarray[Unknown, Unknown] | None, which means it can either be an ndarray or None. However, the percentile function expects the argument a to be of type _ArrayLikeComplex_co | _ArrayLikeTD64_co | _ArrayLikeObject_co. The error suggests that the argument ir_image might be None, which is not compatible with the expected array-like object type. You can add a check to ensure that ir_image is a valid ndarray object and not None before passing it to the numpy.percentile function.

[jquintanilla4]

Hi @StellaHuang95, I totally agree with your assessment. Thanks for the feedback. I've updated my code since my previous reply to account for this potential issue. However, I am still getting a pylance error on the norm_type in the cv.normalize() function. Which makes no sense, sense there is a norm_type parameter.

No parameter named "norm_type"Pylance[reportGeneralTypeIssues](https://github.com/microsoft/pyright/blob/main/docs/configuration.md#reportGeneralTypeIssues)

Which again, seems to be specific only to vscode insiders edition.

[StellaHuang95]

Thanks for the report @jquintanilla4. I will send a fix for it.

[jquintanilla4]

Thanks