warpPerspective does not seem to do anything

[rstoye]

I'm trying to recreate a python script in typescript and it seems everything works fine, except that the output image looks like nothing was actually done to it.

This is the python code that works flawlessly.

import cv2
import numpy as np

# Load the template and scanned images
template_img = cv2.imread('template.jpg', 0)
scanned_img = cv2.imread('scanned.jpg', 0)

# Detect ORB features and compute descriptors
orb = cv2.ORB_create(5000)  # You can adjust the number of features to detect
keypoints1, descriptors1 = orb.detectAndCompute(template_img, None)
keypoints2, descriptors2 = orb.detectAndCompute(scanned_img, None)

# Match descriptors using Brute-Force matcher
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(descriptors1, descriptors2)
matches = sorted(matches, key=lambda x: x.distance)

# Filter matches based on distance
good_matches = matches[:int(len(matches) * 0.15)]  # Keeping 15% of the best matches

# Extract location of good matches
src_pts = np.float32([keypoints1[m.queryIdx].pt for m in good_matches])
dst_pts = np.float32([keypoints2[m.trainIdx].pt for m in good_matches])

# Compute Homography
M, mask = cv2.findHomography(dst_pts, src_pts, cv2.RANSAC, 5.0)

# Warp the scanned image to align with the template
aligned_img = cv2.warpPerspective(scanned_img, M, (template_img.shape[1], template_img.shape[0]))

# Save or display the aligned image
cv2.imwrite('aligned_scanned.png', aligned_img)

It takes the template image and the scanned image and warps the scanned image according to the template. It also scales the scanned image up to match the dimensions of the template image (1653 x 2338 -> 2480 x 3508).

template -> scanned -> aligned

Now here is my typescript code:

import cv from '@u4/opencv4nodejs';
import { dirname, join } from 'path';

export const runFeatureMatching = async () => {
  const path = '/home/rstoye/Documents/feature-matching/example/';
  const templateImage = cv.imread(join(path, 'template.jpg'), 0);
  const scannedImage = cv.imread(join(path, 'scanned.jpg'), 0);

  const orb = new cv.ORBDetector(5000);
  const templateKeypoints = orb.detect(templateImage);
  const scannedKeypoints = orb.detect(scannedImage);

  const templateDescriptor = orb.compute(templateImage, templateKeypoints);
  const scannedDescriptor = orb.compute(scannedImage, scannedKeypoints);

  const matches = new cv.BFMatcher(cv.NORM_HAMMING, true).match(
    templateDescriptor,
    scannedDescriptor
  );
  matches.sort((a, b) => a.distance - b.distance);

  const good_matches = matches.slice(0, Math.floor(matches.length * 0.15));

  const imgMatches = cv.drawMatches(
    templateImage,
    scannedImage,
    templateKeypoints,
    scannedKeypoints,
    good_matches
  );
  cv.imwrite(join(path, 'imageMatches.jpg'), imgMatches);

  const srcPts = good_matches.map((m) => templateKeypoints[m.queryIdx].pt);
  const dstPts = good_matches.map((m) => scannedKeypoints[m.trainIdx].pt);

  const { homography } = cv.findHomography(srcPts, dstPts, cv.RANSAC);
  const alignedImage = scannedImage.warpPerspective(
    homography,
    new cv.Size(templateImage.cols, templateImage.rows)
  );

  cv.imwrite(join(path, 'alignedImage.jpg'), alignedImage);
};

But instead of the expected aligned image like the one above I only get this one:

Which as you can see doesn't seem to have been warped at all.

I compared the srcPts and dstPts with the ones from the python code and they match. And the matches look like this.

I tried a few different approaches and searched online for similar problems, but I wasn't able to find anything. So I don't understand what I'm doing wrong.

Any help would be really appreciated. Thanks in advance.

---

OpenCV version: 4.9.0

OS: Manjaro Linux 24.0.2

[UrielCh]

I'm busy converting the project to N-API, I can not check that for now.

[rstoye]

Alright, thank you for the response. I'll stick to the python script for now and check back later.

[RGdevz]

you have small bug cv.findHomography(srcPts, dstPts, cv.RANSAC); should be cv.findHomography(dstPts, srcPts, cv.RANSAC);

[rstoye]

you have small bug cv.findHomography(srcPts, dstPts, cv.RANSAC); should be cv.findHomography(dstPts, srcPts, cv.RANSAC);

Well this is embarrassing, but you're absolutely right. I switched the two and now it's working as intended. Thank you very much, I appreciate the help.