justadudewhohacks / opencv4nodejs

Nodejs bindings to OpenCV 3 and OpenCV 4
MIT License
4.97k stars 827 forks source link

warpPerspective does not seem to do anything #884

Closed rstoye closed 2 months ago

rstoye commented 2 months ago

I'm trying to recreate a python script in typescript and it seems everything works fine, except that the output image looks like nothing was actually done to it.

This is the python code that works flawlessly.

import cv2
import numpy as np

# Load the template and scanned images
template_img = cv2.imread('template.jpg', 0)
scanned_img = cv2.imread('scanned.jpg', 0)

# Detect ORB features and compute descriptors
orb = cv2.ORB_create(5000)  # You can adjust the number of features to detect
keypoints1, descriptors1 = orb.detectAndCompute(template_img, None)
keypoints2, descriptors2 = orb.detectAndCompute(scanned_img, None)

# Match descriptors using Brute-Force matcher
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(descriptors1, descriptors2)
matches = sorted(matches, key=lambda x: x.distance)

# Filter matches based on distance
good_matches = matches[:int(len(matches) * 0.15)]  # Keeping 15% of the best matches

# Extract location of good matches
src_pts = np.float32([keypoints1[m.queryIdx].pt for m in good_matches])
dst_pts = np.float32([keypoints2[m.trainIdx].pt for m in good_matches])

# Compute Homography
M, mask = cv2.findHomography(dst_pts, src_pts, cv2.RANSAC, 5.0)

# Warp the scanned image to align with the template
aligned_img = cv2.warpPerspective(scanned_img, M, (template_img.shape[1], template_img.shape[0]))

# Save or display the aligned image
cv2.imwrite('aligned_scanned.png', aligned_img)

It takes the template image and the scanned image and warps the scanned image according to the template. It also scales the scanned image up to match the dimensions of the template image (1653 x 2338 -> 2480 x 3508).

template -> scanned -> aligned

Now here is my typescript code:

import cv from '@u4/opencv4nodejs';
import { dirname, join } from 'path';

export const runFeatureMatching = async () => {
  const path = '/home/rstoye/Documents/feature-matching/example/';
  const templateImage = cv.imread(join(path, 'template.jpg'), 0);
  const scannedImage = cv.imread(join(path, 'scanned.jpg'), 0);

  const orb = new cv.ORBDetector(5000);
  const templateKeypoints = orb.detect(templateImage);
  const scannedKeypoints = orb.detect(scannedImage);

  const templateDescriptor = orb.compute(templateImage, templateKeypoints);
  const scannedDescriptor = orb.compute(scannedImage, scannedKeypoints);

  const matches = new cv.BFMatcher(cv.NORM_HAMMING, true).match(
    templateDescriptor,
    scannedDescriptor
  );
  matches.sort((a, b) => a.distance - b.distance);

  const good_matches = matches.slice(0, Math.floor(matches.length * 0.15));

  const imgMatches = cv.drawMatches(
    templateImage,
    scannedImage,
    templateKeypoints,
    scannedKeypoints,
    good_matches
  );
  cv.imwrite(join(path, 'imageMatches.jpg'), imgMatches);

  const srcPts = good_matches.map((m) => templateKeypoints[m.queryIdx].pt);
  const dstPts = good_matches.map((m) => scannedKeypoints[m.trainIdx].pt);

  const { homography } = cv.findHomography(srcPts, dstPts, cv.RANSAC);
  const alignedImage = scannedImage.warpPerspective(
    homography,
    new cv.Size(templateImage.cols, templateImage.rows)
  );

  cv.imwrite(join(path, 'alignedImage.jpg'), alignedImage);
};

But instead of the expected aligned image like the one above I only get this one:

Which as you can see doesn't seem to have been warped at all.

I compared the srcPts and dstPts with the ones from the python code and they match. And the matches look like this.

I tried a few different approaches and searched online for similar problems, but I wasn't able to find anything. So I don't understand what I'm doing wrong.

Any help would be really appreciated. Thanks in advance.


OpenCV version: 4.9.0

With OpenCV-contrib? (extra modules): I'm not sure, but probably not.

OS: Manjaro Linux 24.0.2

rstoye commented 2 months ago

Nevermind, I'm stupid. I created the issue in the wrong repo.

If anyone finds this and is looking for answers, refer to this one instead: https://github.com/UrielCh/opencv4nodejs/issues/158