Black screen when using remote image url with VirtualBackgroundProcessor

[iamhrayr]

• [x] I have verified that the issue occurs with the latest twilio-video-processors.js release and is not marked as a known issue in the CHANGELOG.md.
• [x] I reviewed the Common Issues and open GitHub issues and verified that this report represents a potentially new issue.
• [x] I am not sharing any Personally Identifiable Information (PII) or sensitive account information (API keys, credentials, etc.) when reporting this issue.

Code to reproduce the issue:

const getImage = (imageUrl: string) => {
  return new Promise((resolve, reject) => {
    const img = new Image();
    img.src = imageUrl; // <-- some public url, e.g. https://cdn.pixabay.com/photo/2021/11/21/21/14/mountain-6815304_960_720.jpg
    img.onload = () => resolve(img);
    img.onerror = reject;
  });
};

// ...

const virtualBackgroundProcessor = new VirtualBackgroundProcessor({
  assetsPath: virtualBackgroundAssets,
  backgroundImage: await getImage(remoteImageUrl),
  fitType: ImageFit.Cover,
});
await virtualBackgroundProcessor.loadModel();
addProcessor(virtualBackgroundProcessor);

Expected behavior: IT should show the correct background or throw some error in case any.

Actual behavior: Shows black screen

Software versions:

• [x] Browser(s): Chrome Version 96.0.4664.55 (Official Build) (x86_64)
• [x] Operating System: MacOS Big Sur | Version 11.0.1 (20B50)
• [x] twilio-video-processors.js: v1.0.2
• [x] Third-party libraries (e.g., Angular, React, etc.): React v16.14.0

[charliesantos]

Hi @iamhrayr , Thanks for reporting. What is your application's domain? If it's different than your image's domain, you are probably running into cross origin issues. Please let me know what location.origin returns from your dev console.

If it's cross origin, please check this section of our docs when loading an image from a different origin. Basically, you need to make sure the server hosting your image should return the correct CORS headers, and set the crossOrigin attribute of the image object.

const getImage = (imageUrl: string) => {
  return new Promise((resolve, reject) => {
    const img = new Image();
    img.crossOrigin = 'Anonymous'; // <-- IMPORTANT if serving from a different origin
    img.src = imageUrl;
    img.onload = () => resolve(img);
    img.onerror = reject;
  });
};

Unfortunately, you will not see an error when you run into this kind of issue. The browser fails silently when running into CORS issues when loading an image to a canvas.

[iamhrayr]

@charliesantos thanks for the quick response. Yeah, it was the problem. After configuring the CORS of the server everything works as expected

closing the issue since it is not related to this library.