Upload with w3s subdomain URL failed with "file could not be imported from URL because it was not accessible" in task-runner

[yondonfu]

I encountered this problem while following the Transcoded Video Storage and Playback with web3.storage guide using the nodejs code example which uses the transcode API under the hood.

The input file URL was https://bafybeigwz6ocuxhih3s3lm3hc4b4pmxc2hgy2rtjsjrtin6o5beumbvapy.ipfs.w3s.link/. If you copy and paste this in your browser you should be able to playback the video in browser from the w3link gateway. When I used this input file URL with the w3s nodejs code example, I saw that the task hit the max # of retries and the final status was:

"status":{"phase":"waiting","retries":2,"updatedAt":1677186972566,"errorMessage":"file could not be imported from URL because it was not accessible"}

I believe the errorMessage value is the humanized error set in task-runner when catalyst cannot download the file from the URL.

https://bafybeigwz6ocuxhih3s3lm3hc4b4pmxc2hgy2rtjsjrtin6o5beumbvapy.ipfs.w3s.link/ is the subdomain version of https://w3s.link/ipfs/bafybeigwz6ocuxhih3s3lm3hc4b4pmxc2hgy2rtjsjrtin6o5beumbvapy which will automatically re-direct to https://bafybeigwz6ocuxhih3s3lm3hc4b4pmxc2hgy2rtjsjrtin6o5beumbvapy.ipfs.w3s.link/ when you copy and paste it in your browser. If I use https://w3s.link/ipfs/bafybeigwz6ocuxhih3s3lm3hc4b4pmxc2hgy2rtjsjrtin6o5beumbvapy as the input file URL, the task completes successfully.

For more information on "path style" and "subdomain style" for w3link refer to the docs.

The failure with the subdomain URL was surprising because I could paste that into the browser and I was able to playback the video fine.

[yondonfu]

Issue moved to livepeer/catalyst-api #485 via Zenhub