search

webp-sh / webp_server_go

Go version of WebP Server. A tool that will serve your JPG/PNG/BMP/SVGs as WebP/AVIF format with compression, on-the-fly.
https://docs.webp.sh
GNU General Public License v3.0
1.83k stars 175 forks source link

404 not found when proxy these images #215

Closed bhzhu203 closed 1 year ago

bhzhu203 commented 1 year ago

404 not found when proxy these images,

1) http://static-webp.selleroa.com/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png

2) http://static-erp.selleroa.com/2109/02/com/yfn/product/skus/KYB0114/pic/d2c76b41d02e9fa9073b4494c2ee46c4KYB0114%E9%99%88%E6%B1%89%E8%B6%8520210305-DSC_4398%E5%89%AF%E6%9C%AC%E8%9A%8A%E8%94%BC.jpg

3) http://static-webp.selleroa.com/yfni/test/1668151497580aa7295.jpg http://static-erp.selleroa.com/yfni/test/1668151497567e90685.jpg

error output: http://static-erp.oss-cn-shenzhen-internal.aliyuncs.com/yfni/test/1668151497580aa7295.jpg is not image, remote returned application/octet-stream

Please let webp-server be compatible with application/octet-stream type

n0vad3v commented 1 year ago

It seems /yfni/test/1668151497580aa7295.jpg is a corrupted image, trying to convert using libvips can meet the following error:

[govips.info] failed to understand image format size=11100605
[57:main.convertFilter.func2()] unsupported image format

From Aliyun's documentation https://www.alibabacloud.com/help/en/object-storage-service/latest/how-can-i-preview-an-object-when-i-access-the-object-by-using-its-url we know that:

The browser does not support previews of the object or image in this format.

Testing from your previous issue: https://github.com/webp-sh/webp_server_go/issues/211, you claim AliyunCDN can convert thoses images:

curl -I http://cdn3.selleroa.com/yfni/test/1668151497580aa7295.jpg
HTTP/1.1 200 OK
Server: nginx/1.23.4
Content-Type: application/octet-stream
...

So I think it's not something we should [be compatible with application/octet-stream type], try fixing your images first.

bhzhu203 commented 1 year ago

Thanks ,so how to fix the reset issues ? 1. and 2.

n0vad3v commented 1 year ago

Same problem.

I think you can test by replacing your Aliyun CDN address http://cdn3.selleroa.com for the domain, taking the first one as example:

curl -I "http://cdn3.selleroa.com/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png"
HTTP/1.1 200 OK
Server: nginx/1.23.4
Content-Type: application/octet-stream

It's returning Content-Type: application/octet-stream as your CDN might not be able to convert it too, probably a bad image as well.

n0vad3v commented 1 year ago

Sorry I've made a mistake from earlier reply, at least image /yfni/test/1668151497580aa7295.jpg can be correctly converted to WebP using WebP Server Go, we'll update this issue if we've found a better way to determine content-type from origin.

Stay tuned.

bhzhu203 commented 1 year ago

Same problem.

I think you can test by replacing your Aliyun CDN address http://cdn3.selleroa.com for the domain, taking the first one as example:

curl -I "http://cdn3.selleroa.com/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png"
HTTP/1.1 200 OK
Server: nginx/1.23.4
Content-Type: application/octet-stream

It's returning Content-Type: application/octet-stream as your CDN might not be able to convert it too, probably a bad image as well.

Not a format error ,just not found

图片

bhzhu203 commented 1 year ago

Same problem. I think you can test by replacing your Aliyun CDN address http://cdn3.selleroa.com for the domain, taking the first one as example:

curl -I "http://cdn3.selleroa.com/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png"
HTTP/1.1 200 OK
Server: nginx/1.23.4
Content-Type: application/octet-stream

It's returning Content-Type: application/octet-stream as your CDN might not be able to convert it too, probably a bad image as well.

Not a format error ,just not found

图片

Is the URL has special characters which webp-server can not handle ？

n0vad3v commented 1 year ago

Please have a try on my branch fix-proxy-mode in PR https://github.com/webp-sh/webp_server_go/pull/216, I've tested /2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png locally and it's working now.

Webp Server Go is Running on http://0.0.0.0:3333
INFO[2023-05-24][121:main.proxyHandler()] Remote Addr is http://static-erp.selleroa.com/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png, fetching info... 
INFO[2023-05-24][136:main.proxyHandler()] Remote file not found, fetching...           
INFO[2023-05-24][233:main.convertLog()] WebP@80%: remote-raw/3a7e67f36e6cc9d210e5a4e32884eb040f67b670-etag-5af6317dbb7c241316ac6a07b184d33359d78da9->exhaust/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107（1）.png.1684935469.webp_width=0&height=0 242035->39318 16.24% deflated 
WARN[2023-05-24][249:main.findSmallestFiles()] exhaust/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107（1）.png.1684935469.avif_width=0&height=0 not found on filesystem 
21:37:48 | 200 |   4.23s |       127.0.0.1 | GET     | /2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png

bhzhu203 commented 1 year ago

Please have a try on my branch fix-proxy-mode in PR #216, I've tested /2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png locally and it's working now.

Webp Server Go is Running on http://0.0.0.0:3333
INFO[2023-05-24][121:main.proxyHandler()] Remote Addr is http://static-erp.selleroa.com/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png, fetching info... 
INFO[2023-05-24][136:main.proxyHandler()] Remote file not found, fetching...           
INFO[2023-05-24][233:main.convertLog()] WebP@80%: remote-raw/3a7e67f36e6cc9d210e5a4e32884eb040f67b670-etag-5af6317dbb7c241316ac6a07b184d33359d78da9->exhaust/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107（1）.png.1684935469.webp_width=0&height=0 242035->39318 16.24% deflated 
WARN[2023-05-24][249:main.findSmallestFiles()] exhaust/2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107（1）.png.1684935469.avif_width=0&height=0 not found on filesystem 
21:37:48 | 200 |   4.23s |       127.0.0.1 | GET     | /2106/17/com/yfn/product/skus/KYB0107/pic/80fff47109bec5feb2e92dba19499ecbKYB0107%EF%BC%881%EF%BC%89.png

404 issues are all solved , but the proxy mod etag issue still exist. I need to do a hard refresh to show it manually

图片

n0vad3v commented 1 year ago

Great, closing this issue now and let's continue discuss this problem in issue: https://github.com/webp-sh/webp_server_go/issues/213