search

weserv / images

Source code of wsrv.nl (formerly images.weserv.nl), to be used on your own server(s).
https://wsrv.nl/
BSD 3-Clause "New" or "Revised" License
1.86k stars 187 forks source link

Access speed using the service is slower than the original image #291

Closed zhuang010203 closed 2 years ago

zhuang010203 commented 3 years ago

微信截图_20210511111920 微信图片_20210511112007 Why use the URL of this site, access speed, not the original URL access speed?

zhuang010203 commented 3 years ago

https://share.dmca.gripe/zroi5Q3fpRm2z5Sr.png

andrieslouw commented 3 years ago

The first request will always be slower, especially when not resizing the image (the service is build for fast resizing). The next requests should be faster, but our cache will clear out images if they are not requested often.

For images.weserv.nl we process ~6 million requests per hour, and we have more than 1 million images in cache at any given moment. Our cache hit rate is around 90%, so there are ~600000 requests which are not cached each hour. It is impossible for us to cache all requests, we download around 400TB of images each month, which would need enormous amount of RAM to cache them this long.

Even 1 day of caching for all images on our public service would require 13TB (!) of very fast storage which can handle being rewritten continuously (so only RAM or really high-end SSD's). This is the reason images are removed really quickly from cache when not used often.

If we don't have the cached version of the image (anymore), they need to be fetched from the origin, which takes a bit of time. The origin (in your case dmca.gripe) can be slow, we cannot help this.

Also, when requesting an image without parameters, we will still do some sort of processing, and will generate a new .png file, this can increase the image size, especially when not resizing (we are looking into optimizing png in the near future). We won't pass any image without processing, as it would be unsecure to do so (we want to prevent someone from using our service to serve dangerous files or viruses masked as image-file).

GitBoudewijn commented 2 years ago

What if you also cache the original images on a slower drive (maybe after it is downloaded for the second time)? Then when the processed image isn't in the cache anymore it can just process the image again from there. Also that way when you use different parameters on the same image it won't have to download it again every time. (I'm always cautious when requesting the same image multiple times because I don't want your ip's to get blocked.)

kleisauke commented 2 years ago

Caching original/processed images on a hard drive will not happen due to the storage issue mentioned above. Please consider using our code to set up your own solution, if you want to do this.