jiangwenyuan / nuster

A high performance HTTP proxy cache server and RESTful NoSQL cache server based on HAProxy
Other
1.91k stars 157 forks source link

Will caching work for forward proxy? #125

Open sorbing opened 2 years ago

sorbing commented 2 years ago

I try use Nuster as cache proxy for external forward proxy. But cache not saved on disk.

global
    nuster manager on
    master-worker
    nuster  cache on  data-size 1m  dir /tmp/nuster  clean-temp off
    nuster  nosql off
defaults
    mode http
    timeout client  30s
    timeout connect 30s
    timeout server  30s
    http-reuse always
frontend fe
    mode http
    bind *:8080
    default_backend forward_proxy
backend forward_proxy
    #mode http
    option http-buffer-request
    nuster cache on dir /tmp/nuster
    http-response set-header X-Nuster-Cache ON
    server s1 XXX.XXX.XXX.XXX:XXXXX

Proxy works but directory /tmp/nuster is always empty. What am I doing wrong? Thanks.

jiangwenyuan commented 2 years ago

Can you check debug info?

sorbing commented 2 years ago

I suspect that there is no way to cache transit traffic in the middle between the client and the parent proxy. I'm a bit confused about network theory.. As I understand it, if you specify an external forward proxy server (for example, Luminati / Bright Data) as a backend, then nuster / haproxy will simply proxy the HTTP CONNECT method. And nuster/haproxy will not cache redirected traffic. Is this a correct guess? Or am I still wrong, and is it possible to configure nuster / haproxy to cache traffic in front of another external proxy, being in the middle of the chain from the client to the external proxy? I use nuster as forward proxy-server for many sites:

curl -x my_nuster_proxy:8080 https://2ip.fun

The cache is not saved with this use of the nuster server. Do you still want to see the debug log? Thanks.