search

ckan / datapusher

A standalone web service that pushes data files from a CKAN site resources into its DataStore
GNU Affero General Public License v3.0
77 stars 154 forks source link

Document download proxy config option #259

Open ThrawnCA opened 2 years ago

ThrawnCA commented 2 years ago

This service supports the ckan.download_proxy option, but the README file doesn't mention it.

Configuring a secure proxy server for file downloads is important in any environment with privileged network access, such as running on an Amazon EC2 instance. Without a filter, and with a service that displays resource contents to the end user (via the datastore, in this case), anyone capable of creating a resource can point it at a private IP address and have CKAN display the potentially sensitive contents of that URL. Thus, the README file for this service should mention the importance of setting up a filtering proxy.

An example of an appropriate filter configuration is at https://feeding.cloud.geek.nz/posts/restricting-outgoing-webapp-requests-using-squid-proxy/ which gives a Squid config block (assuming Squid listen port 3128):

acl to_localnet dst 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl to_localnet dst 10.0.0.0/8            # RFC 1918 local private network (LAN)
acl to_localnet dst 100.64.0.0/10         # RFC 6598 shared address space (CGN)
acl to_localnet dst 169.254.0.0/16        # RFC 3927 link-local (directly plugged) machines
acl to_localnet dst 172.16.0.0/12         # RFC 1918 local private network (LAN)
acl to_localnet dst 192.168.0.0/16        # RFC 1918 local private network (LAN)
acl to_localnet dst fc00::/7              # RFC 4193 local private network range
acl to_localnet dst fe80::/10             # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl CONNECT method CONNECT

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny manager
http_access deny to_localhost
http_access deny to_localnet
http_access allow localhost
http_access deny all

http_port 127.0.0.1:3128