PfSense Configuration

[Dannyzen]

Curious if anyone has any success with setting PfSense up to use an algo enabled server and if they wouldn't mind sharing how they went about applying their configuration.

[professoruss]

I'm working on getting this working with not so great of success so far. My fork is located here: https://github.com/trailofbits/algo/compare/master...professoruss:pfsense_is_silly It seems as if PFSense either only lists sha256 instead of sha2-256 even though the latter is available in the shell, just not exposed to the GUI so I added some lower algorithms to be able to establish the tunnel. Still having issues getting traffic routed.

[davidemyers]

I've got something that seems to work. It requires changing the Algo configuration in order to accommodate pfSense. Much more testing is needed.

I wrote up some instructions here.

Feedback on this approach is appreciated.

[professoruss]

Nice, that seems to mostly work but I'm getting a bunch of denies in the firewall logs. Some stuff works, some doesn't, I have an ipsec rule allowing all traffic. Not quite sure what's going on.

[Nnyan]

I'll give this a try when I get home tonight.

[bensyverson]

I spent all day on this with pfSense tech support… We followed the guide @davidemyers posted, to a T, and were able to get pfSense to connect to an Algo instance on Digital Ocean. However, once connected, local clients can no longer reach the internet. The connection comes back as soon as IPSec is toggled off. There are no denies on outgoing traffic, and I have an IPSec rule to allow everything. Any help would be greatly appreciated!

[davidemyers]

Are you using 2.4 like I am? Maybe that's the difference.

[bensyverson]

@davidemyers — Hmm, no, I'm on 2.3.3. I'll look at the changelog for 2.4.

[bensyverson]

Okay, I upgraded to 2.4, rebuilt the Algo instance, and everything is working! 🎉

I'm getting great throughput using Algo on Digital Ocean talking to a Netgear SG-2220. My native speed is about 90/11, and I'm measuring around 89/10 through IPSec.

I think my problem was in the vpn_network setting in config.cfg. It should match your pfSense network, and it should be specified in subnet format (10.0.1.0/24 rather than the router's address, e.g. 10.0.1.1). D'oh!

Huge thanks to @davidemyers for the guide!

[grimsbee]

@bensyverson did you need to do any particularly tuning (other than MSS clamping)? I'm using a machine a little more powerful than a SG-2220 and can't get past about 20Mbps (1/9th of native speed) on Digital Ocean. Nothing obvious showing up in packet captures or logs, so I'm hoping I've just missed something dumb.

[davidemyers]

@grimsbee Under System / Advanced / Miscellaneous does it help to select AES-NI for Cryptographic Hardware?

[grimsbee]

not at all, neither pro or con, and the cpu supports the instruction set

[bensyverson]

@grimsbee Hmm, I didn't adjust anything else, and my tunables should be set to the defaults. I wiped the stock pfSense entirely and installed 2.4.0.b.20170411.1439, if that helps.

If you download a file from your DO instance with IPSec turned off, are you getting above 20Mbps? I'm wondering if the throughput from your local ISP to that DO datacenter could be an issue.

[grimsbee]

I'm able to get 120MBps or so on OpenVPN to a same size droplet in the same AZ, so i don't think it's that. I had been hoping IPSEC would improve from there.

[shorage]

Finally, I was able to get this working with the following changes

-pfSense-CE-memstick-2.4.0-BETA-amd64-20170415-0221 -Phase 1 Hash was changed to SHA512 (would not work otherwise)

So, the following are observations;

• All Lan Traffic goes through Algo. Nothing gets checked by my Squid package (Any ideas how to rectify?)
• Cant access any other interface from lan (Trying to RDP into a windows box in my DMZ for example
• Cant access my hosted website (even with public ip address)

Any ideas guys? So far so good. I am getting 209MBps! But i need the other items rectified.

Thanks

Frank

[davidemyers]

I see that Algo changed from using SHA256 to SHA512 a few days ago, breaking my instructions. I'll add a note to the instructions and later revise all of the screenshots.

EDIT: @shorage I think you need to also change your Phase 2 Hash setting or your tunnel might fail when it tries to rekey.

[dguido]

Yes, that was for this ticket: https://github.com/trailofbits/algo/issues/247

[msmollin]

Been hassling trying to get this working on 2.4.0.b.20170506.2251. Followed the instructions in the google doc to a T (which are awesome btw). Have deleted and recreated the config inside pfsense at least 4 times. Here's what I see on the pfsense side (IP addresses scrubbed):

https://gist.github.com/msmollin/ce1d88dff362200f0b28fe25d2b54578

and here's what I see on the Algo side:

https://gist.github.com/msmollin/89d6ef94c7e91999e8022f5bd44fdfe2

The logs should match up pretty well. There's a ridiculous amount of this garbage in the logs on the pfsense side:

May  8 00:19:20 guardian charon: 06[KNL] creating acquire job for policy <pfsense ip>/32|/0 === <algo ip>/32|/0 with reqid {1}
May  8 00:19:20 guardian charon: 02[CFG] ignoring acquire, connection attempt pending

Which seems like its rekeying super aggressively but I haven't set anything to cause that? Any thoughts?

[davidemyers]

I haven't seen that behavior, but I've run into so many bugs in IPsec in pfSense 2.4 that I've given up on it for now.

What I'd really like to get to work is using a Transport Mode phase 2 with a GRE or GIF tunnel, as that will allow much more control over what goes over IPsec and what does not, but I keep running into bugs with incorrect firewall states.

I haven't encountered any bugs around rekeying, but that could have changed in a recent build.

[drics]

I can't run ./algo on my pfSense

[2.3.4-RELEASE][admin@pfSense.localdomain]/: ls -l total 32869 -rw-r--r-- 2 root wheel 898 May 4 00:02 .cshrc -rw-r--r-- 1 root wheel 188 May 4 00:02 .profile drwxrwxr-x 2 root operator 512 May 20 16:30 .snap -r-------- 1 root wheel 33521664 May 20 16:30 .sujournal -r--r--r-- 1 root wheel 6142 May 4 00:02 COPYRIGHT drwxr-xr-x 9 root wheel 512 May 20 19:50 algo-master drwxr-xr-x 2 root wheel 1024 May 4 00:15 bin drwxr-xr-x 8 root wheel 1536 May 20 19:45 boot -rw-r--r-- 1 root wheel 12 May 20 19:44 boot.config drwxr-xr-x 3 root wheel 512 May 3 20:07 cf lrwxr-xr-x 1 root wheel 8 May 4 00:15 conf -> /cf/conf drwxr-xr-x 2 root wheel 512 May 20 16:32 conf.default dr-xr-xr-x 10 root wheel 512 May 20 19:21 dev drwxr-xr-x 26 root wheel 4096 May 20 19:45 etc drwxr-xr-x 2 root wheel 512 May 4 00:15 home drwxr-xr-x 3 root wheel 1536 May 4 00:15 lib drwxr-xr-x 3 root wheel 512 May 4 00:15 libexec drwxr-xr-x 2 root wheel 512 May 4 00:01 media drwxr-xr-x 2 root wheel 512 May 4 00:01 mnt dr-xr-xr-x 2 root wheel 512 May 4 00:01 proc drwxr-xr-x 2 root wheel 2560 May 4 00:15 rescue drwxr-xr-x 2 root wheel 512 May 4 00:15 root drwxr-xr-x 2 root wheel 2560 May 4 00:15 sbin drwxr-xr-x 2 root wheel 512 May 20 16:31 scripts lrwxr-xr-x 1 root wheel 11 May 4 00:01 sys -> usr/src/sys drwxrwxrwt 3 root wheel 1024 May 20 19:47 tmp drwxr-xr-x 14 root wheel 512 May 3 20:07 usr drwxr-xr-x 27 root wheel 512 May 20 16:45 var [2.3.4-RELEASE][admin@pfSense.localdomain]/: cd algo-master/ [2.3.4-RELEASE][admin@pfSense.localdomain]/algo-master: ls -l total 116 drwxr-xr-x 2 root wheel 512 May 20 19:50 .github -rw-r--r-- 1 root wheel 65 May 16 23:30 .gitignore -rw-r--r-- 1 root wheel 2012 May 16 23:30 .travis.yml -rw-r--r-- 1 root wheel 761 May 16 23:30 CONTRIBUTING.md -rw-r--r-- 1 root wheel 1080 May 16 23:30 LICENSE -rw-r--r-- 1 root wheel 16025 May 16 23:30 README.md -rw-r--r-- 1 root wheel 13499 May 16 23:30 algo -rw-r--r-- 1 root wheel 356 May 16 23:30 ansible.cfg -rw-r--r-- 1 root wheel 2958 May 20 19:40 config.cfg drwxr-xr-x 2 root wheel 512 May 20 19:50 configs -rw-r--r-- 1 root wheel 2596 May 16 23:30 deploy.yml -rw-r--r-- 1 root wheel 1455 May 16 23:30 deploy_client.yml drwxr-xr-x 2 root wheel 512 May 20 19:50 docs -rw-r--r-- 1 root wheel 77 May 16 23:30 inventory drwxr-xr-x 2 root wheel 512 May 20 19:50 library -rw-r--r-- 1 root wheel 10263 May 16 23:30 logo.png drwxr-xr-x 3 root wheel 512 May 20 19:50 playbooks -rw-r--r-- 1 root wheel 149 May 16 23:30 requirements.txt drwxr-xr-x 13 root wheel 512 May 20 19:50 roles drwxr-xr-x 2 root wheel 512 May 20 19:50 tests -rw-r--r-- 1 root wheel 1784 May 16 23:30 users.yml [2.3.4-RELEASE][admin@pfSense.localdomain]/algo-master: ./algo ./algo: Permission denied. [2.3.4-RELEASE][admin@pfSense.localdomain]/algo-master: chmod +x algo [2.3.4-RELEASE][admin@pfSense.localdomain]/algo-master: ls -l total 116 drwxr-xr-x 2 root wheel 512 May 20 19:50 .github -rw-r--r-- 1 root wheel 65 May 16 23:30 .gitignore -rw-r--r-- 1 root wheel 2012 May 16 23:30 .travis.yml -rw-r--r-- 1 root wheel 761 May 16 23:30 CONTRIBUTING.md -rw-r--r-- 1 root wheel 1080 May 16 23:30 LICENSE -rw-r--r-- 1 root wheel 16025 May 16 23:30 README.md -rwxr-xr-x 1 root wheel 13499 May 16 23:30 algo -rw-r--r-- 1 root wheel 356 May 16 23:30 ansible.cfg -rw-r--r-- 1 root wheel 2958 May 20 19:40 config.cfg drwxr-xr-x 2 root wheel 512 May 20 19:50 configs -rw-r--r-- 1 root wheel 2596 May 16 23:30 deploy.yml -rw-r--r-- 1 root wheel 1455 May 16 23:30 deploy_client.yml drwxr-xr-x 2 root wheel 512 May 20 19:50 docs -rw-r--r-- 1 root wheel 77 May 16 23:30 inventory drwxr-xr-x 2 root wheel 512 May 20 19:50 library -rw-r--r-- 1 root wheel 10263 May 16 23:30 logo.png drwxr-xr-x 3 root wheel 512 May 20 19:50 playbooks -rw-r--r-- 1 root wheel 149 May 16 23:30 requirements.txt drwxr-xr-x 13 root wheel 512 May 20 19:50 roles drwxr-xr-x 2 root wheel 512 May 20 19:50 tests -rw-r--r-- 1 root wheel 1784 May 16 23:30 users.yml [2.3.4-RELEASE][admin@pfSense.localdomain]/algo-master: ./algo env: bash: No such file or directory [2.3.4-RELEASE][admin@pfSense.localdomain]/algo-master: cd .. [2.3.4-RELEASE][admin@pfSense.localdomain]/: algo-master/ ./algo algo-master/: Permission denied.

[davidemyers]

Although I previously wrote that I was giving up on pfSense for now, I've implemented a better way to modify Algo to accept connections from pfSense or other routers. Unlike my previous instructions, you no longer have to dedicate the Algo server to the router. You also don't need to know what subnets you wish to route before setting up the Algo server.

The new instructions for pfSense can be found here.

The script I wrote that gets installed on the Algo server is there as well. The instructions for installing the script are found within the script itself. It should support connections from routers other than pfSense.

Edit: The new instructions are now in a Gist rather than Google Docs. Edit again: Now the instructions are in a Repository rather than a Gist.

[msmollin]

Might try to give this a whirl this weekend. Still working for you?

[davidemyers]

I just set up a new Algo instance this morning and it seems to still work fine. Algo just reverted back to strongSwan 5.3.5 from 5.5.1 and I wanted to make sure the script still worked.

I need to upgrade my router hardware before I can run pfSense 2.4 on it so I'm still just testing this in a VM and not using it regularly, but for IPv4 tunnels it's working well.

[msmollin]

Finally gave the new script a try last night. Any thoughts on why, on the pfsense side, I'd be seeing no private key found for "CN=router" ? I tried upgrading pfsense to the latest 2.4 beta but that didn't fix anything. I might try removing that line from the rightid in the updown script (maybe it got changed - so hard to follow their betas).

[davidemyers]

Where are you seeing this message in pfSense, the IPsec log? I'm assuming you copied in the private key when adding the certificate?

pfSense doesn't fully support the ECDSA keys created by Algo, but they've been working for me anyway.

[msmollin]

Indeed yeah the IPSec log on the pfSense side. I don't see anything relevant on the Algo side... just that eventually it tears down the half-open tunnel.

I did copy in the private key, and yes it seemed happy with the Cert+Key combo. I am going to try deleting the cert and re-importing it first, and then if that fails remove the CN= part from the script and see if that helps.

[davidemyers]

My cert looks like this:

[msmollin]

Looks similar:

[msmollin]

I blanked out the IP addresses incase I decide to keep this droplet :)

[defunctio]

Could you guys please take this conversation over to gitter. We are trying to keep general support, especially for features that are unofficially supported out of github issues.

Thanks.

[msmollin]

@defunctio Can we get a specific room designated for pfSense clients then? The general Algo room is quite noisy and I would rather we not lose debugging information due to gitter's lack of threaded conversations.

[davidemyers]

Maybe we can just move to the Gist where I have the instructions.

[defunctio]

I don't think you can create rooms but you can try the #algo-support slack channel since it supports threading.

[msmollin]

@davidemyers so instead of using the Gist I forked algo and opened an issue in my fork to continue the discussion. This way we will continue to receive github notifications which Gist's do not support.

https://github.com/msmollin/algo/issues/1

Of note, that issue contains how I resolved my issue with this, and successfully connected.

[atwoodjw]

@davidemyers Can you elaborate on: "You must also have a Phase 2 that covers the pfSense LAN interface address."?

I'm following your new instructions. I set the DNS Resolver to LAN under Outgoing Network Interfaces, but I'm seeing ads which should be blocked (and are blocked on normal Algo VPN clients).

[davidemyers]

It sounds like you want to have pfSense use the ad blocking DNS forwarder installed on your Algo instance, which is something I haven't tried. My instructions are for sending the router's normal DNS traffic through the VPN and out to the Internet.

To use the ad blocking forwarder for all DNS from your router you'll have to configure the DNS Resolver to act as a forwarder and set your DNS server to 172.16.0.1 in System > General Setup, but this seems like a bad idea to me as you won't have DNS at all on your router when IPsec is down.

Alternatively you can try having your DHCP Server advertise 172.16.0.1 as the DNS server for clients to use. Your clients won't be able to use DNS when IPsec is down but your router DNS configuration won't have to change.

[atwoodjw]

Brilliant. Thank you.

[RoyTinker]

Any idea how to bypass the VPN on pfSense for certain destination IP addresses? A lot of online services have “VPN blockers” that don’t allow cloud machine IPs (like DigitalOcean).

(I used the 2nd pfSense guide)

[davidemyers]

The only way I know of is to create multiple Phase 2 entries specific to the hosts you do want to send over the VPN. Each entry can match a single IP address or a range of addresses using a mask. You could reassign your host addresses into "goes out the VPN" and "bypasses the VPN" ranges.

This is no fun and quickly gets unwieldy when you try to handle IPv6 addresses, which are usually automatically assigned and might change every day.

What pfSense is missing is the ability to do policy routing with IPsec. I've read that this might one day be added.

I find it easier to connect each of my clients directly to the VPN rather than use pfSense as it is now. If you have multiple clients connecting through pfSense be sure to change your pfSense NAT settings as I describe in #520.

[RoyTinker]

What pfSense is missing is the ability to do policy routing with IPsec. I've read that this might one day be added.

It looks like routed IPSec via VTI was just released in the pfSense stable release line (v2.4.4). Here's the doc page for it: https://www.netgate.com/docs/pfsense/vpn/ipsec/ipsec-routed.html

Any tips on setting that up with Algo?

[davidemyers]

I took a look at using VTI, but from what I could tell, in order to use VTI on pfSense with IPv6 you have to use a Phase 1 connection over IPv6, but Algo only accepts IPsec connections over IPv4.

[RoyTinker]

@davidemyers I don't need IPv6 at the moment. My Algo instance has a static IPv4 address.