10.11 El Capitan Beta 1 Upgrade Breaks Pow

[epmatsw]

Obviously 10.11 is still pre-release, but it seems to break existing pow sites. I don't know enough to be able to debug what's actually going wrong, but a site that worked before the upgrade now just a "Problem loading page" page with "The connection was reset".

I tried re-running the install script and re-linking my site, but no dice :(

[mcfadden]

Here's what I've found:

I can hit pow on port 20559: http://127.0.0.1:20559 I can still access my sites using the xip.io syntax, ex: http://myapp.192.168.12.2.xip.io The resolver is working properly, as I can still ping myself via ping myapp.dev (which resolves to 127.0.0.1 properly)

I have uninstalled and reinstalled with no change in behavior.

If there's something else I can do to help debug, I'm glad to help.

[drewdeponte]

I am having the same problem.

I have also verified that I can ping my app.dev and it is properly resolving to 127.0.0.1. It seems that it is the redirect that isn't working properly for some reason with pf.

Granted I am no pf expert but I tried to set it up manually and it doesn't seem to be working still.

The following is what I believe is possibly relevant from pfctl -sa.

No ALTQ support in kernel
ALTQ related functions disabled
TRANSLATION RULES:
nat-anchor "com.apple/*" all
rdr-anchor "com.apple/*" all
rdr pass inet proto tcp from any to any port = 80 -> 127.0.0.1 port 20559
rdr pass inet proto tcp from any to any port = 20559 -> 127.0.0.1 port 20559

FILTER RULES:
scrub-anchor "com.apple/*" all fragment reassemble
anchor "com.apple/*" all

DUMMYNET RULES:
dummynet-anchor "com.apple/*" all

STATES:
ALL tcp 127.0.0.1:20559 <- 127.0.0.1:50324       FIN_WAIT_2:FIN_WAIT_2
ALL tcp 127.0.0.1:80 <- 127.0.0.1:50325       TIME_WAIT:TIME_WAIT

Trying to hit port 80 on 127.0.0.1 fails with connection refused.

$ curl http://127.0.0.1
curl: (7) Failed to connect to 127.0.0.1 port 80: Connection refused

However, when I hit port 20559 on 127.0.0.1 it returns the expected "Please setup a rack app page from Pow".

This tells me that the Pow rack app is running.

So it seems we have name resolving working, redirection not working, and the rack app working.

I thought maybe it was forwarding not being enabled, but nope.

$ sysctl net.net.ip.forwarding
net.inet.ip.forwarding: 1

At this point my knowledge has been exhausted. If anyone can point me further in a direction I am glad to try things. My best guess at this point is that some how pf changed in this release and it is somehow preventing the redirection from working properly.

[drewdeponte]

I also tried the comments in #480 because it seems like exactly the same symptoms.

[smeevil]

Same deal here, confirmed it works when adding the port 20559 with it.

[eploko]

Confirmed, it's broken in El Capitan.

[tomeric]

I have found a quick fix that will get the .dev domain working again. You want to edit line 37 of dns_server.js. Instead of returning "127.0.0.1", make it return "127.127.127.127".

For me this file is located in ~/Library/Application\ Support/Pow/Current/lib/dns_server.js

For some reason, port forwarding on "127.0.0.1" doesn't work, but any other IP is fine, that is why the xip.io domains work as well.

[drewdeponte]

@tomeric I am not quite sure I understand your fix.

I added an alias of 127.127.127.127 to my lo0 interface, changed the redirect rule to redirect to 127.127.127.127 and made the change you suggested but still have exactly the same problem. With exactly the same behavior.

[tomeric]

I'm afraid I don't understand my fix either ;-). You might want to try killing pow so it restarts (I don't know if you did this, but since I didn't put that in my original comment, you might not have)?

I didn't add any aliases and have removed all custom pf related configurations (that excludes /Library/LaunchDaemons/cx.pow.firewall.plist which pow generates). Tested it on 2 different machines and different networks, and it works for us.

We just figured out that since the xip.io stuff was working on port 80, that the forwarding must be active in some way. I first changed the DNS server to resolve to my local network IP, and that worked fine. Unfortunately, I switch networks quite often, so I wasn't satisfied with that fix. Had a hunch and tried to access http://127.0.0.2/ and got a response from pow. Since all IP's starting with 127. point to your own machine, I just changed it to that and it works fine.

[epmatsw]

Confirmed, the 127.127.127.127 fix followed by a restart fixed the issue for me.

[ksmandersen]

+1 for the 127.127.127.127 fix

[arcticleo]

127.127.127.127 fix doesn't seem to be an option when you use subdomains. I had to use port 20559 (subdomain.mysitename.dev:20559) to get subdomains to work.

[tomeric]

Subdomains work fine for me.

[arcticleo]

Ah subdomains started working after reboot.

[terran42]

Using 127.0.1.1 works as well. Oddly in El Capitan beta you can't ping 127.0.1.1 (or any of 127.0.0.0/8 except for 127.0.0.1), but telnet 127.0.1.1 80 still works.

[alotofnoodles]

:+1: for the 127.0.1.1fix

[mrsweaters]

I couldn't get these fixes to work in Beta 2. Anyone else still having issues resolving the DNS?

[mrsweaters]

Oh strange, the port in the /Library/LaunchDaemons/cx.pow.firewall.plist file didn't match the port specified in /etc/resolver/dev for some reason. The one in the resolver was set to 20560 instead of 20559. Changed them both to 20559 and it works now!

[LeZuse]

BTW, this might be relevant: Discoveryd is Out mDNSresponder is Back Rare Apple Backpedal

[craigtsmith]

@mrsweaters having issues resolving .dev also, your suggested fix didn't help me so back to using xip.io

[aaronjensen]

Changing the port in /etc/resolver/dev and changing line 37 to 127.127.127.127 worked for me.

[livecano]

Having same issue with 10.10.4, I tried adding the port 20559 and it works well, my guess is that any changes introduced in El Capitan already took place on the 10.10.4 OS X update, which is already breaking Pow.

[aaronjensen]

It stopped working after a reboot :/ It's different now, now it just spins "resolving host"

[agate]

@livecano I have the same issue. After I upgraded to 10.10.4 pow stop working. 20559 solution works.

[scarlac]

Can confirm that forwarding to 127.127.127.127:80 works, whereas forwarding to 127.0.0.2:80 did not seem to work for me. Requires a DNS change if you're going to use something like local.yourcompany.com, but at least it's an easy fix.

[martinbjeldbak]

For anyone still having trouble after fixing the inconsistent ports pointed out by @mrsweaters AND changing line 37 to 127.127.127.127 as described by @tomeric above, make sure to add the following to your /etc/hosts file

127.127.127.127 xxx.dev

where xxx is the site you are trying to host from ~/.pow. Ugly, but it's what finally got it to work for me.

[guidobouman]

On the latest beta uninstalling & installing through pow.cx fixed the issue for me.

[jeremy]

510

[jeremy]

(Apparently fixed in later El Cap releases.)

[jun1st]

@jeremy seems not, I having this error after a clean install of EI Cap

[ABewsher]

Hi.

Pow 0.5.0 running on Yosemite worked fine. Upgraded to El Capitan OS X 10.11 (15A284) last week - Pow stopped working.

Tried uninstall / reinstall of pow - no luck.

Tried changing 127.0.0.1 to 127.127.127.127 in dns_server.js and restarting Pow (even rebooting) no luck.

When I run pfctl -sa I don't have the lines: rdr pass inet proto tcp from any to any port = 80 -> 127.0.0.1 port 20559 rdr pass inet proto tcp from any to any port = 20559 -> 127.0.0.1 port 20559

Or anything like them.

Would really like to get to the bottom of this. Could those who managed to get it working before please confirm that it is still working. If this is the case - what versions are you running and what simple step did I miss?!

Thanks for your help.

Adam

[marczking]

reinstalling worked for me, without rebooting

[ABewsher]

@marczking, what is the output of pfctl -sa when you run it?

[marczking]

When i run pfctl -sa I get pfctl: /dev/pf: Permission denied. Not really sure what pfctl is for though ^^)

[tirdadc]

Just upgraded to El Capitan today and ran into this issue. Changing the port didn't resolve it, going to try the other suggestions.

UPDATE: uninstalling + re-installing pow did the trick, everything's back up!

[jspooner]

None of the help above fixed the issue for me. I have three people on my team and we have the same issue for each machine.

OS X Version 10.11.5

What doesn't work

[no] Uninstall and re-install [no] Update ~/Library/Application\ Support/Pow/Current/lib/dns_server.js to use 127.127.127.127 [no] edit /etc/host and add 127.127.127.127 xxx.dev [no] /Library/LaunchDaemons/cx.pow.firewall.plist'

What does work

[yes] http://127.0.0.1:20559 Pow is running [yes] ping api.dev does resolve to 127.0.0.1 [yes] api.dev:20559 is available

[ivana-mcconnell]

@jspooner I'm also having this issue. Going to: my-app.dev just returns the page that says "It works!" Going to my-app.dev:20559 shows my site correctly, but how do I fix that? How do I make my-app.dev display what's on my-app.dev:20559 without having to add the port?

I can follow instructions and have some terminal experience but not capable of advanced dev stuff :/ Thanks in advance.

[mileandra]

+1 for reopening. This still seems to be an issue in El Capitan. I can access all sites through port 20559, but not through port 80.

[jeremy]

If you can access on port 20559 (where Pow listens) but not 80 (where a pf firewall rule directs traffic to port 20559), please reinstall Pow so the firewall rule is recreated.

[srt32]

I had the same symptoms as mentioned in https://github.com/basecamp/pow/issues/507#issuecomment-228841798 and uninstalling / reinstalling did not resolve it.

Following the steps in https://github.com/basecamp/pow/issues/172#issuecomment-288258119 solved the problem for me.

TLDR:

sudo pfctl -f /etc/pf.conf

followed by

sudo pfctl -e

I'm on OSX 10.12.3 and Pow 0.5.0