campjs / campjs-iii

The magic behind campjs.com
http://campjs.com/
32 stars 12 forks source link

Let's make campjs 2014 local network connectivity awesome. #3

Closed eugeneware closed 7 years ago

eugeneware commented 10 years ago

Wifi and local network connectivity was a big challenge for the last campjs. Let's all put on our propeller hats and think of creative ways to make the next campjs better.

eugeneware commented 10 years ago

My suggestions:

  1. Worse case - set up a wired network for the large hacking areas. Still have WIFI, but that should be option B, not option A.
  2. Look at setting up a separate "presenter/staff only network" to make sure that speakers can at least get the access they need to deliver their material in the best possible way.
  3. Get some wifi conference guru to help us with the issue.
  4. Ban any local wifi hotspots as they will contribute to the wifi contention.
  5. Have our local campjs server on a wired connection.
  6. Add some 3G routers to supplement the internet connectivity with some good external antennas to maximise the local camp bandwidth.
aussiegeek commented 10 years ago

I don't have advanced wifi knowledge, but happy to help in general with making networking and such more awesome

eugeneware commented 10 years ago

Found some useful links:

timoxley commented 10 years ago

The device we currently have at the internet source, as recommended by a network dude is a RB2011UAS-2HnD-IN: 523_l

At the campsite also had 3 of these UniFi AP units:

"Our standard model 802.11n MIMO UniFi AP. Capable of speeds up to 300 Mbps with a range of up to 400 ft."

ubnt-uap-1

timoxley commented 10 years ago

@geoffreyd @mipearson you may want to chime in on this with what you observed/learned. I noticed things were a lot smoother after @mipearson had a proxy set up.

mipearson commented 10 years ago

So, @geoffreyd did the initial improvement by moving one of the UniFi APs into the main hall, thus sharing the ~100 device load over two aps. These APs are rated to above 30 devices per AP, not 50 or 100.

My work was simpler: I set up a transparent proxy. This had three benefits:

geoffreyd commented 10 years ago

Yup, the load balancing was the biggest improvement to reducing local packet loss, and the proxy for reducing internet packet loss.

Personally, I'm not sure I'd go for the UniFi AP's again, unless we had a whole bunch. I've seen much better performance from Airport Extremes, and easier setup to boot.

If we where to have Internet again, I think we might be better of blocking all access unless through a non transparent proxy, this means it would need to be intentionally setup, so only those who really wanted it would get it.

The router we got was great, while configuring it wasn't as easy as something like IPCop, it was very powerful, and changes went into effect immediately (most devices need to reboot for changes).

Banning local hotspots will help, but also providing separate 2.5 & 5Ghz networks will also help spread the load. I think if we have a solid wireless setup, we shouldn't have any need to provide a separate presenter one.

nog3 commented 10 years ago

It's important to note that some Macs with broadcom wifi will have issues with network equipment. Unifi stuff is usually pretty solid but it may be worth investigating a mac-only network or really just a separate g and n network.

sidorares commented 10 years ago

My 2c: create raspberry pi image to use PIs (without any additional configuration) as access point (i think we had over 10 PI in total, maybe 20). Have ethernet hub with enough cables for all PIs. Install some advanced dhcp/dns/http proxies. I'm happy to try to implement dns/dhcp part in node.

timoxley commented 10 years ago

The PI solution sounds interesting, especially since the devices are so cheap.

geoffreyd commented 10 years ago

Having so many access points in the same area seems like it would cause alot of noise ... 2.4ghz only really has 3 non overlapping bands (1, 6 & 11) ... I guess we'd have to see if this is more of an issue than too many devices on the same channels.

Can the Pi's to 5gz? if they can, then we could spread things out more.

With getting the Pi's to be AP's are they able todo 'extended networks', as in allowing devices to seamlessly move between AP's ... this is something the UniFi's and Airports can do, but is not a standard feature for many wireless devices.

DNS/DHCP is not an issue, the router that we already have is more than capable of doing this, and has a nice interface for it.

Geoffrey D.

On Thu, Aug 15, 2013 at 11:16 AM, Tim Oxley notifications@github.comwrote:

The PI solution sounds interesting, especially since the devices are so cheap.

— Reply to this email directly or view it on GitHubhttps://github.com/campJS/campjs/issues/3#issuecomment-22679404 .

mipearson commented 10 years ago

This sounds clever, but risky: has this been attempted before with over 100 clients?

Remember that each AP needs to sit on its own channel, and those channels have crossover.

mipearson commented 10 years ago

Oh. What @geoffreyd said :)

Paging @wolfeidau to the thread

sidorares commented 10 years ago

@geoffreyd - I only tried 2.4Ghz usb wifi with PI (you can get them for 5 to 12$) 5Ghz seems to be more expensive - http://www.ebay.com/bhp/usb-wireless-adapter-5ghz - 20 to 50$

mipearson commented 10 years ago

Worth noting that the dual-band Airport Express is much cheaper at $120/unit than the dual band UniFi PRO at $270/unit.

I agree with @geoffreyd, and I think we should just buy a few airport express units and leave it at that.

eugeneware commented 10 years ago

I agree. The airport express option is sounding pretty good. I guess we put them into bridge mode and hard wire the expresses to the main router? That way the main router can handle the NAT / DHCP management without melting down.

Were the APs at campjs this year wired together with ethernet?

mipearson commented 10 years ago

Correct. Also, I suspect the bigpond modem (not the routeros unit) handled NAT&DHCP, which may have been part of the problem. Not 100% sure on this though.

geoffreyd commented 10 years ago

Yeah, we had Cat5e connecting all the AP's and the router was doing the NAT/DHCP/DNS.

I'm not sure about the newest Airport express's, but the old (non apple tv looking ones), couldn't handle lots of connections, it was the Airport Extremes that stood up well. Given that they also do duel band, this would help spread the load.

Geoffrey D.

On Thu, Aug 15, 2013 at 12:25 PM, Eugene Ware notifications@github.comwrote:

I agree. The airport express option is sounding pretty good. I guess we put them into bridge mode and hard wire the expresses to the main router? That way the main router can handle the NAT / DHCP management without melting down.

Were the APs at campjs this year wired together with ethernet?

— Reply to this email directly or view it on GitHubhttps://github.com/campJS/campjs/issues/3#issuecomment-22681520 .

geoffreyd commented 10 years ago

@mipearson The net connection was double NAT'd, so the bigpond router saw our router as the only device. The biggest issue was just the pure number of connections ... I think having a manual proxy would eliminate a number of these issues.

mipearson commented 10 years ago

According to apple, both the most recent Express & Extreme handle "up to 50 simultaneous users". No idea what that means in real world terms.

mipearson commented 10 years ago

I agree with the manual proxy idea, btw.

geoffreyd commented 10 years ago

Yes, but in real life, they do closer to ~150 pretty well. This post linked to by @eugeneware above seems to agree.

It might be that the expresses have upped their game recently.

iamkevinv commented 10 years ago

I'm not convinced that we really need to manual proxy - kind of defeats the purpose of providing a network at all if we need people asking us "please sir, may I have some Internet?".

If we really enforce the Internet free campjs idea then that's probably appropriate, but as interesting the idea is, I think campjs needs Internet. The transparent prox and other systems in place did really well, the only thing I feel we need to hit on the head is the number of LAN connections. Once you had a connection the Internet speed was more than good enough. I'm actually really keen to do research into using a mesh network at campjs. The more people you have, the better it works. It's an area of interest for me, so ill check it out. We'd end up patching in the endpoints of a network back into the AP's or cat5.

Also FYI, as much as I love raspberry pi, they aren't great networkers. The 10/100 on board is actually on the USB bus, and uses CPU. Also, I've not got 5ghz dongles to work under raspian. I spent ages and compiled the drivers required for one I have, and it will work great until it drops the connection every 20min. Forget it for now.

iamkevinv commented 10 years ago

Eugene's points 4,5,6 are really good to put in place. Wifi scanning tools periodically can ping hotspots and name and shame and block :-)

chrisdewar commented 10 years ago

I am also pro-internet... even bad internet is better than no internet.

Regarding airport express's - it seemed there were a lot of apple users at campjs 2, i'll bet more than a few have airport express's - i know i keep one in my gadget pack as an essential. What about BYO airport?

timoxley commented 10 years ago

Is there some easy way we can load test a solution prior to the camp?

colingourlay commented 10 years ago

Yes, have everyone connect at once from phones and laptops at a BrisJS. That should give us 40+ devices.

chrisdewar commented 10 years ago

Vendor-Independent Access Point Analysis

timoxley commented 10 years ago

tl;dr.

chrisdewar commented 10 years ago

tl;dr = airport extreme > *

geoffreyd commented 10 years ago

It also show we had probably the worst choice for such a high number of apple devices. The unifi's pretty much came last.

On Thursday, August 15, 2013, Chris Dewar wrote:

tl;dr = airport extreme > *

— Reply to this email directly or view it on GitHubhttps://github.com/campJS/campjs/issues/3#issuecomment-22702254 .

Geoffrey D.

eugeneware commented 10 years ago

This is some great info guys. So it's sound like that getting a bunch of Airport Extremes and using our existing router is looking like the way to go?

timoxley commented 10 years ago

can anyone give me a summary as to why airport extremes are good for this stuff? What do they have that other devices do not?

geoffreyd commented 10 years ago

Basically, what they have that others don't is: people recommending (and successfully using) them.

Geoffrey D.

On Fri, Aug 16, 2013 at 3:27 PM, Tim Oxley notifications@github.com wrote:

can anyone give me a summary as to why airport extremes are good for this stuff? What do they have that other devices do not?

— Reply to this email directly or view it on GitHubhttps://github.com/campJS/campjs/issues/3#issuecomment-22748220 .

eugeneware commented 10 years ago

I think (from the report that @chrisdewar linked to) - it was that the channel width was 40Mhz as opposed to 20Mhz for a lot of the other offerings. Not that I really know what that means though! :-)

chrisdewar commented 10 years ago

IIRC Airport was the first Wifi, Steve Jobs presented it with an ibook and a hoola hoop... so Apple have been in the Wifi game since day dot. As for what it has that it's competitors don't, that is a tough one... especially when Apple plays it's cards so close to the chest, but I'll bite...

and something that most AP vendor's don't have...

If I read you correctly @timoxley, we were using the Unify UAP's, not the UAP-PRO's. That means no 5Ghz, 20dBm and 2 antennas.

I pasted the AP analysis above for two reasons.

  1. It was the first one I found
  2. It tested against Apple devices

To me the analysis read less like "Apple AP > *" and more like "Apple AP + Apple Device > Non-Apple AP + Apple Device", which was why I thought it was important to bring to this discussion... because love it or hate it, the CampJS install base was quite heavily swayed towards MacBooks, and it would seem that they play especially nice with Airport devices.

All that being said (gasp), I didn't really have any Wifi connection issues, I got disconnected when I took my laptop (15" rmbp) over to the camp fire... but could always connect when within range, and the local network worked pretty smoothly for me. My only bad network experience was with regard to the internet...

How many people were complaining of a bad network connection vs a bad internet connection?

iamkevinv commented 10 years ago

That guy's anecdotal claim of airport extreme hitting 170 connections is kinda wonderful, if it were true. However purely data wise how many of you read the "Vendor-Independent Access Point Analysis" report as above?

...with regards to the Airport Extreme, the advantage is it's wider channels == higher throughput in best case situations. However, if we're purely speaking about our needs... we need the highest reliability under LOTS of users. The one rated to do this in the stress test was "Ruckus 7982". See page 65 and compare with 53, paying close attention to the dark grey and the red line. The dark grey average throughput line is actually the same between the two (meaning the Airport Extreme has no advantage on average) and the Ruckus performs way better between 16 to 25 users than airport (though Airport is the best out there for home usage as this shows, bar none). It still got ~11% through at 25 users. http://www.ruckuswireless.com/products/zoneflex-indoor/7982

Also if we're dividing user groups over the different bands 5ghz+2.4ghz, that would be quite adequate. 3 access points, 3 wifi channels on 2.4ghz (1,6,11), run 3 channels on 5ghz too. If you wanted, you could get an extra access point on the 5ghz and spread the channels evenly too.

Worth knowing: The Ruckus looks like they are ~$900 while the new Airport extreme is $249.00.

PS: I'll let you know if a mesh network is viable. I'd like to get it working on macbooks without much/any extra hardware. If not, then i'm inclined to leave it until i have a better suggestion.

eugeneware commented 10 years ago

Hi @kevdesign - thanks for summary.

Forgive my ignorance, but in what you're proposing, how do you divide the user groups across the two bands? Is this an automatic thing handled at the AP side, or will we need to manually split the load?

And the mesh network idea - does that mean that all the attendees can run some some software to join the mesh, or will we just put it on some mac hardware around the venue?

iamkevinv commented 10 years ago

You're absolutely right @eugeneware - it's manual unless you have the firmware facility to tell the 5ghz clients to fall back to 2.4ghz and try again. Some high end AP's do this, but i really have no idea if it exists on these AP's. Some AP's will only show up as 2.4ghz AP's once the 5ghz is full. Which is also pretty useful behaviour. More research required.

However, i was thinking...We could design our own gateway page to say Hi and allow people to monitor which AP's they should join (we keep a central count and give a blatant recommendation to hop onto another SSID or not); then a manual process is handled by attendees. Even without the fancy tech, if they're having trouble joining on one network, they can easily choose the other if we split the networks with independent SSID's. Call it human load balancing?

Yeah the mesh would require some client software to make it easy for people. Provide it before the event, and set up some AP's with a gateway page (yep, again, it's convenient) which hosts a link to the file and allow people to download the client or config. Only required first time. A few clients near some hardware around the venue would allow the mesh to join the regular wireless/wired network. It's all good theory but i have to find a good way to make it happen. But hey, my github page will enjoy some useful code to share for once ;-)

timoxley commented 10 years ago

Had a professional weigh in:

Ok so I read it. And tbh lots of wifi devices in a small space just sucks It's hard no matter what, but honestly it's difficult to get better than the unifi pro APs but you also need someone to set them up properly otherwise it just doesn't work very well. Consider also the low-end Aruba APs which are around $350 each (and I happen to sell) The meshing is quick and automatic but only one radio so no 5ghz, the higher end ones are way better and I know everyone loves apple stuff but its not built for the complexity and scale People hate fucking around with that stuff at cons they just want it to work I'll need to ask you some stuff about the requirements about what needs to talk to what and etc but minimising chatter and segregating the wireless as much as possible helps avoid excessive broadcasts Realistically people will be moving with their phones and sitting with their computers so there are lots of options as to how you break it up.

sidorares commented 10 years ago

What if we just ask people to bring ethernet cables and routers with them and try to put ethernet on each (or each second) table? This will make life those on wifi easier as well

cgiffard commented 10 years ago

Ethernet seems like a pretty good way to avoid wifi congestion. Just tell people with ethernet-connected devices to switch off Wi-Fi.

timoxley commented 10 years ago

Information about the available network:

• The Camp has an ADSL connection (not adsl 2+). Download speeds are approx. 4MB and upload is approx. 0.80MB. • There are wireless networks in the Mess Hut, Staff Hut and most of the Exec Hut. If it makes it easier for them, I can make them all the same name and password so they don’t have to connect to different wireless networks. • There is a wired local network in the Murdoch Room – but no internet or wireless. However, if we put the modem from Caretakers house in the Murdoch Room, the Murdoch Room (and possibly the downstairs area of the Coltman Hut – I haven’t tested this) will have a wireless network with similar upload and download internet access speeds.

timoxley commented 10 years ago

somerscampplan_june2012 copy 2

mipearson commented 10 years ago

Okay. Given that one of the big problems at the last camp was the border getting overloaded with connections (poor widdle consumer grade NAT table), can we set up an interstitial RouterOS proxy again?

Also, any indication of what gear they're using for the wireless? I'm assuming $50 officeworks stuff, but might be surprised.

On Thu, Mar 6, 2014 at 4:05 PM, Tim Oxley notifications@github.com wrote:

[image: somerscampplan_june2012 copy 2]https://f.cloud.github.com/assets/43438/2342079/f55da414-a4ec-11e3-95d2-f5558e0d729d.jpg

Reply to this email directly or view it on GitHubhttps://github.com/campJS/campjs/issues/3#issuecomment-36825546 .

Michael Pearson

timoxley commented 10 years ago

The gear at the venue:

Staff Hut – NETGEAR N300 DGN2200v4
Bursar’s – TP-LINK N750 TL-WDR4300
Kitchen – TP-LINK N600 TL-WDR3600

cc @mipearson

timoxley commented 10 years ago

can we set up an interstitial RouterOS proxy again?

@mipearson what's involved in this?

mipearson commented 10 years ago

I'm not sure on the status of the routeros box we used last time - if it belongs to CampJS and the configuration hasn't been changed, then it should be mostly plug & play. Maybe a little bit of fiddling. I forget.

On Thu, Mar 6, 2014 at 7:23 PM, Tim Oxley notifications@github.com wrote:

can we set up an interstitial RouterOS proxy again?

@mipearson https://github.com/mipearson what's involved in this?

Reply to this email directly or view it on GitHubhttps://github.com/campJS/campjs/issues/3#issuecomment-36834063 .

Michael Pearson

timoxley commented 10 years ago

@mipearson yep, that was one of these and it's sitting, unchanged, in a box at my home.

mipearson commented 10 years ago

Quick google shows me that all the APs are OfficeWorks <$100 items. So -- probably not good enough, certainly worse than the UniFI items used at last camp.

If possible (budget?), I recommend we go with the Airport Express plan. We set them up in addition to the APs they already have. We keep the passwords for the TP-LINK & NETGEARs secret, put everybody on the Airport Expresses, make sure they're not crossing channels.

The Airports will go through the router OS box & be proxied & shut off from apple update / youtube / dropbox. The existing wireless routers will go direct to the modem. Presenters & staff can have access to the TP-LINK / etc.

There are other, better options, but this is the lowest budget one :)

On Thu, Mar 6, 2014 at 7:32 PM, Tim Oxley notifications@github.com wrote:

@mipearson https://github.com/mipearson yep, that was one of thesehttp://routerboard.com/RB2011UAS-2HnD-INand it's sitting, unchanged, in a box at my home.

Reply to this email directly or view it on GitHubhttps://github.com/campJS/campjs/issues/3#issuecomment-36834571 .

Michael Pearson