Plan and fit the excellent networking in the new space

fawkesley commented 6 years ago

For those of us that spend most of our time working remotely with others, the quality of our network connection is a high priority for video calling and collaboration.

And an excellent tech community deserves an excellent internet connection!

It was clear from pub discussion that I have some strong opinions on what that means!

Here are my suggestions:

Encourage wired internet

Naturally some devices need to connect via WiFi, but the performance of WiFi degrades exponentially with the number of clients. So it's better for everyone if we encourage wired connections for those devices which support it.

Practically, we should:

aim to get ethernet to every desk
help people understand why it's good for everyone to use that
recommend a USB ethernet dongle thing for fat-free laptop users

Have a good uplink & download speed

Of course, we need a good internet speed both ways, and we should be sure we're getting what we think we're getting.

Not sure what performance we actually need, but for reference a single device doing HD video will use ~3.5Mbps.

Have (at least 1) static IPv4 address

We're going to be doing our own NAT (sadly) and we shouldn't have any more NAT boxes upstream of us. This means we can manage our own port forwarding and run servers and stuff inside DoES.

Also, NAT is evil, so...

Have IPv6

The UK is embarrassingly behind most of the world on IPv6.

Since DoES is clearly the most interesting tech scene in the city which will apparently soon have world-class connectivity, we should be blazing the trail here. (I've had IPv6 at home for a few years now...)

No captive portals, ever!

Captive portals are those awful things which pop up and break your internet connection while you wait for a pointless 'Connect now' button to appear.

They complicate getting things online - particularly internet-of-things things - and they serve absolutely no purpose.

We can measure and monitor performance

From my experience it's actually really complicated diagnosing why a video call doesn't work well. It's really helpful if you've got some data to look back on.

I'm happy to volunteer on this one - we should be a bit pre-emptive about measuring and monitoring our connection and our WiFi clients, channels and throughput so that when we do have problems we can identify whether it's the router, too many clients, unreliable provider, whatever.

We could try out a city-wide network name

This is bit off the wall and it's harking back to the MerseyNet idea from a few years ago.

Wouldn't it be lovely if we and other places in the city used the same network name (SSID), so you could just roam freely between places?

@amcewen's suggestion is still my fave: network name: ourwifi password ourwifi

Please pitch in and let's work out what we want!

zarino commented 6 years ago

@ajlennon I’d suggest moving that to a new issue ticket, because it doesn’t have anything to do with planning excellent internet.

ajlennon commented 6 years ago

Is there a "planning excellent mains power" ticket or something similar? I don't want to start opening up random orphaned issues. In fact - scratch that - it does relate to planning the internet as I say in point 2

zarino commented 6 years ago

I’m one of those people who has no problems creating tiny, single-topic tickets: https://github.com/DoESLiverpool/somebody-should/issues/686

ajlennon commented 6 years ago

CABLE ROUTING as of 12th March

@magman2112 has annotated on @johnmckerrell 's original network diagram where he thinks the routing needs to go. A lot of this is near where the electrician has already lifted boards and drilled through joists.

This is on the basis we want to lift as few boards as possible as the floor is going to be left uncovered and we want it to look as good as possible and creak as little as possible
@magman2112 also suggests we screw down the lifted floorboards to ensure they are secured and easily accessible in future.
To minimise radiated interference we need to ensure the network cabling is not too close to the mains cabling where it parallels the cabling.
The routes also try to minimise the amount of joists we have to drill through
The two different routes into the patch panel from RHS and CENTRE/LHS are taken to avoid introducing structural weaknesses.
We've introduced numbers to show the number of planned network points.
This may need to be revised and we may need more network and power points in the quiet space. (@amcewen ?)
The co-working space / event space has been planned out but not yet annotated (@magman2112 if you can get me a snapshot of the updated diagram I can go over in red here?)

cable routing

The panel panel is in the space presumably below where @mattwilsondotuk is bringing in the Internet. The space will likely not be easily accessible. On the right we'll have a sink or sinks so there will likely be no access from there. On the left we'll have a bench/nook area so the thinking is there'll be an access panel there to enable people to get into the space which will be used for some storage. The patch panel will be higher up on the wall of the space to enable access, getting the cabling in and so forth
Cable runs are looking to be about 20-25m
@magman2112 has been totting up the bits and pieces that will need to be purchased, probably from ScrewFix, e.g. plastic and metal housings and so forth

ajlennon commented 6 years ago

OK - then can you ensure that ticket #686 is acted upon - thanks @zarino

ajlennon commented 6 years ago

Does anybody know where we are with @mattwilsondotuk 's external Internet coming in?

Could we agree a tentative date for when we should be able to start connecting our network box up and moving packets around to the outside world?

ajlennon commented 6 years ago

I've followed your lead @zarino and broken out #687

johnmckerrell commented 6 years ago

The external internet feed depends on power being provided to their hardware which they're waiting for the landlord to provide, there will then be a period of potentially weeks. Talking to @mattwilsondotuk this morning it certainly looks like this could be a blocker to moving in on the 25th but it really depends on when he can get power to his hardware.

ajlennon commented 6 years ago

@johnmckerrell yikes. When will we have clarity on the date? How does failing to leave Gostins on w/e of 25th affect tenancy agreements and so forth? How difficult is the power issue? i.e. is it just plugging something in and switching it on or is it digging up a road to lay a cable? Is there any other way to support Internet in the short term as a plan B?

magman2112 commented 6 years ago

Thanks to Simon and JR Peterson for their excellent help today with the network cabling. We just managed to meet our primary target of fully installing the 12 cables that releases the builder to board the remaining walls in the space.

A lot of the work today was prep work. Simon wore out one drill bit, due to the number of holes we needed to drill, plus we needed to lift quite a few floor boards to enable some runs and holes. Some drilling is still needed tomorrow, but we have now agreed to limit noise disturbance to other tenants between 10am and 3pm, I am planning to go into Tapestry early tomorrow to complete any noisy work that may limit the cables we can install tomorrow. Then we can concentrate on pulling in as many cables as possible.

Quite a few (20) of the remaining cable runs are over 22 metres in length, with the cable on hand now being shorter than this (we have already used all the longer lengths we could find). After discussing this with Jackie, I recommended that I pick up a box of Cat 6 cable tomorrow that will enable us to complete this part of the cabling. I expected this to cost circa £100 inc VAT for a 305m (1000ft) box from Screwfix. I have just figured out though that we actually need approx 450m of cable for these 20 runs. From Screwfix, a 100m reel is £29.99, so 5 reels of 100m is £10 cheaper than 1 of 305m plus 2 of 100m, so this is actually better value for more cable. I will check this with John before going ahead though.

If anyone knows of any other long lengths of cable, please let me know before 10am, when I plan to go and purchase the new cable. New cabling is also about twice as fast to install compared to re-using cable.

johnmckerrell commented 6 years ago

@ajlennon we're ok in Gostins for longer if we need

johnmckerrell commented 6 years ago

FYI we've had a complaint about the noise of the fit-out from the ground floor tenants. They're being very nice about it but if you're working in the building please try to limit the noise you make between 10-3 Monday-Thursday and Saturday 9:30 - 4:30. Thanks!

johnmckerrell commented 6 years ago

We've had an offer of 3 free Unifi AP-AC-LR units from @skos-ninja which is great! Current thinking is that we use these for the coworking room, events room and the big main room and see how we get on in the meeting room (which will be well equipped for hard-wired networking anyway). I don't think we've finalised the position for the APs though so will still need to do that, and then wire in cables.

magman2112 commented 6 years ago

Good progress was made on the network wiring today, with 21 cables being installed. I had excellent help from Chris Fairey today (apologies if I spelt Chris’s surname wrong as I got this via a telephone call with Jackie). I am sure we will both be feeling the workout we had today for a day or two.

If you are wondering about the odd one cable in the 21, this was a high level connection in the meeting room for a possible WLAN device.

With a few recent additions and assuming 4 points at most locations, the total cable count currently stands at 93 for the full installation. So far, 33 of these have been run. I have drawn up a clearer plan of the cable runs on paper today, which I plan to convert to an electronic format tomorrow hopefully.

I have other activities outside of DoES for the next few days, but plan to get back to this wiring on Saturday.

magman2112 commented 6 years ago

One thing I have mentioned to both Jackie and John is the socket boxes on the brick walls.

The electrician has installed metal back boxes like these

http://cpc.farnell.com/mk/k2214alm/box-2-gang-with-knockouts/dp/PL14730?st=Metalback%20boxes

From an aesthetic point of view, we should ideally use the same type of back boxes, along with black conduit up from the floor, but obviously the costs are mounting up for this installation. I suspect though that we may be able to get an equivalent back box cheaper than the one I listed above and I now I have found a LAN front plate to match the back box at a reasonable price.

https://www.sparksdirect.co.uk/BG-Nexus-4-Module-Rectangular-Metal-Clad-Front-Plate-100-50mm-Euro-Plate#.Wqj5t1qnyhB

These faceplates take Euro size modules, which I suspect the LAN sockets in the faceplates we have already got are the same size (and hopefully any more faceplates we recover from Gostins). I managed to find the same faceplate on eBay for £3.75 including postage, so I bought one to test this assumption.

Overall, we currently need 10 of these back boxes and faceplates for the network installation.

JackiePease commented 6 years ago

At close of play Sunday 18 March, all wiring in main room installed. Next is the installation of socket boxes and terminations, and cabling of coworking and event space.

We need to order (@magman2112 will order on Monday if no objections):

10 2 gang metal back boxes £3 each £30 2 1 gang metal back boxes £2.50 each £5 10 4 way Euro faceplates (metal) £3.75 each £37.50 2 2 way Euro faceplates (metal) £3 each £6 3 reels 100m cat 6E £30 each £90 93 cat 6 sockets c£2 each £186

Estimated additional expenditure £354

We still need to decide on the cabinet and how it’s going to be installed. @magman2112 suggests freestanding to allow easier access to power etc.

mattwilsondotuk commented 6 years ago

Just a reminder about the Internet, now we have power on the site as of yesterday (yipee!) we can now start to build the exchange and various other systems. For Does, we need a network diagram, details of the firewall you are using (if any) and how you intend to terminate the connection into the office.

I was at the site on Sat and ran in the Internet feed cable with one of the volunteers who made a good job of the cabling, taking it into the rack area ready.

Positions of the AP's and wiring for them has been confirmed with @johnmckerrell

It would be good if your office firewall/router supported VLAN's so we can offer Layer 2 services up at Does, i.e direct connections to amazon, azure, IOT LAN's etc..

skos-ninja commented 6 years ago

@mattwilsondotuk after speaking with @johnmckerrell a fair bit I think we were intending to get a Unifi Security Gateway to allow us to do QoS and we should be able to setup multiple VLAN's fairly easily with this setup.

I'm currently working on contacting Ubiquiti to see if they can do anything to help sponsor this equipment and should hear back soon from them.

johnmckerrell commented 6 years ago

Thanks Jake. We don't currently have a network diagram. Networking's not my thing so I'd struggle to prepare one. We generally just block everything incoming apart from the webcams and the doorbots. I was also going to confirm what hardware we already have that we can continue to use. We definitely need some PoE gear as we have none currently. This issue is really holding out for a hero, people are giving great feedback but there's no one person leading this one.

johnmckerrell commented 6 years ago

Just checked and we have a HP Procurve E2510-24 Switch (J9019B) currently. It's a managed switch with gigabit ethernet. All of the other equipment we currently have is either not gigabit, noisy or desktop (i.e. 4-12 port type thing).

@mattwilsondotuk I thought I'd asked via email but don't now see it. Will the router you provide offer firewall capabilities, port forwarding, DHCP?

mattwilsondotuk commented 6 years ago

@johnmckerrell yes it will but only as part of a managed service, i.e you won't be able to access it from within Does as its a customer CPE so customers don't have access to it as standard. Normally customers connect their firewall to it.

JackiePease commented 6 years ago

Volunteer who helped @mattwilsondotuk on Sunday was JR (@thingomy)

magman2112 commented 6 years ago

@johnmckerrell That switch should be plenty for the coworkers for the move, but ideally we should keep an eye out for 1 or 2 similar switches in the near future to increase the wired coverage at Tapestry. We may have to do a bit more patching in the short term, as people start to figure out how they are going to use the new space.

I suspect we also still need a PoE switch in the short term, to power the Wi-fi routers that have been donated.

skos-ninja commented 6 years ago

After discussing with @johnmckerrell we think for the moment getting 1 router and an 8/16 port poe switch will work best.

At the moment we are after using a Raspberry Pi as a Unifi Controller and then getting a Unifi Security Gateway and one of their Unifi PoE Switches to compliment the 3 Unifi AP's we already have to allow ease of management and overview of the network. We suspect the main usage of the internet will still be via WiFi so adding the existing HP Procurve E2510-24 should give us enough ports for now with options to upgrade in the future.

I have offered to @johnmckerrell to get a US-16-150W to help with this so DoES should only need to purchase a UniFi Security Gateway and find a spare Pi.

xj25vm commented 6 years ago

In case it helps (just ignore me if it doesn't):

I have a bunch of TP-Link gigabit 16 port PoE switches installed and running wifi points and cctv cameras for 5-6 years now - and they seem pretty solid to me - and cheaper than other brands (although they have gone up from £90 last year to nearly £150 now!). I have just received today in the post a replacement of the above model in warranty - after it started to go wobbly after 2.5 years of service. They have a lifetime warranty - and it's good that TP-Link honours it (after some wrangling over email with their returns department, in all fairness):

https://www.amazon.co.uk/TP-Link-TL-SG1016PE-16-Port-Gigabit-Switch/dp/B071ZGZ56M/

If PoE is not needed, a TP-Link unmanaged gigabit switch of 24 ports is only £67 at the moment. I run sites of up to 100 network points on unmanaged switches, and I think that, unless the features of managed switches are truly needed, they add a significant whack to the budget - and to the complexity of the setup. Many sites don't need managed switches, and don't really have the skills to use those feature, IMHO.
For a firewall and QoS, provided that the skills are available, a Linux box running on commodity hardware is hard to beat. The configurability and flexibility is almost endless, it can be replaced in a pinch for almost no cost if it goes bang and you are not tied to the standards or compatibility of a particular manufacturer.

OK - I'll get my coat now :-)

skos-ninja commented 6 years ago

My main thought with the router is that we need something that is going to be reliable and has customer support if we need it as internet is pretty much a requirement at all times at DoES so we can't be having something that goes bang ;)

xj25vm commented 6 years ago

It is healthy to assume that everything might go bang at one time or another, and be ready for it :-)

Of course, it is best to go with whichever option you are most comfortable with.

mattwilsondotuk commented 6 years ago

Just confirming that we've made 100MB upload and download live early on this week on the DoES floor and on Monday 26th March the 500MB service will start after making quite a lot of progress this week on it.

For this we've had to forfeit static ip's, VLAN's the IX Liverpool exchange connection IOT network and Layer 2 services that will come on-line later in April now. but at least DoES has the "full fat" connectivity from Monday now :)

skos-ninja commented 6 years ago

A thing of note is the HP ProCurve 2510-24 seems to only support 1000Mbps on 2 ports so I'm not sure how useful this switch is.

skos-ninja commented 6 years ago

Just to note it down here.

The three unifi ap's that had arrived have been setup and are ready to be plugged in via PoE injectors for now. Also we now have a raspberry pi setup running the UniFi controller software on https://192.168.1.2:8443 that is in the cabinet, do note that this pi is set to a static ip of 192.168.1.2 which the ap's are expecting for the controller software. This pi can be moved and plugged in else where if need be and is not required for the ap's to be working.

@johnmckerrell has ordered the other pieces of equipment and we hope it should arrive early this week. I'm happy to come by after work one day once it has arrived to help set this up too.

The login details for the UniFi software and the pi are currently on my laptop, if an organizer would like the details please contact me and I can forward them to you.

Thingomy commented 6 years ago

Current state of play is, wifi is up, and the 4 network points in event space are live, but most of it is still on 100Mb due to issues with the switch. All of this is somewhat temporary as things are still a bit up in the air.

Wifi coverage seems excelent from a brief walk around.

skos-ninja commented 6 years ago

One thing I have noticed is that the router @mattwilsondotuk has provided is doing DHCP but is locked to balticbroadband which means we can't do static ips right now as we are unable to reserve the addresses. Is there a way around this?

Thingomy commented 6 years ago

Last night all of the remaining network lines were put in bundles and fed up into the networking cupboard ready to be terminated by @magman2112 . Room side termination was done on the pillar in the middle of the main space.

magman2112 commented 6 years ago

The current state of play for the network wiring is that all of the accessible wall plates and sockets have now been installed and terminated (thanks to JR for his great efforts on this). The remaining terminations are:

two floor boxes which are currently buried under flooring or furniture
two wall mounted boxes between the front windows
One socket high on the meeting room wall
the connections in the quiet room, as the termination point has not been decided upon yet.

We need 4 more wall plates to complete this fit out, 1 metal clad and 3 plastic. I will order these later today.

In the wiring closet, 16 of the cables have been terminated so far. This covers all of the wi-fi and doorbot sockets and all of the sockets in the Events room. There is currently only one LAN connection patched through to the Events room, but more can be patched if required for Monday’s event. @johnmckerrell has ordered the patch panel we need to complete the termination in the wiring closet, this should be with us in the next few days, then we can look to complete the termination for all of the cables, ideally within the next week.

Wi-Fi is up and appears To be working well. I suspect some tuning is required though, as two of the devices appear to be using the same channel and @mattwilsondotuk has suggested that the nodes be set to a lower power mode.

There have been some discussions about LAN switches recently, we may need to discuss this further, especially deciding how many LAN ports that actually need to be patched in the space, along with whether VLAN’s are needed to separate co-workers from other potentially disruptive LAN traffic from the maker space, etc.

The only other aspect to completing this wiring is documentation. As much as possible, we are aiming to make the patching self documenting, by using logical naming and labelling standards. We will still need to generate some documentation, ideally electronically (especially for wiring runs, etc.). I asked @amcewen about drawing files for The Tapestry, as these would seem the most logical starting point for a wiring diagram. I will also build a spreadsheet for all of the wiring points to allow recording of usage as required.

amcewen commented 6 years ago

I've just dug out the SVG that I was talking about @magman2112, only to realise that it was before all of our fit-out, and so doesn't have any of the internal walls in it.

@zarino, maybe you could put the one you prepped somewhere-safe?

zarino commented 6 years ago

@amcewen your wish is my command! https://github.com/DoESLiverpool/somewhere-safe/blob/master/does-tapestry-floor-plan.pdf

johnmckerrell commented 6 years ago

@magman2112 FYI the patch panel has arrived.

skos-ninja commented 6 years ago

At the end of play today I have now installed our Unifi Security Gateway.

The network is now setup to run on the 10.0.0.0/8 subnet. All of the networking gear is given static ip's in 10.0.0.0/24. Currently anything given DHCP address is given an ip in 10.0.29.0/24.

Again I still have the details for how to login which an organiser will need to grab off of me.

If a local static IP is needed then someone with access to the unifi dashboard will be able to give them one.

I had a chat with @mattwilsondotuk about the network tonight in which we both agreed that DoES being given a static IPv4 and possibly a range for IPv6 would be our best bet as currently the network is getting triple NAT'd which isn't great. Matt has said it will come in the future and he will email DoES when he's ready to switch to a static IP.

Once we have a range for IPv6 we should allow people to request an IP with ports forwarded if they want for any little IoT device that they would like to access from the external network.

ajlennon commented 6 years ago

Fantastic news! Very keen that before any ports might be forwarded internally we have a collective chat about VLANs, security and such?

skos-ninja commented 6 years ago

Yeah a security talk will need to happen before any port forwarding is done and even then it might be worth looking at separating the network more as currently everything is just sat on the same VLAN with no security around it which doesn't seem ideal.

Another thing that was mentioned by Julian yesterday was if we could give people a page in which they could find all devices on the network and their IP. I'm not sure how we would want to do this at the moment but seems like a good idea for all the Pi's!

ajlennon commented 6 years ago

"Julian yesterday was if we could give people a page in which they could find all devices on the network and their IP. I'm not sure how we would want to do this at the moment but seems like a good idea for all the Pi's!"

Absolutely. If we could do something around Bonjour/Zeroconf discovery and make that easily available to people I think that would be neat.

skos-ninja commented 6 years ago

Well with the joys of UniFi is that their dashboard already has this information available so we can just query this API to get most of the information required.

ajlennon commented 6 years ago

Awesome!

skos-ninja commented 6 years ago

I also wonder if it's worth designing a mount for our router, pi and baltic broadband's router to fit in a 1U as currently they aren't secured very well as they're just sat on top of one of the switches and it would tidy up the cabinet a bit.

ajlennon commented 6 years ago

This is on my todo list as well @skos-ninja. If you want to get together to chat about how we mount RPis in cabinets I would be v. keen to do so. I have a plan for some automated testing....

ajlennon commented 6 years ago

This looks interesting. Quite simple to print

https://www.thingiverse.com/thing:177705

Fits on a 19in tray apparently?

http://www.ebay.co.uk/bhp/19-inch-rack-shelf

skos-ninja commented 6 years ago

Yeah that does look good.

I'm around Thursday evening to help @johnmckerrell with some doorbot setup and want to sort out #616 so you should see me hanging around the cupboard then.

johnmckerrell commented 6 years ago

This cupboard is an important bit of DoES infrastructure and shouldn't be considered fair game by all and sundry to start piling hardware into without having a conversation first about what you want to put in here. (Obviously @skos-ninja designing something to better host the various infrastructure hardware is a great idea)

ajlennon commented 6 years ago

For myself, I have no intention of putting anything in the cupboard. My kit will be going elsewhere.

However commonality of mounting between @skos-ninja and what I am doing is helpful to us both.

skos-ninja commented 6 years ago

I have got @johnmckerrell to order http://cpc.farnell.com/pulse/rksu-1u/rack-shelf-universal-1u/dp/DP32725 for us to mount the networking gear on. We hope it will arrive by tomorrow for Maker night however they make no guarantees ☹️

DoESLiverpool / somebody-should