High level idea questions

[lwr20]

So initially all the Pis are connected in a private LAN connected to a cheap router. Presumably we're expecting the router to have a DHCP server? It may also have a firewall (which might be interesting when the system is connected to the Internet later).

Is the intention to write everything from scratch or piggyback on other projects? It sounds like there are 2 networks going on here - a central registration server (with all the Pis connected in a star) and the conga itself (peer-to-peer communication). There's also talk of being able to kick or ban users. Sounds quite a lot like an IRC chatroom with private messaging on the side? Or maybe an XMPP setup?

There's talk of an audio chat system - remember that a Pi doesn't have a microphone by default - you'd need to add one with a USB soundcard. And then you'll have fun figuring out which soundcard to send the audio out of (the built in one or the USB one).

[Lukasa]

The central registration server has a problem when someone new joins a conga. It needs one of the following things:

• The ability to push updates to conga members to tell them to send their data somewhere new (not HTTP, probably XMPP)
• Conga members to poll it on a regular basis (every 5 seconds, say) to ask if they should send their data somewhere new (HTTP)

1500 Raspberry Pis polling every 5 seconds is non-trivial load: each request would have to be turned around in 200ms or less. Definitely do-able with a standard HTTP web framework and a few easy-ish optimisations, but not trivial.

XMPP looks like the better idea from that perspective: 1500 concurrent connections should be no big deal.

[neilcollins]

Piggybacking is good but it's important that kids can get into the program and modify it in some way. I.e. just using something off the shelf like IRC would bring with it too much complexity I think.

Yes - router might have DHCP but the best option, by far, would be that the Pis start off by figuring out other Pis on the network. Looks like this can be done by looking at MAC addresses.

Firewalls will be a problem - definitely!

Good points on the audio portion - that's probably going to be too ambitious I think. Both in terms of getting it working and because of the microphone problem you flag.

[lwr20]

HTTP usually just works. Anything that uses a port != 80 may run into trouble with school firewalls/admins.

[neilcollins]

Might be able to limit polling from the client by doing something like having the client simply having an upstream contact and a downstream contact and only going to the central server if these are lost.

[lwr20]

The thing about Python is there's usually a library for it. import irclib doesn't sound like it would stop kids from 'programming'.

If a Pi starts up on a network with no DHCP server, it will simply have no IP address. Are you suggesting that they pick a random one and then send traffic to find other Pis on the LAN?

[neilcollins]

I think using a library in that way would be fine – so long as there’s an understandable API.

Good point about not having an IP address. You can definitely start Pis with a hard coded IP address (we did this for a classroom setup for exactly this reason) but that would be pretty unusual so, yes, the normal case would be that there is a DHCP server and the Pis get their IP address dynamically.

On the other point – yes, dynamically finding Pis on the network would be an excellent start. Not sure how well it’d work but doing something like what is suggested here would be great : http://www.raspberrypi.org/phpBB3/viewtopic.php?f=31&t=32460

The idea here is to get things started quickly and build from there. If it’s complicated or unreliable then no problem having the kids have to do something else but having something that just sort of works out of the box but then you need to change the code to make it do more and exciting things is the idea

From: lwr20 [mailto:notifications@github.com] Sent: 20 June 2013 15:10 To: neilcollins/piconga Cc: Neil Collins Subject: Re: [piconga] High level idea questions (#1)

The thing about Python is there's usually a library for it. import irclib doesn't sound like it would stop kids from 'programming'.

If a Pi starts up on a network with no DHCP server, it will simply have no IP address. Are you suggesting that they pick a random one and then send traffic to find other Pis on the LAN?

— Reply to this email directly or view it on GitHubhttps://github.com/neilcollins/piconga/issues/1#issuecomment-19755078.

[peterbrittain]

Security is likely to be the main issue with some of these ideas. I agree with lwr20 that HTTP is probably going to be the protocol that is most likely to get through school/LEA firewalls.

I'm also a bit nervous about the idea of port scanning for other Pis... This may look like potentially malicious activity to some network monitoring tools. Probably OK in all but the most paranoid networks, but worth making the scan optional IMO.

[Lukasa]

Can we not just have the Pis send broadcast packets advertising themselves?

[lwr20]

In much the same way as Dropbox does? That seems sensible (to me at least).

From: Cory Benfield [mailto:notifications@github.com] Sent: 20 June 2013 16:39 To: neilcollins/piconga Cc: Lancelot Robson Subject: Re: [piconga] High level idea questions (#1)

Can we not just have the Pis send broadcast packets advertising themselves?

— Reply to this email directly or view it on GitHubhttps://github.com/neilcollins/piconga/issues/1#issuecomment-19761545.

[ZsigE]

(This is Phil, for those of you who don't recognise my username...)

I think it's worth bearing in mind that however we implement the actual Conga data-sending will have to work both on a local LAN and over the public Internet (as it would be slightly self-defeating to have separate code for the two tasks), which gives us the problem of getting that data through firewalls and NATs. Sending it would be OK, but having a Pi listen for input on a port means that port will need to be forwarded somehow to the outside world.

That's not impossible, but I don't see an immediately obvious way of doing it without getting each school's sysadmin to port-forward whichever port we select, and that would surely limit us to using one Pi per school for the big Conga.

[peterbrittain]

This is why we're currently mulling over use of HTTP. See https://github.com/neilcollins/piconga/issues/2 for some more detailed thoughts on the matter.

[Lukasa]

@ZsigE If someone wants to play about with a new network protocol, most home routers these days come with UPnP. This allows programmatic port forwarding. It is possible (don't hold me to this, I don't know UPnP very well) that we could implement some UPnP functionality to automatically port forward when connecting to a non-LAN conga.

[peterbrittain]

Feels pretty advanced for a simple client to me... Possibly too advanced for the target audience?

[ZsigE]

From reading over that, I'm not entirely sure we've even got the same network architecture in mind. The way I understood it, we're doing something like this:

LAN mode:

• No central server
• Pis find each other through some as-yet undefined discovery mechanism
• Pis then send messages directly P2P

Internet mode:

• Central control server tracks which Pis are part of the Conga
• Server tells each Pi in some way who it's sending to next
• Pis send data directly to each other

Some of Pete's suggestions seemed to imply that his mental model of Internet mode was more like this:

• Central control server does practically everything
• Pis send messages to the server and poll it for the next message
• Pis only ever get information via HTTP POST and therefore don't need forwarded ports

I may, of course, be entirely wrong about that! Any comments?

[Lukasa]

@ZsigE Yeah, I was about to write that. I don't really want the central server involved in each leg. The load will be unpleasant.

[neilcollins]

Do you think it's worth doing this in two stages:

• Stage 1 - do the easy http only (Web server in every hop)
• Stage 2 - try to go direct Pi-to-Pr.

Seems to me like we'd probably need to have the ability to fall back on "Server-in-every-leg" anyway if we failed to get through the firewall in any other way so Stage 1 wouldn't be wasted effort.

[peterbrittain]

Yup - that was what I was thinking. It's going to be hard to avoid involving the server given some of the security/audit requirements in there. If we're not careful we'll end up with 2 completely different applications.

[ZsigE]

Hmm. Not wild about the idea of doing the whole thing client-server, mainly from a conceptual standpoint (not to mention @Lukasa's load concerns) - implementing something that is functionally identical to one system updating a Wiki page and all the others bouncing it back and forth off the server in turn stretches the definition of "sending a message round a loop" somewhat. That said, having it as a fallback for cases where the next Pi in the loop is unreachable isn't a bad plan.