mozilla-mobile / android-components

⚠️ This project moved to a new repository. It is now developed and maintained at: https://github.com/mozilla-mobile/firefox-android
https://github.com/mozilla-mobile/firefox-android
Mozilla Public License 2.0
2.02k stars 474 forks source link

New component: Nearby #1773

Closed jonalmeida closed 3 years ago

jonalmeida commented 5 years ago

We want to be able to find devices in the area that we may not already be paired to so we can share sessions.

This component would mostly be for discovering/connecting to other devices. The data itself can be figured out later (separately?).

┆Issue is synchronized with this Jira Task

jonalmeida commented 5 years ago

EDIT: Renamed 'Share Manager' to 'Nearby Manager' to avoid immediate confusion with android sharing.

There are two major parts for local sharing controlled by the main entry point, Nearby Manager:

  1. Discoverability
  2. Transfer Protocol

The 'Nearby Manager' co-ordinates the Discovery Manager and the Tunnel Manager and allows the app to be notified of connections established/failed and turning services on/off.

```mermaid graph TD; A[Nearby Manager] --> B[Discovery Manager] A --> C[Tunnel Manager] B --> D[Wifi] B --> E[Bluetooth] C --> F[Wifi] C --> G[Bluetooth] F --> H[Server] F --> I[Client] G --> J[Server] G --> K[Client] ```
screen shot 2019-01-21 at 11 37 51 am

Discoverability

Discovery Manager

Interface

interface DiscoveryService {
  fun scanningStarted()
  fun scanningStopped()
  fun deviceFound(Service device)
  fun deviceLost(Service device)
  val bearerType: ServiecType
  val deviceName: String /* Update discovable name of device */
}

sealed class Service(val deviceType: DeviceType) {
  data WifiDevice(val uuid: String, val host: String, val port: Int)
  data BluetoothDevice(val uuid: String, val macAddress: String)
}

enum class ServiceType {
  Wifi,
  Bluetooth,
  Unknown
}

enum class DeviceType {
  Mobile,
  Desktop,
  Unknown
}

class DiscoveryManager(discoveryBearers: List<DiscoveryBearer>) {
  fun init(enableWifi: Boolean, enableBluetooth: Boolean)
  fun enableService(/* TBD */)
  fun disableService(/* TBD */)
  fun deviceFound(device: Bearer, type: BearerType)
  fun deviceLost(device: Bearer, type: BearerType)
}

Transfer Protocol

Tunnel Manager

interface TunnelServer {
  fun start() 
  fun stop()
  fun tunnelOpened(tunnel: Tunnel)
  fun tunnelClosed(tunnel: Tunnel)
  fun broadcast(data: ByteArray) = Unit // not required
  fun broadcast(data: String) = Unit // not required
}

interface Tunnel {
  fun close()
  fun send(data: ByteArray)
  fun send(data: String)
  fun isOpen()
}

abstract class TunnelClient implements Tunnel {
  abstract fun onConnect(tunnel: Tunnel)
  abstract fun onClose(tunnel: Tunnel)
  abstract fun onMessage(tunnel: Tunnel, data: ByteArray)
  abstract fun onMessage(tunnel: Tunnel, data: String)
}

class TunnelManager(val servers: List<TunnelServer>) {
  fun init() = servers.forEach{ it.start() }
}

Nearby Manager

Connects the DiscoveryManager and TunnelManager together.

class NearbyManager(
  val discoveryManager: DiscoveryManager, 
  val tunnelManager: TunnelManager
) {
  val priorityList: List<BearerType> = listOf(Wifi, Bluetooth)
  val deviceName
    get() = field.value
    set(value) { /* notify discovery service */ }

  /* various notifiers for the app; tbd */

  discoveryManager.onServiceSelected ( service -> service.connect() )
  tunnelManager.onConnected { tunnel -> notifyObservers { connected(Tunnel) } }
}

User Flows

  1. User A has Wi-Fi; app has enabled Wi-Fi service:
    1. Wifi tunnel server started.
    2. User A discovers User B.
    3. User B selected and WebSocket client connection attempted and established (app notified to send data)
  2. User A has Bluetooth; app has enabled Bluetooth service:
    1. Bluetooth tunnel server started.
    2. User A discovers User B.
    3. User B selected and Bluetooth client connection attempted and established (app notified to send data)
  3. User A has Wifi and Bluetooth enabled, app has enabled both services:
    1. User finds and selects User B on Wi-Fi and Bluetooth.
    2. Nearby Manager considers the priority list to choose which service to use.
    3. Connects to [insert higher priority device].
  4. User fails to connect to a service:
    1. User A finds and selects User B.
    2. A client connection attempted and failed (app notified of failure).
    3. Note: Do we try again? If there are multiple services available, do we try the second one?
jonalmeida commented 5 years ago

Other things to consider:

pocmo commented 5 years ago

Thank you, @jonalmeida.That's a great detailed issue. :)

My assumption from the mocks is that a "neary" feature will be a killer feature once it is supported on all devices (including desktop and iOS). Did you research this a bit? Does iOS have some capabilities built-in (like Google's nearby API?)? Getting the desktop world onboard may be harder - did you reach out to any teams yet?

jonalmeida commented 5 years ago

Does iOS have some capabilities built-in (like Google's nearby API?)?

iOS supports mDNS as well so we'd be able to use that for Wi-Fi i.e. this could be an iOS component that uses the same mDNS service type to discover over Wi-Fi, and WebSockets to create a server/client connection.

For Bluetooth, I assume there is a similar BLE API, but I haven't looked into it yet.

Getting the desktop world onboard may be harder - did you reach out to any teams yet?

I haven't reached out to the FlyWeb folks yet, but I'll make it a point to do so and see what their thoughts are.

That's also a point I forgot to make: this implementation would not be platform-specific, we're using open standards so everything here is just implementation details that we'd have to share across other platforms. Scoping this to work for all platforms with just Wi-Fi is probably a more achievable task.

jonalmeida commented 5 years ago

I spoke with @justindarc about the FlyWeb implementation and got some useful insights out of it:

Drawbacks:

Misc Notes:

One of the things that helped us get past this [securing a connection] problem though was a notion of trust after the first connect. So, if I connected to device XYZ once, it could be trusted to connect subsequent times thereafter but, that first connection to device XYZ could be compromised.

This is a useful implementation idea if we decide to have a concept of trusted devices. It also brings in more questions about how do we associate trusted devices with synced devices if there are overlaps that requires discussion with FxA.

justindarc commented 5 years ago

One point of clarification to the above -- the mDNS interface is still in the mozilla-central tree, but the "web server" part of FlyWeb has been removed AFAIK. So, unfortunately, that means that the browser-to-browser demo video linked above will no longer run in Firefox Nightly. However, like I mentioned, all the mDNS/DNS-SD (service discovery) APIs should still be there as I believe they were also being used for other things besides FlyWeb (such as the Presentation API and some of the Dweb work).

Also, I believe the formal name for the trust model described above is Trust On First Use (TOFU). This security model is widely used today for lots of things including SSH (i.e. when you get prompted to trust the key of the server on the first connect).

jonalmeida commented 5 years ago

How do we avoid spoofing attacks is another big security question that I'm unsure off that probably needs a security review to get solutions on.

Focus on getting the functionality working first will probably be better.

jonalmeida commented 5 years ago

Random thoughts: this would make sense to have this made into a rust component similar to how application services are doing it - bindings for native applications, single logic source.

With the success of concept-fetch and the GeckoViewFetchClient we could do something similar here as well. It would mean that browsers with gv or supported gv multicast could use that implementation or fallback to the Android native/Google Nearby™️ implementation.

Downsides would be longer development and integration time. I'm not sure how much we're willing to invest on this.

pocmo commented 5 years ago

@jonalmeida I'll remove the fenix label here. This seems to be not in scope for MVP. :)

pocmo commented 3 years ago

Closing for now. We just removed one nearby component. Let's open a new issue once we need it again. :)