Miracast or streaming

[mjmeans]

So it's already possible on most laptops to plug in an external projector to the laptop's HDMI port and extend the desktop to it. A second copy of RPTools would be shown on this second display as a player copy. However, for table top gaming, the projector should be above the table facing down onto it, making the cabling awkward at best. WiFi would be a better choice.

So the projector would need to support WiDi, WiFi Direct, MiraCast, AirCast, or PlayTo media server technology. The old Western Digital TV Live Streaming Media Player is a good example of this.

but a lower cost solution is possible. I have already built a simple app for the Raspberry Pi 2 and Windows 10 IoT Core that receives and displays a PlayTo cast media stream wirelessly. The only thing is that RPTools is unable to create a media stream of its window to cast to the receiver.

Alternatively to PlayTo media streaming, the other technologies such as the open standard MiraCast is supported by Windows 8.1 or later. If RPTools is run on Windows 8+ then its possible to extend the desktop to a WiFi projector, TV or monitor. There are a number of Miracast receivers on the market ranging from $15 to $50 that can turn a non-WiFi capable projector, TV or monitor into a casting receiver. I have not tried any of them. The Microsoft Wireless Display Adapter seems to be the best rated for use with Windows. But it also seems that these are for very high bandwidth full HDMI movie quality. The bandwidth usage of these devices is very high. And the host computer requires Wireless N and in order to use an older laptop or computer host you would have to buy a new WiFi dongle such as TL-WN725N that supports WiFi Direct (a prerequisite for Miracast). While they are not expensive, it's yet another thing to plug in to get it to work.

If RPTools were able to capture and cast a media stream of it's own window to a PlayTo media player, this would be great! This would work on computers without WiFi Direct capability (older computers, slower WiFi). The receiver could be a cheap Raspberry Pi 2 & Windows 10 IoT (and possibly even cheaper RPi 0 with Linux).

[mjmeans]

Another possible advantage to using PlayTo streaming is that it may be possible to set parameters such as table image size and have MapTool automatically scale to 1 inch squares.

Oh, another possibility, create and cast a player map without opening a new RPTools instance.

[mjmeans]

So a bit of systems analysis here: It would require a UPnP AV/DLNA streaming. It looks like this project here is an open source Java API that handles the UPnP part for discovery and control of a Smart TV (or projector) to command it to play the stream. Other needs would be to have a way to create a hidden virtual window (of arbitrary size) to contain the maptool player view; then capture that view to a video stream; setup up a streaming video server and pass the IP/URL to the UPnP library to send to the Smart TV or projector.

I don't know Java programming, but I've been programming professional (in several other languages) since the early 1980's.

[Azhrei]

So... You can run a VNC server on your Windows/Mac/Linux host, run MapTool inside of it, mirror that VNC server frame buffer to an external device, then minimize the VNC server window. The server will continue to update in the background even though you won't be able to see it, but that means the projector will still update.

I'm not exactly sure what this solves other than the hassle of running an HDMI cable to a projector. Given how an Apple TV can be connected to a wireless network, and then to the projector using an HDMI cable, it accomplishes the same thing -- assuming you have an AirPlay-compatible device. My Mac laptop, for example, can project an extended desktop to an AirPlay device such as an Apple TV using built-in software. If a Google ChromeCast can do this with a web browser and a TV, is there nothing similar that can do the same thing for Windows? (ChromeCast wouldn't work for MapTool, since MapTool doesn't run inside a browser window and AFAIK ChromeCast only works with a browser. But surely there's something?)

Also, you can get a device that streams video wirelessly (at 1080i, IIRC) for about $150US. It bridges the HDMI video signal (so laptop -> HDMI -> blackbox, then wirelessly to blackbox_receiver -> HDMI -> projector). I investigated getting one of these when I installed a projector in my game room. I think I found one at ProjectorPeople.com but I don't remember as I ended up hiring an electrician to crawl through my ceiling and run a proper HDMI cable (it was only a few dollars more and offered a lot of other benefits). It might've been the Iogear model; be careful, as most are 5GHz line-of-sight and walking between the transmitter and receiver can affect picture quality, so placement is critical.

I doubt MapTool will be generating a video stream any time soon, but there is a plan to run a minihttp server so that a browser could connect to it and see map changes. If that browser window could be ChromeCasted to a projector, that would solve your problem, right? (I'm not working on the minihttp service, but I do remember seeing discussion about it from Craig.)

[mjmeans]

Wireless AirCast, ChromeCast or MiraCast devices will not work with older projectors, even HD projectors because they don't implement HDCP copy protection.

Problems with running VNC Server on Windows in a way that you suggest is that it doesn't creates a virtual desktop in a way that you can then open up other software inside of it. At least not in any open source version of VNC Server that I have found. There are commercial VNC implementations that can do as you suggest or even stream just a single application window. But that doesn't solve another problem that would occur... VNC server should never be run without a password. And a dumb remote projector isn't going to have any way to access that. Using an RPi that is also remotely installed up at the projector to control it might be able to be set up to automatically run a script to continuously try to open and log in to the VNC server, but updating that script if the IP address of the PC or the WiFi settings change in any way. The projector is 8 feet above the floor, it's really inaccessible to do anything other than use the remote control to turn it on.

There are only three ways I think this can be done on a single PC with MapTool the way it is:

1. Connect a second monitor and extend the Windows desktop to it. Run the GM MapTool on the main desktop and the Player GM on the extended desktop. Then run VNC Server to mirror that second desktop to the projector. The disadvantages of this is of course the need for a second display. Windows will not create a virtual second display, it needs something actually attached, or it needs a HDCP compliant project-to target. Most older projectors, even older HD projectors can't do it.

2. Log in to a PC with two user accounts simultaneously, open VNC server and Map Tool as player in one account and stream that desktop, then using user switching change to another user desktop to run the MapTool as GM. The disadvantage of this is way too much system resource overhead and it won't work on older slower PC's.

3. I have tested this and it works: A Raspberry Pi 1 Model B with a USB WiFi dongle running the latest Raspbian. It has built-in VNC server and Java 8 with hard float and Java JIT compiler. Then run VNC player to log in to the RPI remotely and run MapTool directly on the RPi. It works, but not optimal because the RPi 1 Model B has only 512MB of RAM and 128MB is used for the GPU to be able to run VNC Server. That doen's leave much RAM for maps, so I have to keep the map files small. The RPi is mounted next to the projector. The RPi WiFi and VNC settings have to be configured before it is mounted up in the ceiling with the projector.

As I said, I've tested this. It works. Dell 3400MP projector 76" from surface. RPi 1 Model B ($15 on ebay) as specified, TL-WN725N WiFI Adapter ($10), HDMI to M1-AD ($5) adapter. The projected image is 25 x 39 inches. Perfect size for miniatures. But as I said, not ideal because of memory consumption on this RPi. It will probably work much better with an RPi 3 Model B ($40 amazon) that is many times faster CPU and has twice the memory.

If MapTool eventually implements the player map as media stream (like RTSP) rather than a web page then any modern web browser can simply enter the URL (rtsp:///player.mp4) and display the stream. An http URL can also be created as a stream manager like a web cam control page (set zoom, pan, player options, etc). Free media players can also play the same stream without needing a web browser. But also, if it is implemented as a stream like this, then the UPnP API library I mentioned can also command a Smart TV, projector or other media player to display that stream without the need to use a lame TV remote control to try to manually enter a URL.

Lastly, it's possible to use a $10 Raspberry Pi Zero and a WiFi dongle to turn any HDMI or DVI TV or projector without HDCP into a UPnP/DLNA media player capable of playing media streams. If MapTool implements RSTP or other media streaming of the player map, then this would be the coolest possible outcome with compatibility with the most possible media display devices.

[mjmeans]

More information.... from https://en.wikipedia.org/wiki/Digital_Living_Network_Alliance ....

Home Network Devices

• Digital Media Server (DMS): store content and make it available to networked digital media players (DMP) and digital media renderers (DMR). Examples include PCs and network-attached storage (NAS) devices.
• Digital Media Renderer (DMR): play content as instructed by a digital media controller (DMC), which will find content from a digital media server (DMS). Examples include TVs, audio/video receivers, video displays and remote speakers for music. It is possible for a single device (e.g. TV, A/V receiver, etc.) to function both as a DMR (receives "pushed" content from DMS) and DMP ("pulls" content from DMS)
• Digital Media Controller (DMC): find content on digital media servers (DMS) and instruct digital media renderers (DMR) to play the content. Content doesn't stream from or through the DMC. Examples include tablet computers, Wi-Fi enabled digital cameras and smartphones.

Ideally, MapTool ...

1. Would operate a digital media server (DMS) by creating graphic frames of the player map and present them as a video media stream as a UPnP Media Server. Any home media player would be able to discover the MapTool player map and show it.

2. Would act as digital media controller (DMC) and be able to cast its media directly to any digital media renderer (DMR) such as a Smart TV or WD TV Live Streaming Media Player or RPi media player.

3. Provide a web interface with a Java based media player capable of playing the digital media stream. The web interface would also have other controls that allow map scrolling, scaling, etc. as the users might need if the map is displayed in a browser window.

4. Would not require, any third party streaming service such as twitch, youtube gaming, facebook live, etc. It would be nice to support third party streaming services for internet games, But, requiring an unnecessary third party internet service to run a home game is a bad design choice.

[Azhrei]

These are interesting ideas. I can see some of this being pretty useful (streaming to a TV). For a streaming video service, I don't see any practical way to pan and zoom, so you'll see what the server wants you to see.

(I'm going to label the person watching the streaming video the "observer".)

I suppose it would be easy enough to have modes so that Mode A always shows the visible portion of the map (which might be zoomed out too far to see or recognize individual tokens), Mode B might be zoom-the-active-tokens (which could get disconcerting when multiple different tokens are selected sequentially by a macro), Mode C might be zoom-to-party so that all PC tokens fit on the screen (doesn't allow an observer to follow NPCs that might be owned by a PC), and probably a few others.

I don't see any obvious way to switch between modes from a client. If the video stream worked like a DVD (a typical MPEG does not), there could be programmable buttons on the user interface. If there are multiple different rendering sources inside MT to support the above modes, what is the cpu load on the server going to be like? If a GM wants to control the observer's experience, how do they do that?

In any case, useful ideas for the future. I don't have much of an interest in video, but undoubtedly someone does. I'm curious how JavaFX might support being a DMS, either now or in the future.

Edit: I did a little research. It appears that RTSP is not a transport protocol, but a control protocol. And according to https://developer.mozilla.org/en-US/Apps/Fundamentals/Audio_and_video_delivery/Live_streaming_web_audio_and_video, there is no browser support for RTSP 2.0 yet without using plugins.

While a live streaming technique might solve the display of the media to a projector in your case, it's likely to be too bandwidth heavy. And I, for one, am not convinced there is a large enough need for this feature. Consider that a web browser can be handed the background map, the FOW overlay, and the token images as separate assets and then JavaScript can be used to place them with an open WebSocket to obtain updated positions as they happen. When something changes, only the asset that changed must be sent to the browser. That will likely be the FOW and the token positions. This means much less bandwidth and this can be done now in any browser without plugins.

However, as I said I'm curious how JavaFX might fit into this, so I'll be reading up on that aspect in the future. :)

[mjmeans]

In reference to your last paragraph, while a web browser could do this, it would not work on a TV monitor which doesn't have a web browser without using an RPi or other IoT device driving the TV. And even at that, you would still need some way to control the IoT device to tell it what IP address to connect to and the WiFi connection password, etc. Most smart TV's have WiFi capability built-in but their built-in web browsers are terrible (well, at least my Samsung 3D TV is terrible; I can't actually say that most are since I've only ever tried one other smart TV).

[jrmiller82]

I think you'd just want to be able to stream to the TV as if it was a dumb projector right?

[JamzTheMan]

So, I'm going to go out on a limb and say in this sort if set up, you are only wanting a "view" only setup, right? You don't need to control the map or anything. And really you just want a Fullscreen view?

So, would it be better/faster/easier is the MT server client took a "player view" and streamed that as a 1080p video stream?

I think that was the original point as well. JavaFX does support mp4 http streaming so this could be possible?

And only one instance of mtbwould be required. Not sure how much CPU/GPU this would consume...

[Azhrei]

I did some reading on JavaFX last night and I only saw references to being a media renderer, not generating a stream with JavaFX. However, I didn't look at pay-for third-party products because, well, I'm not going to pay for it. :smile:

I'll have to play with the browser on my Sony tv and see what it looks like. But yes, I think he was looking for a view-only display. He's willing to spend money on a Pi and other hardware, but said he found commercial streaming apps that could do this on Windows (but I assume at a higher price).

@mjmeans You are right that the VNC server should be run with a password, but it's not required. Especially when you run the server in read-only mode so that clients cannot take over the keyboard or mouse; in that case, the password would protect viewing of the image but you really don't care if a neighbor is viewing your game, right? (Assuming they had the wifi password for your router.)

The more I think about it, the more I like the browser approach with some intelligent JavaScript. It would work for tablets, phones, and so on. It has reduced bandwidth. Browsers will only get better and more intelligent. It would allow for future support of interaction with the MT server. And it relies on current standards for things like JavaScript. Given the number of options for displaying browser content on a tv, it seems pretty reasonable to me. Of course, the infrastructure for the browser will require significant developer resources so it won't be accomplished overnight.

[JamzTheMan]

Hmm you could be right, but I thought it could stream out as well.

[mjmeans]

@Azhrei Actually my desire is to be able to use ANY digital projector as wireless map display of the player map view projected to a table surface. There are many low cost projectors on the market but more importantly there are lower cost ($25) used projectors at thrift stores. I have a Microsoft Wireless Display Adapter ($50) which is capable of acting as a media renderer (as well as acting as a Miracast receiver). It would be perfect except for the fact that it only works with HDMI devices that support HDCP so it will not work with those old cheap used projectors. I'd bet there are other (and cheaper) clones of the Microsoft Wireless Display Adapter that do not have that restriction and there are also HDMI to VGA adapters that could be used for projectors that don't even have DVI. But I don't want to waste money buying several items to find a solution especially since MT cannot yet stream the player map to a media renderer TV.

As for bandwidth, not an issue. The displayed map is static with only very rare changes when the GM moves something or adds a fireball radius or whatever. The frame rate will be high and the bandwidth will be low because there will be virtually no change from one key frame to the next. MP4 has very high compression with essentially static sequential video frames.

As for control of the player map, the current feature of MT to force players to a particular map, center and zoom is good enough for control of a projected map. GM selects the map and zooms in or out sending Ctrl-E,-F until the grid size is close enough for minis. Only takes a few seconds. Of course, if MT was given the projection size (such as 48 x 32 inches) then the map could be automatically zoomed to the appropriate level to achieve 1" squares as projected.

So what I have now, as I've stated, is a PC with WiFi running RealVNC Player to connect to a Raspberry Pi using VNC server physically connected to the projector's DVI port using an HDMI to DVI adapter. So to use this setup I have to run MT on my PC and start a MT server. Then boot the Raspberry Pi and log in to it's desktop using VNC. Running it's own copy of MT and connect to the MT server as a player. Then wait for the resources to load. Then set it to maximize it's window. Then I can close VNC player to eliminate the bandwidth and CPU resources it takes up on the Raspberry Pi when there is an active connection. Then on the GM PC, I select the map I want projected and CTRL-E+F to set the zoom level the way I want it.

The way I'd like it to work is this: Turn on the projector. Start MT on my PC. Select a menu item called "Cast to Device", MT looks for WiFi media renderer's on the local network and displays any it finds. I select the device I want to cast and press OK. Presto map is displayed. No need to have done anything with the projector at all other than turn it on. Quite a bit simpler. This is in fact how Windows Media Player can play and cast audio or video to a TV.

[Azhrei]

So is it typical for streaming servers to push to the renderers? I would expect that the server would advertise its streams but the renderer would be expected to pull the content. In other words, MT would listen on a given network port, and the renderer would broadcast to that port on the network and wait to see who responds, letting the user pick the streaming server they wish to connect to.

(Sorry, I'm a bit ignorant of the specifics of streaming technology.)

[mjmeans]

Sort of. Here's the way I understand it:

The TV (or whatever) operates as a DMR (digital media renderer). In the case of a smart TV that has a remote control interface, the TV can have it's own app for browsing and selecting media content advertised by a DMS (digital media server). I believe DMR also advertises that it can be a destination for media, but it doesn't implement any kind of automation like a DMC does.

A DMC (Digital Media Controller) is a device which is able to request a DMS (Digital Media Server) to send a media stream to a DMR (Digital Media Renderer) and to control it. A DMC may also manipulate the content enabling it to achieve the best output for the device. That's how "Cast to Device" works.

I believe DMR was originally designed to pull the media from and advertising DMS. But that is speculation on my part based on my perception that DMC and the "cast to" features in PC's seems to have come as a later development.

So in order for MT to do as I suggest, it would have to act as both a DMC and DMS; automating the DMR's connection to itself (as DMS) and controlling the media stream. Without DMC, the monitor or projector would need some kind of IR (or other) remote control to browse for and be able to select the MT stream to display it. Thus "dumb" monitors or simple WiFi monitor adapter would not be work.

[thelsing]

Out of focus. There are enough applications to stream the desktop that can be used for this.