search

SteffeyDev / atemOSC

Control ATEM video switchers over the network with OSC messages
http://www.atemosc.com
202 stars 32 forks source link

Hyperdeck Control (via TCP) #58

Closed iamjohnbarker closed 11 months ago

iamjohnbarker commented 6 years ago

I've been wondering if it's possible to add hyperdeck control to atemOSC?

I figure that the Studio/Studio mini being controllable over ethernet would make this possible, but perhaps I am wrong.

Would be great to connect to a Hyperdeck and then send it commands from a midi keyboard!

SteffeyDev commented 6 years ago

Currently AtemOSC uses the Switcher SDK primarily, with doesn’t work with the HyperDeck as far as I know. However, as you identified, there are documented ways of talking to it over the network, so support is not out of the question.

We would either need a developer with access to a HyperDeck to test on (@blair do you have one?), or you would need to be available to regularly test changes throughout the development.

And of course it is up to @danielbuechele to decide if this is within the scope of the project

sneat commented 6 years ago

I know people who have one, but not any that could do regular testing for us.

iamjohnbarker commented 6 years ago

Absolutely only if it's in the scope, but I'm definitely voting for it ;)

As for regular testing, I have the TVS HD and HyperDeck Studio Mini here so happy to test throughout if we decide to do it!

danielbuechele commented 6 years ago

How would a integration of Hyperdeck support look like? Does Hyperdeck has it's own SDK we would need to integrate in our app? Before deciding whether we add this or not, I would like to better understand the implications of it. That being said, if any of you is up for integrating this, I am open to this.

SteffeyDev commented 6 years ago

HyperDeck devices listen on TCP port 9993 for raw ascii text input, with a set of commands outlined in the user manual (pg. 57): http://documents.blackmagicdesign.com/HyperDeck/HyperDeck_Manual_2015-01-19.pdf

@iamjohnbarker what commands would you be looking to control via midi (e.g. playback speed, video input mode, ...)

danielbuechele commented 6 years ago

So basically we could send strings with the commands to /hyperdeck/sendCommand. The software would just take care of opening the connection.

SteffeyDev commented 6 years ago

I assume the api would be something along the lines of /hyperdeck/... or /atem/hyperdeck, and it would be almost a direct mapping of OSC commands to hyperdeck tcp commands.

The protocol is very well documented by BlackMagic, and the only other thing we would need is a new UI for handling connection information with the hyperdeck.

SteffeyDev commented 6 years ago

@danielbuechele that would work, personally I would be a bit more in depth and actually build a full wrapper for each command, but any command not wrappped could use that. But as a starting point it would be fairly trivial to just forward the commands as you suggest

danielbuechele commented 6 years ago

This approach allows us to support any kind of commands and not needing to update the software when new commands become available. We can still have "shortcuts" for certain commands.

danielbuechele commented 6 years ago

So what would the UI look like? We just need an IP for the hyperdeck, and a connection indicator showing a green light while connected. Is there anything else we need?

blair commented 6 years ago

The @blair you cc'ed is not the Blair you're looking for :)

SteffeyDev commented 6 years ago

Sorry @blair, I meant @sneat, no auto complete on mobile so I just go for it.

@danielbuechele I think the UI would actually be that simple, so just a subset of what we are currently using.

sneat commented 6 years ago

My only other suggestion would be too look at supporting multiple hyperdecks right off the bat. Easier to build in the support from the beginning rather than retrofitting it in later.

And @blair you got a good github username :)

iamjohnbarker commented 6 years ago

@SteffeyDev Personally, Record stop/start would be my main uses for it, but building in the function for the buttons on the front (at the very least) would make it useful for other users for sure.

recordandmix commented 6 years ago

I have four HyperDeck Minis and an ATEM Television Studio HD. Would love to get them talking to each other via OSC. Happy to bang on them with code if would help move this feature forward!

sneat commented 6 years ago

@recordandmix yeh it would help. Have you done any objective c before?

recordandmix commented 6 years ago

I’m coming at this more from the OSC side of things, unfortunately. At the moment I’m using the Hyperdecks when triggered from TCP via Applescript. I also do a fair amount of OSC in QLab, so my interest in this feature is to unify that process a bit, get the Hyperdecks under OSC control. I’ve dabbled in ASobjC a little & would be happy to do some more — but I’m fairly noob in that regard. ;)

recordandmix commented 6 years ago

Just noticing this as I'm setting up my rig for an event in a couple of weeks...as we know, ATEM Software Control (7.2) allows you to control up to four HyperDecks from within the application. But the HyperDecks will only allow one client to be connected via TCP at a time, and they even include a "watchdog" function in the deck's firmware to kick that client off after "x" seconds if no further commands are received.

The bad news is that ATEM Software Control appears to be using a TCP session with the HyperDeck to send control commands back and forth, and is "keeping alive" that connection for as long as the two devices are on the same network and the ATEM is configured to control the HyperDeck. This is true whether ATEM Software Control is open or not.

So, scenario: you've configured ATEM Software Control to control your HyperDecks. That means commands can only come from the app; you can no longer send any other TCP commands to the deck; the deck will ignore you and shut down the connection. The only "workaround" is to not connect the HyperDecks to the app at all, and deal with communicating with them separately.

So I would suggest that having atemOSC support control of attached HyperDecks is more than a simple "nice to have." If the user is trying to control the HyperDecks from the ATEM, doing so forecloses the other communication channel with the HyperDeck for other apps.

MarcoCorzani commented 6 years ago

One more vote for integration of Hyperdeck Mini control into atemOSC (-:

SteffeyDev commented 6 years ago

@MarcoCorzani According to @recordandmix, You couldn't use both the app and atemOSC couldn't be connected. Would you be OK just using atemOSC, and not using the app? (disclaimer: I've never used the app, and don't know what functionality it provides)

MarcoCorzani commented 6 years ago

@SteffeyDev Since controlling the Mini in the app is a real pain I would prefer just using atemOSC. The buttons and switches in the app are way to small to be useful.

recordandmix commented 6 years ago

@MarcoCorzani @SteffeyDev to clarify, if you have set up ATEM Software Control to control your Hyperdecks, there is no other way to control them besides pushing buttons on the front. And as long as the ATEM and HyperDeck are on the same network, they will maintain an open communications channel between them, whether ATEM Software Control is currently open or not. So the choice is not "atemOSC or ATEM Software Control". Rather, it's "ATEM Software Control or some other method that lets you communicate with your HyperDecks via TCP". If you go down the rabbit hole of letting ATEM Software Control control your HyperDecks, you foreclose any other way of remotely communicating with them.

And it's for that reason that I think atemOSC should support HyperDeck control. If atemOSC is essentially creating OSC hooks to various ATEM Software Control features, being able to support this one (that is, provide another way of controlling the HyperDecks without having to use the admittedly poor UI of the ATEM Software Control app) eliminates the all-or-nothing aspect of controlling your HyperDecks through ATEM Software Control.

SteffeyDev commented 5 years ago

Ok, so it appears that it is not actually ATEM Software Control that connects to the hyper decks and maintains the connection, but the switcher itself. That is why the TCP connection stays even when ATEM Software Control is closed.

On the bright side, it looks like if you connect the HyperDeck to the switcher (as it sounds like most are), atemOSC can still control it for free through the Switcher SDK. Seems pretty straight forward to me.

What I need to know now for this is exactly what HyperDeck functionality you guys want atemOSC to expose?

Here's a list of functions the SDK has available, let me know which ones you need @iamjohnbarker @recordandmix @randallpacker. Obviously the more you need, the longer it will take to implement:

randallpacker commented 5 years ago

Greetings Peter.

So happy to see such significant new improvements!! I have been using Macros to control my three Hyperdecks from Max/MSP, but this is an incredibly laborious method, so it would extraordinary to have direct control. Besides all the basic functionality you have already listed, one thing that is not currently controllable as far as I know is cueing up a video at a given time point, not just the start. I guess this is what you are referring to as CurrentClipTime? So be able to play a video from a specific clip time would be great. Also to be able to play a video at any speed, forwards and backwards would be fantastic.

That's all I can think of for now. But thanks so much for all the great work! Randall

SteffeyDev commented 5 years ago

It looks like shuttle allows you to change the speed: The Shuttle method starts playback on the connected HyperDeck at the requested speed. I believe you are correct about CurrentClipTime. Not sure how you cue up a video, seems like SetCurrentClip requires you to know the ID of the clip. I've never used a HyperDeck, do you normally set current clip by ID?

To be clear, the methods listed above are those supported by the BlackMagic SDK, not by atemOSC. I could implement any number of those in atemOSC, or all of them if desired. Logic tells me that certain endpoints like Get/SetNetworkAddress don't need to be implemented in OSC though.

recordandmix commented 5 years ago

Hi Peter,

Thanks for jumping on this! I haven’t looked at the SDK recently, but does BMD not offer the “record clip with name:” command to the ATEM? That’s the biggie for me. I have four decks that I’m controlling via TCP to start and stop recordings of the same event from 4 different angles. I then have to pull them up again on demand, so the file names are pretty critical to my application.

SteffeyDev commented 5 years ago

It looks like the SDK does have a list of clips, including the name of the clip and the ID. So atemOSC could grab a list of clips and you could play them back by name or ID. However, I don't see a method of specifying the name of the clip before recording. If ATEM Software Control offers this, can you screenshot or video this functionality so I can better understand what you are talking about?

SteffeyDev commented 5 years ago

Ah, I think I found what you are referring to on page 57 of the manual (http://documents.blackmagicdesign.com/HyperDeck/HyperDeck_Manual_2015-01-19.pdf)

recordandmix commented 5 years ago

yes! That's it. Currently I'm accessing that feature using one-shot netcat (nc) commands sent from the command line. You send record: name: "[clip name]" and you can specify the name the drive will use to write to disk. I'm tagging them as "[scene name]_Cam[XX]", which allows me to recall specific clips as needed. Great as a poor man's instant replay, because if you know the name of the clip when you wrote it, recalling it later is a piece of cake.

Also, I should throw this out there...earlier in the thread Daniel & others mentioned that they were looking for HyperDeck owners who knew a little Obj-C that could bang on code. I still don't really know any Obj-C, but I've been working on a couple of other projects learning Swift...would that skill be helpful on this project at any point?

SteffeyDev commented 5 years ago

I can't seem to see an equivalent method in the SDK, but I can email BlackMagic support and see if they have plans to add this anytime soon.

randallpacker commented 5 years ago

I just looked at the list of protocols and "go to timecode" seems like a good way to cue up a video. I see you can also record a video with a specific name, that would be excellent because there is no way to do that using the software control. I currently rename it manually by opening the SSD in my computer. Finally, I would like to be able to group a set of commands for immediate playback such as: (1)set loop (2) go to clip id (3) go to timecode (4) set speed (5) play. Would this require separate OSC commands or could it be grouped into a single message?

SteffeyDev commented 5 years ago

And @recordandmix yes! Obj-c and swift are syntactically different but functionally do the same things, and you've probably been looking at some Obj-c on StackOverflow anyways while learning swift. It would be great if you can pull down the HyperDeck branch to test some code out once we start development.

@randallpacker Every function would be a separate OSC command, we don't have a way to support bulk OSC commands yet. However, if you send those commands in quick successions (e.g. from a script), it is equivalent to them being grouped. I made an OSC scene controller (https://github.com/SteffeyDev/osc-scenes) that I use to group commands together into a scene, which I believe you could use for your purposes as well. There may be other OSC scene controllers out there as well that have a better community around them (QLab?).

SteffeyDev commented 5 years ago

Related:

recordandmix commented 5 years ago

BTW, @SteffeyDev I would note that the manual you linked (from 2015) has been updated a couple of times since that version, most recently in May 2018. I don't think there's a huge list of changes since then but I do know that BMD has been fiddling with the 9-pin control protocol, so it's possible there's been some spillover into the TCP commands since 2015.

https://documents.blackmagicdesign.com/UserManuals/HyperDeck/20180706-aa8906/HyperDeckManual.pdf

randallpacker commented 5 years ago

Thanks Peter, I have essentially been using Macros to group Hyperdeck commands, so you are right, this can be done with scripting as well, and I could build the equivalent of macros in Max/MSP/Jitter or VDMX. I am using atemOSC for live networked performance from my studio, which would essentially be impossible without it!!

recordandmix commented 5 years ago

Related: https://forum.blackmagicdesign.com/viewtopic.php?f=4&t=85000&p=471599&hilit=record+name+HyperDeck#p471599

@SteffeyDev, wow, so funny to see people complaining about the lack of this particular feature, since we've been talking about it a long time here, too (particularly the fact that the HyperDecks only support one network connection at a time). And they've absolutely nailed the issue -- BMD is all about the "democratization of the television industry," but if you can't name/tag/identify clips as you're recording them, and have to disconnect from the switcher in order to do any housekeeping, you're really cutting this product off at the knees as a production video deck. In my case, as I mentioned earlier in the thread, I have decided to NOT use the HyperDeck integration with the ATEM software panel because I absolutely MUST have the ability to label files as I'm recording them.

SteffeyDev commented 5 years ago

Thanks for the latest. It appears in light of the missing record to filename feature, we may have to go the TCP route instead of going through the switcher SDK. That means that atemOSC has to handle session maintenance and all of that overhead, but may be the best way forward.

SteffeyDev commented 5 years ago

I know this is a long shot, but does anyone have an old/broken/unused HyperDeck they could send me to test with (even if temporarily)? I don't think I could get the TCP sessions working without a physical device. Alternatively, I would be fine VPN-ing in to someone else's environment and testing remotely. Or someone else who has a HyperDeck could spearhead development of this feature of course.

recordandmix commented 5 years ago

There may be other OSC scene controllers out there as well that have a better community around them (QLab?).

@randallpacker I'd be a strong +1 for QLab in this respect, for a couple of reasons. One, it speaks OSC, so grouping together OSC cues in a block, then firing them in sequence, is a piece of cake. Two, it also speaks AppleScript, so sending command line TCP/netcat commands is another option. That's actually how I do it in my rig -- OSC commands to the ATEM when I need to switch views/crossfade/etc., and AppleScript-based do shell script statements to send netcat commands to the HyperDecks.

SteffeyDev commented 5 years ago

And @randallpacker If there are commands that are commonly used together by most users, we could provide a single endpoint for it in atemOSC, but I don't want to make atemOSC API too use-case specific.

recordandmix commented 5 years ago

I know this is a long shot, but does anyone have an old/broken/unused HyperDeck they could send me to test with (even if temporarily)?

Ask me again in about a month! I have a couple of shows coming up through mid-March & can't take apart my rig, but after that we could certainly come up with some sort of a VPN to test things out.

randallpacker commented 5 years ago

Peter, you are welcome to remote into my studio, I have two Hyperdeck Studios and one Hyperdeck Mini. Here is my cell phone: 202.439.4306, feel free to give me a call. Just let me know how I can assist, the Hyperdeck control is really critical to my work.

recordandmix commented 5 years ago

Related:

I know I sound like a broken record on this, but this is exactly the argument for atemOSC to directly support TCP commands to the HyperDecks. If BMD is not going to fix the software control to allow users to access the full suite of TCP commands (or at least the ones everyone seems to be clamoring for), then using software control to control the decks becomes much less attractive (in my case, a non-starter). So supporting the TCP commands in atemOSC, even though we'd be technically bolting on a different process that doesn't actually go through the software control panel, would provide another way to offer the kind of functionality and control lots of people seem to be asking for.

The decks are happy to report back all sorts of status messages on what they're doing, where the transport is, what file is playing, etc. and BMD's insistence that the software panel and decks have a single, persistent TCP channel between them is much more than needed for the other use cases we've been discussing on this thread.

SteffeyDev commented 5 years ago

Yea, I'm on board with the TCP tie in given what we have here, though I understand keeping a TCP connection open because it makes the commands execute faster. If it had to re-open a TCP connection per command then there a lot of extra network traffic going on and it can add lag to the system. That being said, the amount of time that the TCP session is open for is customizable, and can be 0 if desired.

@danielbuechele Would you be interested in doing the UI for this? It sounds like we need form fields to connect to an arbitrary number of HyperDecks (unless there's a reason to limit number of connections), and then maybe some way to specify an alias for each HyperDeck? That way the command would by /atem/hd/<alias|IP>/<command>.

randallpacker commented 5 years ago

Greetings all. I wanted to know if there has been any progress on the AtemOSC control of the Hyperdecks. I want to reiterate the offer to remote into my studio where I have two Hyperdeck Studios and one Mini. Let me know what I can do: I wish I could offer programming help!! Thanks, Randall

SteffeyDev commented 5 years ago

I wish I had time, but very busy with work for the next few weeks at least :/ definitely on my list of things to do when I have time though.

randallpacker commented 5 years ago

Thanks Peter for the update... am REALLY looking forward to real-time, non-linear control of Hyperdeck media.

SteffeyDev commented 4 years ago

Ok, released 2.5.7 with very basic hyperdeck control. The available commands can be seen from the in-app list when you have a hyperdeck connected to ATEM Software Control (8.1). So far record, play, and stop are working. Shuttle, job, clip time, and timeline time may work, but I've been having some issues so I'd love to know how it works for you guys.

randallpacker commented 4 years ago

This is a real game changer to control the Hyperdeck!! You mention in the notes it's made specifically for Catalina. I assume it's also compatible with Mojave? I am about to test it out. Thanks, Randall

randallpacker commented 4 years ago

OK, I just tested and it's mostly working great!! Clip, Play, Stop, Shuttle all working. Not sure about Jog, not having any luck with that. What are the numerical ranges for Shuttle and Jog? That would help.

Also can you provide an example of how to construct the string for Current Time and Current Timeline Time? I assume this is SMPTE time? I don't have that working yet. I'm not clear about the difference between Time and Timeline Time.

Thanks again Peter, this is really magnificent work!!

Randall