Hammerspoon / hammerspoon

Staggeringly powerful macOS desktop automation with Lua
http://www.hammerspoon.org
MIT License
12.04k stars 584 forks source link

hs.tangent - Add support for Tangent Panels & iPad App #1626

Closed latenitefilms closed 6 years ago

latenitefilms commented 6 years ago

Tangent make a whole bunch of high-end panels to use with creative applications such as editing software, sound editing software (DAWs), etc. They have a range of physical panels, but also an iPad app that uses the same API. The full version of the iPad app is pretty expensive, but they offer a "sample" version which is fully featured but only runs for 1 hour a day.

Looking at the Tangent Developer Support Pack, it looks like it's just using standard TCP sockets for communication. To quote the API documentation:

Communication between the application and the Hub is via streaming TCP/IP sockets. The hub should always be running as a background task and listening for a connection on port 64246.

You can download the API documentation and a sample app here:

http://www.tangentwave.co.uk/developer-support/

I know this is fairly specialised, but I'd love to add Tangent support to Hammerspoon, either via an extension if I have to use Objective-C, or maybe even as a Spoon if I can do it all in Lua-land?

Given this... @cmsj or @asmagill - They offer a Mock Application written in C, which in theory I could try and translate into an (Objective-C) Hammerspoon Extension (similar to what I've done with hs.midi) - however, I'm also thinking given that it's using sockets, I could just do this all in Lua using hs.sockets? Any chance either of you could have a quick look at the example code, and let me know if using hs.sockets is a possibility, as I've had a quick play and haven't had much luck, but I think that's because hs.socket:write() expects a string, and I have no idea what this string should be to communicate correctly with the Tangent Hub (i.e. the background process that runs on your Mac monitoring these socket messages).

Thoughts?

asmagill commented 6 years ago

I haven't look at the link yet, but if it's truly socket based, hs.socket will most likely be sufficient.

As to the string expected by hs.socket:write, remember that a string in Lua doesn't haven to be UTF8 -- lua treats a string as concatenated 8-bit characters which may or may not represent actual ascii or UTF8 data. If you know you need a string of specific bytes, you can create it with a series of string.char commands, e.g. string.char(byte1) .. string.char(byte2) ... ... string.char(byteN) where each byte# is an integer between 0 and 255 inclusive. Depending upon the result, it may or may not actually print, so you can use hs.utf8.hexDump or hs.utf8.asciiOnly to show the contents of your "string" while debugging.

Actually looking closer at the docs, you can pass a series of numbers separated by commas to string.char, so the following would also work if you prefer working with an array of the byte numbers:

byteArray = { 65, 0, 66, 1, 67 } -- a sample which won't print correctly because of the null in the middle
byteString = string.char(table.unpack(byteArray))
print(hs.utf8.hexDump(byteString))
latenitefilms commented 6 years ago

@asmagill - Legend, thanks mate! HUGELY appreciated!

That's all definitely good to know, but I'm still having trouble making it work.

To save you digging through the docs, I'll post some of the stuff here if you have time to read (no stress if you don't!).

The manual says...

Communication between the application and the Hub is via streaming TCP/IP sockets. The hub should always be running as a background task and listening for a connection on port 64246.

To open a communication socket to the Hub the application could use code similar to the following:

typedef int Socket
Socket              socketHandle;
struct sockaddr_in  address;
// create the socket
socketHandle = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
if (socketHandle == -1)
{
    // create socket failed
exit(0); }
// prepare the address
memset(&address, 0, sizeof(address));
address.sin_len = sizeof(address);
address.sin_family = AF_INET;
address.sin_port = htons(64246);
address.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
// connect to the hub
if (connect(socketHandle, (struct sockaddr *) &address, sizeof(address)) == -1)
{
    // connect failed
exit(0); }

Once the application has established a socket connection, message passing is handled by reading and writing data from and to the socket. You will be provided with some platform independent sample code to assist with this.

The Hub then initiates communication by sending an InitiateComms (0x01) command to the application.

15. TIPC protocol format As explained above messages encapsulate a number of commands. Each message starts with a 32 bit integer value which indicates the number of bytes that will follow to complete the message. Commands then follow one by one until the message is complete.

Format: <numBytes>, <command1>, <command2>, ...

Commands passed between the Application and the Hub all start with a 32 bit command ID with the content and length defined by the command itself.

15.1. Data Types The byte order for each data type is defined by the host platform’s hardware. As TIPC comms always remain within one host platform there will never be any conflict between the sender and receiver packing and unpacking the data in different ways. This allows the programmer to copy bytes that make up a value from memory to the data stream (and vice versa) without worrying about the endianness of the underlying hardware.

The same is true for floating point values. Bytes should be copied directly from storage memory into the data stream and back in the same order. This avoids the need to manipulate the data depending on the endianness of the underlying hardware.

Unsigned Int A 32 bit unsigned integer value.

Signed Int A 32 bit signed integer value.

Float A 32 bit signed floating point value.

Bool A particular type of Unsigned Int which is restricted to a value of 1 or 0, indicating True or False respectively.

Character String A sequence of characters represented by single byte values based on ASCII. Strings are not null terminated. Their maximum length is defined in the command. See Section 16 Character Table for valid characters.

Path String A special instance of a Character String which indicates the absolute path of a directory folder.

Given this, can I just do something like this for starters?

local log = require("hs.logger").new("tangent")
local socket = require("hs.socket")
tangentSocket = socket.new()
    :connect("localhost", 64246, function()
        log.df("CONNECTION TO LOCALHOST ESTABLISHED!")
        local info = tangentSocket:info()
        log.df("%s", hs.inspect(info))
    end)
    :setCallback(function(data, tag)
        log.df("DATA RECEIVED: %s, %s", data, tag)
    end)

To be honest, I'm a little bit confused about the differences between hs.socket:connect() and hs.socket:listen().

Any tips?

asmagill commented 6 years ago

At a glance that looks like it should work (or at least be pretty close). I’ll try and take a closer look tomorrow if you don’t get it figured out before hand.

Depending upon how the tangent protocol works, it may or may not send you any data after connecting (though your connected callback should at least get called). You may need to follow up with a ‘:send’ to actually send a message to the receiver before it responds.

At its most basic, Connect is when you want to connect to a socket that some other program has created; you’re the client and are trying to connect now. Listen is when you’re creating a socket for some other process to connect to; you’re creating the server and don’t know when (or even if) the connection will actually occur.

latenitefilms commented 6 years ago

@asmagill - Do I need to trigger hs.socket:read() before hs.socket:setCallback() will actually start working?

Adding hs.socket:read("\n") after hs.socket:setCallback() seems to make the callback do something, which I guess is good, but I still really have no idea what I'm doing.

latenitefilms commented 6 years ago

@asmagill - Sorry for all the basic questions! Do I need a seperate hs.socket instance for receiving data versus sending data?

latenitefilms commented 6 years ago

@asmagill - Ok, so I've made some progress. I can get data back from the Tangent Hub, so at least I know hs.socket will definitely work, which is very exciting! Thanks so much for your help!!

However, I'm still unsure of how best to "listen" for incoming data. I was ASSUMING that the callback would be triggered whenever data was being sent from Tangent Hub to Hammerspoon, however, it seems like I need to trigger hs.socket:read(4) for the callback to be actually triggered. Am I missing something?

Here's my proof of concept. If you have the Tangent Hub software installed, you should also be able to get a response from the Tangent Hub software.

It's not pretty... but it seems to work. Any ideas, or suggestions welcome!

hs.console.clearConsole()

local hubMessage = {
    ["INITIATE_COMMS"]                          = 0x01,
    ["PARAMETER_CHANGE"]                        = 0x02,
    ["PARAMETER_RESET"]                         = 0x03,
    ["PARAMETER_VALUE_REQUEST"]                 = 0x04,
    ["MENU_CHANGE"]                             = 0x05,
    ["MENU_RESET"]                              = 0x06,
    ["MENU_STRING_REQUEST"]                     = 0x07,
    ["ACTION_ON"]                               = 0x08,
    ["MODE_CHANGE"]                             = 0x09,
    ["TRANSPORT"]                               = 0x0A,
    ["ACTION_OFF"]                              = 0x0B,
    ["UNMANAGED_PANEL_CAPABILITIES"]            = 0x30,
    ["UNMANAGED_BUTTON_DOWN"]                   = 0x31,
    ["UNMANAGED_BUTTON_UP"]                     = 0x32,
    ["UNMANAGED_ENCODER_CHANGE"]                = 0x33,
    ["UNMANAGED_DISPLAY_REFRESH"]               = 0x34,
    ["PANEL_CONNECTION_STATE"]                  = 0x35,
}

local panelType = {
    ["CP200-BK"]                                = 0x03,
    ["CP200-K"]                                 = 0x04,
    ["CP200-TS"]                                = 0x05,
    ["CP200-S"]                                 = 0x09,
    ["Wave"]                                    = 0x0A,
    ["Element-Tk"]                              = 0x0C,
    ["Element-Mf"]                              = 0x0D,
    ["Element-Kb"]                              = 0x0E,
    ["Element-Bt"]                              = 0x0F,
    ["Ripple"]                                  = 0x11,
}

local logger    = require("hs.logger")
logger.defaultLogLevel = 'debug'
local log       = logger.new("tangent")
local socket    = require("hs.socket")
local utf8      = require("hs.utf8")

messageBuffer = {}
messageLength = 0
messageCount = 0

function getPanelType(id)
    for i,v in pairs(panelType) do
        if id == v then
            return i
        end
    end
end

function processData(data)
    local id = tonumber(data[1]..data[2]..data[3]..data[4], 16)
    if id == hubMessage["INITIATE_COMMS"] then
        -- 0x01, <protocolRev>, <numPanels>, (<panelType>, <panelID>)...
        log.df("InitiateComms (0x01) Triggered:")

        local protocolRev = tonumber(data[5]..data[6]..data[7]..data[8], 16)
        local numberOfPanels = tonumber(data[9]..data[10]..data[11]..data[12], 16)

        log.df("    protocolRev: %s", protocolRev)
        log.df("    numberOfPanels: %s", numberOfPanels)

        local startNumber = 12
        for i=1, numberOfPanels do
            local currentPanelType = tonumber(data[startNumber + 1]..data[startNumber + 2]..data[startNumber + 3]..data[startNumber + 4], 16)
            local currentPanelID = tonumber(data[startNumber + 5]..data[startNumber + 6]..data[startNumber + 7]..data[startNumber + 8], 16)
            startNumber = startNumber + 8
            log.df("    panelType: %s        panelID: %s", getPanelType(currentPanelType), currentPanelID)
        end

        -- Respond with ApplicationDefinition (0x81):
        byteArray = { 0x00, 0x00, 0x00, 0x81, 0x00, 0x00, 0x00, 0x0B, 0x43, 0x6F, 0x6D, 0x6D, 0x61, 0x6E, 0x64, 0x50, 0x6F, 0x73, 0x74, 0x00, 0x00, 0x00, 0x60, 0x2F, 0x55, 0x73, 0x65, 0x72, 0x73, 0x2F, 0x63, 0x68, 0x72, 0x69, 0x73, 0x68, 0x6F, 0x63, 0x6B, 0x69, 0x6E, 0x67, 0x2F, 0x44, 0x6F, 0x77, 0x6E, 0x6C, 0x6F, 0x61, 0x64, 0x73, 0x2F, 0x54, 0x44, 0x53, 0x50, 0x76, 0x33, 0x5F, 0x32, 0x2F, 0x54, 0x55, 0x42, 0x45, 0x20, 0x44, 0x65, 0x76, 0x65, 0x6C, 0x6F, 0x70, 0x6D, 0x65, 0x6E, 0x74, 0x20, 0x53, 0x75, 0x70, 0x70, 0x6F, 0x72, 0x74, 0x20, 0x66, 0x6F, 0x72, 0x20, 0x4F, 0x53, 0x58, 0x20, 0x76, 0x33, 0x2E, 0x32, 0x2F, 0x4D, 0x6F, 0x63, 0x6B, 0x41, 0x70, 0x70, 0x6C, 0x69, 0x63, 0x61, 0x74, 0x69, 0x6F, 0x6E, 0x2F, 0x73, 0x79, 0x73, 0x00, 0x00, 0x00, 0x00 }
        byteString = string.char(table.unpack(byteArray))
        tangentSocket:send(byteString)

    else
        log.df("Unknown Reply: %s", hs.inspect(data))
    end
end

log.df("CONNECTING TO TANGENT HUB...")
tangentSocket = socket.new()
    :setCallback(function(data, tag)

        local hexDump = utf8.hexDump(data)
        local data1 = string.sub(hexDump, 6, 7)
        local data2 = string.sub(hexDump, 9, 10)
        local data3 = string.sub(hexDump, 12, 13)
        local data4 = string.sub(hexDump, 15, 16)

        table.insert(messageBuffer, data1)
        table.insert(messageBuffer, data2)
        table.insert(messageBuffer, data3)
        table.insert(messageBuffer, data4)

        if messageCount ~= 0 and messageCount == messageLength then
            processData(messageBuffer)

            -- Reset:
            messageBuffer = {}
            messageCount = 0
            messageLength = 0
        end
        if data and messageCount == 0 then
            messageLength = tonumber(data4, 16) -- Each message starts with a 32 bit integer value which indicates the number of bytes that will follow to complete the message.
            messageCount = 0 -- Reset Message Count
            messageBuffer = {} -- Don't add the first 4 bytes to the table, as we've already used them.
        end
        messageCount = messageCount + 4
        tangentSocket:read(4) -- Read the next 4 bytes.
    end)
    :connect("127.0.0.1", 64246, function()
        log.df("CONNECTION TO TANGENT HUB ESTABLISHED.")
    end)
tangentSocket:read(4) -- Read the first 4 bytes, which will trigger the callback.
latenitefilms commented 6 years ago

In the Mock Application included with the Developers API, it sends the following bytes to set things up (this is the output from the command line of the Mock Application):

TX: [ 123][ 0x00 0x00 0x00 0x81 0x00 0x00 0x00 0x0B 0x43 0x6F 0x6D 0x6D 0x61 0x6E 0x64 0x50 0x6F 0x73 0x74 0x00 0x00 0x00 0x60 0x2F 0x55 0x73 0x65 0x72 0x73 0x2F 0x63 0x68 0x72 0x69 0x73 0x68 0x6F 0x63 0x6B 0x69 0x6E 0x67 0x2F 0x44 0x6F 0x77 0x6E 0x6C 0x6F 0x61 0x64 0x73 0x2F 0x54 0x44 0x53 0x50 0x76 0x33 0x5F 0x32 0x2F 0x54 0x55 0x42 0x45 0x20 0x44 0x65 0x76 0x65 0x6C 0x6F 0x70 0x6D 0x65 0x6E 0x74 0x20 0x53 0x75 0x70 0x70 0x6F 0x72 0x74 0x20 0x66 0x6F 0x72 0x20 0x4F 0x53 0x58 0x20 0x76 0x33 0x2E 0x32 0x2F 0x4D 0x6F 0x63 0x6B 0x41 0x70 0x70 0x6C 0x69 0x63 0x61 0x74 0x69 0x6F 0x6E 0x2F 0x73 0x79 0x73 0x00 0x00 0x00 0x00 ]

Once this is transmitted, if I jump into the Tangent Mapper application, I can see "CommandPost" is added to the "Select Application" list, so I know it's working.

However, when I try to do the same thing with Hammerspoon:

log.df("Responding with ApplicationDefinition (0x81).")
byteArray = {
    0x00, 0x00, 0x00, 0x7B,     -- 123 bytes below:
    0x00, 0x00, 0x00, 0x81,
    0x00, 0x00, 0x00, 0x0B,
    0x43, 0x6F, 0x6D, 0x6D,
    0x61, 0x6E, 0x64, 0x50,
    0x6F, 0x73, 0x74, 0x00,
    0x00, 0x00, 0x60, 0x2F,
    0x55, 0x73, 0x65, 0x72,
    0x73, 0x2F, 0x63, 0x68,
    0x72, 0x69, 0x73, 0x68,
    0x6F, 0x63, 0x6B, 0x69,
    0x6E, 0x67, 0x2F, 0x44,
    0x6F, 0x77, 0x6E, 0x6C,
    0x6F, 0x61, 0x64, 0x73,
    0x2F, 0x54, 0x44, 0x53,
    0x50, 0x76, 0x33, 0x5F,
    0x32, 0x2F, 0x54, 0x55,
    0x42, 0x45, 0x20, 0x44,
    0x65, 0x76, 0x65, 0x6C,
    0x6F, 0x70, 0x6D, 0x65,
    0x6E, 0x74, 0x20, 0x53,
    0x75, 0x70, 0x70, 0x6F,
    0x72, 0x74, 0x20, 0x66,
    0x6F, 0x72, 0x20, 0x4F,
    0x53, 0x58, 0x20, 0x76,
    0x33, 0x2E, 0x32, 0x2F,
    0x4D, 0x6F, 0x63, 0x6B,
    0x41, 0x70, 0x70, 0x6C,
    0x69, 0x63, 0x61, 0x74,
    0x69, 0x6F, 0x6E, 0x2F,
    0x73, 0x79, 0x73, 0x00,
    0x00, 0x00, 0x00 }
byteString = string.char(table.unpack(byteArray))
print(utf8.hexDump(byteString))
tangentSocket:send(byteString)

...nothing happens in Tangent Mapper, so I'm assuming something must be wrong with the way we're transmitting the data.

Any ideas?

latenitefilms commented 6 years ago

I just realised that there's a debug version of the Tangent Hub software, which is really help!

I now know the reason it's failing is because of the following error:

ProcessRead: WARNING - message data size of 3800597986 is larger than maximum of 1024

Now I just need to work out how to solve this.

latenitefilms commented 6 years ago

@cmsj - Thanks so much for all your help today! HUGELY appreciated.

I'm still not having much luck unfortunately. I tried updating CocoaAsyncSocket from 7.4.3 to 7.6.2, but that didn't do anything different (although I wonder if we should update it anyway?).

I'll keep playing, and let you know if I come up with anything!

@asmagill - if you have any ideas, let me know!

Have a great Christmas everyone!

cmsj commented 6 years ago

I will play a bit more this evening, but I think I have an idea - we send the data to CocoaAsyncSocket as a UTF8 string rather than as raw bytes. This could well be interfering with the binary protocol for Tangent. I’ll add a method that sends the NSData object unaltered and see what happens :)

asmagill commented 6 years ago

Sorry I haven't had a chance to look that closely at the low level socket code... If the socket module truly is sending data as an NSString object irrespective of what its being given, then this would be a major limitation of the module... anything that does transport of raw data should use NSData. There are some flags that can be given to LuaSkin's toNSObjectAtIndex:withOptions: method specifically to force all strings to be treated as NSData for this very reason.

Let me know what you find, @cmsj... I'll try to find some time myself to look at this, but it's probably going to be a couple of days.

cmsj commented 6 years ago

Sadly, switching the hs.socket code to using unmodified NSData objects, doesn't help. All of the hs.socket tests pass, but TangentHub still objects to the data it's being given.

cmsj commented 6 years ago

Ok, so looking more closely at the NSData object being sent, it seems like there's two extra bytes at the start of the stream, 0x33 0x38.

According to the console, I'm sending:

00 : 00 00 00 81 00 00 00 0B 43 6F 6D 6D 61 6E 64 50  : ........CommandP
10 : 6F 73 74 00 00 00 0B 2F 55 73 65 72 73 2F 63 6D  : ost..../Users/cm
20 : 73 6A 00 00 00 00                                : sj....

But according to NSLog, the NSData that's getting written to the GCDAsyncSocket is:

33380000 00810000 000b436f 6d6d616e 64506f73 74000000 0b2f5573 6572732f 636d736a 00000000
asmagill commented 6 years ago

Are your code changes somewhere online?

cmsj commented 6 years ago

@asmagill https://github.com/Hammerspoon/hammerspoon/pull/1628

asmagill commented 6 years ago

I think your debugging lines may be causing this -- you'r using lua_tostring which uses lua_tolstring and the docs explicitly state that this function can modify the item on the stack. Try commenting out line 445 and see what that does (or if you really want it, move 446 to before 445).

asmagill commented 6 years ago

It's supposed to only do that for numbers, but... without installing this and trying it myself, it's my only thought at the moment. I'll be able to try it out myself in a couple of hours.

cmsj commented 6 years ago

@asmagill I added the lua_tostring() after observing the extra bytes, so I don't think it's that, but even if it was, under the hood that's what LuaSkin is using anyway.

cmsj commented 6 years ago

@asmagill in terms of trying this, if you want to replicate what I'm doing, this is the Lua: https://gist.github.com/cmsj/89548c9ad9c9ef38d021fd2a1e697cd7 and it's talking to http://www.tangentwave.co.uk/download/developer-support-pack/ (specifically the TangentHub-Debug/TangentHub binary inside it).

The Lua sets up a socket to talk to TangentHub, connects, reads the first command from TangentHub and tries to reply, but sends extra bytes and TangentHub prints something like: ProcessRead: WARNING - message data size of 859308032 is larger than maximum of 1024 (and if your mental hex is on point, you'll note that 0x33 0x38 0x00 0x00 == 859308032

cmsj commented 6 years ago

@latenitefilms (you might also want to look at my gist link above - I refactored your code a little and wrote some helper functions for dealing with the byte strings a bit more easily)

cmsj commented 6 years ago

Update: I'm an idiot :)

asmagill commented 6 years ago
        local byteString = buildMessage(appMessage["APPLICATION_DEFINITION"], {"CommandPost", "/Users/cmsj", ""})
        tangentSocket:send(#byteString..byteString)

Isn't this appending the decimal numerical representation of the length of byteString to the string byteString? Shouldn't #byteString be converted to a sequence of hex bytes and then appended instead?

cmsj commented 6 years ago

@asmagill exactly, it should be tengentSocket:send(numberToByteString(#byteString)..byteString). I literally just realised that while reading the gist. It works now:

IPC_ProcessEvent: received 38 byte(s) from connection ID 0x00010000
[ 0x00 0x00 0x00 0x81 0x00 0x00 0x00 0x0B 0x43 0x6F 0x6D 0x6D 0x61 0x6E 0x64 0x50 0x6F 0x73 0x74 0x00 0x00 0x00 0x0B 0x2F 0x55 0x73 0x65 0x72 0x73 0x2F 0x63 0x6D 0x73 0x6A 0x00 0x00 0x00 0x00 ]
ParseCommandData: IPC_APP_COMMAND_APPLICATION_DEFINITION
ParseApplicationDefinitionCommand: appName is 'CommandPost', sysDir is '/Users/cmsj', usrDir is ''

but it only works with my NSData patch applied, without that the data is still garbled (and it's seriously garbled. This is from Wireshark:

0000   e2 88 85 e2 88 85 e2 88 85 26 e2 88 85 e2 88 85
0010   e2 88 85 ef bf bd e2 88 85 e2 88 85 e2 88 85 0b
0020   43 6f 6d 6d 61 6e 64 50 6f 73 74 e2 88 85 e2 88
0030   85 e2 88 85 0b 2f 55 73 65 72 73 2f 63 6d 73 6a
0040   e2 88 85 e2 88 85 e2 88 85 e2 88 85

)

asmagill commented 6 years ago

Glad I could be of some help, I think :-) I guess I've been lucky that the few things I've used socket for so far were valid strings -- this should definitely be merged!

cmsj commented 6 years ago

@asmagill since I don't use hs.socket (I'm just working on this because @latenitefilms appeared in IRC this morning and I had most of the day to myself and it seemed like a fun challenge), would you be able to validate my PR with your config? I trust the tests, but at least one +1 from someone who uses it, would be reassuring :)

cmsj commented 6 years ago

@latenitefilms fyi, I have fixed my gist (but it still won't work with current Hammerspoon until we nail down this hs.socket NSData issue.

latenitefilms commented 6 years ago

Wow. You guys are AMAZING!

Thank you so much!! HUGELY appreciated!

Will have a play shortly.

latenitefilms commented 6 years ago

Thank you so much @cmsj ! The changes in hs.socket fixed the issue! And the tweaks you made in the Gist are also awesome - and a big time saver. THANK YOU!

You guys are the best!

I'm getting pretty close to making use of all the Hammerspoon extensions now! Need to buy a MiLight LED WiFi bridge so I can give that a test run too - although that's slightly more of a challenge to justify adding to CommandPost! :)

Thanks again!!

latenitefilms commented 6 years ago

@cmsj or @asmagill - Any chance you could help me write a byteStringToFloat function?

@cmsj previously came up with this, which works great:

local function byteStringToNumber(str, offset, numberOfBytes)
  assert(numberOfBytes >= 1 and numberOfBytes <= 4)
  local x = 0
  for i = 1, numberOfBytes do
    x = x * 0x0100
    x = x + math.fmod(string.byte(str, i + offset - 1) or 0, 0x0100)
  end
  return x
end
asmagill commented 6 years ago

If the data is packed in it's IEEE binary format, you can probably use string.unpack. Otherwise I'd need a bit more to go on, like a sample and what it's supposed to represent.

latenitefilms commented 6 years ago

Legend, thanks @asmagill !

Using this online calculator, I'd expect 0xBF800000 to equal -1 in float. Is that correct?

I've tried, string.unpack("f", "0xBF800000"), but that gives me:

12446.046875 5

...so I'm obviously missing something?

Here's what the API documentation says:

ParameterChange (0x02)

  • Requests that the application increment a parameter. The application needs to constrain the value to remain within its maximum and minimum values.
  • On receipt the application should respond to the Hub with the new absolute parameter value using the ParameterValue (0x82) command, if the value has changed.

Format: 0x02, <paramID>, <increment>

paramID The ID value of the parameter Data type: Unsigned Int

increment The incremental value which should be applied to the parameter Data type: Float

Here's the Hex Dump of what I'm getting:

00 : 00 00 00 02 00 03 00 01 BF 80 00 00 : ............

Using the byteStringToNumber code in my previous post gives me:

Increment: 3212836864

So I guess my question is, how do I turn "0xBF800000" into a float value?

If it's helpful, here's how they do it in the Mock Application:

// extracts a 32 bit float value that is stored in the byte buffer, returning the address of the byte in the
// buffer following the read value
uint8 *ReadFloat(uint8 *pByteBuffer, float *pValue)
{
    uint8 *pFloatBytes = (uint8 *) pValue;

    // build up the 32 bit value from the first four bytes in the buffer, preserving the byte order from earlier code
    *(pFloatBytes + 3) = *pByteBuffer++;
    *(pFloatBytes + 2) = *pByteBuffer++;
    *(pFloatBytes + 1) = *pByteBuffer++;
    *pFloatBytes = *pByteBuffer++;

    // return the address of the following byte
    return pByteBuffer;
}

// stores the supplied 32 bit float value in the byte buffer, returning the address of the byte in the buffer
// following the written value
uint8 *WriteFloat(uint8 *pByteBuffer, float value)
{
    uint8 *pFloatBytes = (uint8 *) &value;

    // store the appropriate bytes of the 32 bit value into the first four bytes of the buffer, preserving the byte 
    // order from earlier code
    *pByteBuffer++ = *(pFloatBytes + 3);
    *pByteBuffer++ = *(pFloatBytes + 2);
    *pByteBuffer++ = *(pFloatBytes + 1);
    *pByteBuffer++ = *pFloatBytes;

    // return the address of the following byte
    return pByteBuffer;
}

Thank you!

latenitefilms commented 6 years ago

Actually...

> x=0xBF800000
s=string.pack("i8",x)
f=string.unpack("f",s)
print(f)
2017-12-27 08:59:37: -1.0

So maybe this will work? Am I on the right track?

asmagill commented 6 years ago

I don't know about the calculator you reference, but I get this from the console:

> hs.utf8.hexDump(string.pack("f", -1))
00 : 00 00 80 BF                                      : ....

> string.unpack("f", string.char(0, 0, 0x80, 0xBf))
-1.0    5

-- force big-endian:

> hs.utf8.hexDump(string.pack(">f", -1))
00 : BF 80 00 00                                      : ....

> string.unpack(">f", string.char(0xbf, 0x80, 0, 0))
-1.0    5
asmagill commented 6 years ago

Think of the "data" as raw bytes... they're stored in a string because that can hold a sequence of bytes and not have to worry about being "forced" to fit a specific data type. So we use string.char to "rebuild" the specific sequence of bytes string.unpack expects. If this were part of a larger string of bytes and you knew the types and the order they were in you could grab just one of them by providing an offset:

> s = string.pack("ifif", 4, 23.4, 8, 77.2)

> a,b,c,d = string.unpack("ifif", s)

> print(a,b,c,d)
4   23.39999961853  8   77.199996948242

> e = string.unpack("i", s, 9)

> print(e)
8
cmsj commented 6 years ago

I think we can close this, since #1633 merged earlier today :)