Open ahfontaine opened 3 years ago
colinbdclark
commented
3 years ago Hi @ahfontaine, sorry to hear you're getting an error. It's hard to know what the issue might be without seeing your implementation. Can you post simplified example of your code so that I can take a look and try it out, along with some example data that you're sending from Unity? Thanks!
ahfontaine
commented
3 years ago No problems! Thanks for coming back quickly to me.
I also changed the title of the issue because I'm realizing it doesn't crash, it just gives errors instead. My bad, lots of pressure going on right now on my end.
So, on node's side (this is almost the same as the udp-browser demo)
var osc = require("osc"),
WebSocket = require("ws");
var getIPAddresses = function () {
var os = require("os"),
interfaces = os.networkInterfaces(),
ipAddresses = [];
for (var deviceName in interfaces){
var addresses = interfaces[deviceName];
for (var i = 0; i < addresses.length; i++) {
var addressInfo = addresses[i];
if (addressInfo.family === "IPv4" && !addressInfo.internal) {
ipAddresses.push(addressInfo.address);
}
}
}
return ipAddresses;
};
var udp = new osc.UDPPort({
localAddress: "0.0.0.0",
localPort: 7400,
remoteAddress: "127.0.0.1",
remotePort: 7500
});
let socketList = [];
udp.on("ready", function () {
var ipAddresses = getIPAddresses();
console.log("Listening for OSC over UDP.");
ipAddresses.forEach(function (address) {
console.log(" Host:", address + ", Port:", udp.options.localPort);
});
console.log("Broadcasting OSC over UDP to", udp.options.remoteAddress + ", Port:", udp.options.remotePort);
});
udp.open();
var wss = new WebSocket.Server({
port: 8081
});
wss.getUniqueID = function () {
function s4() {
return Math.floor((1 + Math.random()) * 0x10000).toString(16).substring(1);
}
return s4() + s4() + '-' + s4();
};
wss.on("connection", function (socket) {
console.log("A Web Socket connection has been established!");
var socketPort = new osc.WebSocketPort({
socket: socket,
metadata: true
});
socketPort.id = wss.getUniqueID();
var relay = new osc.Relay(udp, socketPort, {
raw: true
});
socketPort.on("message", function (oscMsg) {
console.log("An OSC Message was received!"," It's from : ", socketPort.id, oscMsg);
});
socketPort.on('close', function(reasonCode, description) {
console.log((new Date()) + ' Peer ' + socketPort.id + ' disconnected.');
});
});On Unity's end, I would use something like extOSC's demos to check if I can get anything going to Node.
Here in this screenshot I moved something in the UI
Which is confirmed by a console.log on udp (that I removed rom the code)
{
address: '/example/8/pad/1',
args: [ 5.605193857299268e-45, 1.401298464324817e-45 ]
}
{ address: '/example/8/pad/1', args: [ 0, 0 ] }Connection from the front-end to the back-end through WebSockets afterwards :
A Web Socket connection has been established!
Sending data over from Unity to the server again this time will give the error, won't display the console.log in node as well. Which makes me believe that something is missing from that way over to WS.
I think it might be because I'm missing something in the interpretation of the data from UDP to WS. Might it be the reason why?
colinbdclark
commented
3 years ago It's still really hard for me to test this given the environment you're using. I assume I'd need a license for Unity and your code there to actually see what's being sent and debug it?
This is interesting, though. The error you're seeing is quite deep in the guts of osc.js, and is one of those "this should never happen" moments. It's totally a bug in osc.js that this isn't caught and a more helpful error message presented. However, this error suggests that somehow an invalid OSC argument type is being received. osc.js supports all the OSC 1.0 and 1.1 argument types. Is it possible that somehow your Unity code is sending an non-spec-compliant OSC message?
Can you simplify your system for testing purposes and reproduce the error when only receiving OSC messages from Unity to Node.js via UDP? In other words, omit the Web Socket server and relaying of messages from the UDPPort to the WebSocket. Are you able to receive messages from unity and log them to the Node.js console successfully?
ahfontaine
commented
3 years ago Hi @colinbdclark
A free license of Unity is all you would need to test this, and trying something like the SampleScene of OSCJack or demo 8 of extOSC (the one used in that screenshot above) is more than enough. You could simply also use the UDP-Browser demo or any other examples from the examples repo to check out what's going on.
I was able to pinpoint my issue as to the Node back-end because of the following :
So it really points out in the direction of Node at that point.
For the OSC libraries, I'm getting the same issues on both OSCJack and extOSC, both free plugins for OSC for Unity. My tests are based on their internal demo. They are both sending spec-defined OSC as per their description but since it seems like the UDP -> Node part is working, I have some doubts in regards to the signal being out of spec.
Here is the Node code in its entierety :
//--------------------------------------------------
// Bi-Directional OSC messaging Websocket <-> UDP
//--------------------------------------------------
var osc = require("osc"),
WebSocket = require("ws");
const METADATA_ADDRESS = "/metadata";
var getIPAddresses = function () {
var os = require("os"),
interfaces = os.networkInterfaces(),
ipAddresses = [];
for (var deviceName in interfaces){
var addresses = interfaces[deviceName];
for (var i = 0; i < addresses.length; i++) {
var addressInfo = addresses[i];
if (addressInfo.family === "IPv4" && !addressInfo.internal) {
ipAddresses.push(addressInfo.address);
}
}
}
return ipAddresses;
};
var udp = new osc.UDPPort({
localAddress: "0.0.0.0",
localPort: 7400,
remoteAddress: "127.0.0.1",
remotePort: 7500
});
let socketList = [];
const userMappings = {};
udp.on("ready", function () {
var ipAddresses = getIPAddresses();
console.log("Listening for OSC over UDP.");
ipAddresses.forEach(function (address) {
console.log(" Host:", address + ", Port:", udp.options.localPort);
});
console.log("Broadcasting OSC over UDP to", udp.options.remoteAddress + ", Port:", udp.options.remotePort);
});
udp.on("message", function (oscMessage) {
console.log(oscMessage);
});
udp.on("error", function (err) {
console.log(err);
});
udp.open();
var wss = new WebSocket.Server({
port: 8081
});
wss.getUniqueID = function () {
function s4() {
return Math.floor((1 + Math.random()) * 0x10000).toString(16).substring(1);
}
return s4() + s4() + '-' + s4();
};
wss.on("connection", function (socket) {
console.log("A Web Socket connection has been established!");
var socketPort = new osc.WebSocketPort({
socket: socket,
metadata: true
});
socketPort.id = wss.getUniqueID();
var relay = new osc.Relay(udp, socketPort, {
raw: true
});
socketPort.on("connection", function (username) {
console.log(username);
});
socketPort.on("message", function (oscMsg) {
switch (oscMsg.address) {
case METADATA_ADDRESS:
// Messages to this address will only contain a single argument,
// a metadata object about the connection
userMappings[socketPort.id] = JSON.parse(oscMsg.args[0].value);
break;
default:
// Normal handling
console.log("An OSC Message was received!"," It's from : ",
socketPort.id, `(${userMappings[socketPort.id] && userMappings[socketPort.id].username})`, oscMsg);
}
});
socketPort.on('close', function(reasonCode, description) {
console.log((new Date()) + ' Peer ' + socketPort.id + ' disconnected.');
});
});It is primarily based on the UDP-Browser example in the example repo, with some tweaks to handle keeping track of how many clients are open and Usernames.
Oh and maybe a detail that I'm thinking might be a cause for issues : I'm using React for the front-end. I don't think it would cause any issues, but my friend and I encountered an issue with a similar "writer" issue, in a different situation : Trying to send a message without a setTimeout connection. This is a code snippet with a workaround, in this case trying to get a "username" defined by the user to be stored in Node.
// Need to investigate this, seems to misbehave if messages are sent
// immediately... can check if connection open?
setTimeout(() => {
port.send({
address: "/metadata",
args: [{
type: 's',
value: JSON.stringify({username: userValue.current.value})
}]
});
}, 100);It's not important and we found this workaround for the time being, but it was odd that it was the same undefined writer issue once again.
Hope this clarifies it a bit!
colinbdclark
commented
3 years ago There's definitely a bug or two in here somewhere, I'll just need some time to take a look and trace through the code.
In terms of your last code snippet, I think the issue there is that you can't send messages on a Port until it has fired its ready event. So you'll need to define a ready listener and send your messages there, as in this example. I'm assuming your code works by luck—you just happen to be waiting long enough with setTimeout that the Port is ready. I'd be curious to see the error message you got in your original implementation, as again osc.js should be providing a more helpful error message.
ahfontaine
commented
3 years ago Hi @colinbdclark, I tried again with a ready event right before the connection and it did work! So something must have been messed up in my front end code when we tried.
Thanks for checking out my code. Also, I wondered if there were any documentation on the osc.Relay anywhere? I'm trying to figure out a way to prevent specific sockets to send OSC unless they are permitted as per an array (e.g. 3 users, User 1 can send OSC for 20 seconds while User 2 and 3 can't, after 20 seconds it's User 2's turn, etc) and just wanted to check if there was some sort of "mute" function I could make.
colinbdclark
commented
3 years ago Hi @ahfontaine,
So just to confirm, all of your issues were resolved by waiting for the ready event? If so, I'll update the example code accordingly, and that gives me a much clearer path to addressing the issue of clearer error messages in this case.
There's not much to the osc.Relay implementation, and I haven't extensively documented it because I haven't wanted to strongly promote its use. I may remove it in a future version of osc.js, since I think this sort of relaying logic can get complex quickly, and is best implemented in application code. For the use case you describe, I think you should be able to implement a custom relayer quite easily, but the osc.Relay object won't be of much help.
ahfontaine
commented
3 years ago Well, on the front-end part of things, it is fixed yes. I still have Unity issues in terms of sending the data over to the Websocket clients, where the original issue of this thread happens. Looking at the error message again, I'm thinking it has to do with the conversion of the UDP OSC message to Websocket.
If that can help you, here are some console.logs on the CollectArgument function. First is the metadata, "on ready" message, and the second one is the one from Unity.
[ { type: 's', value: '{"username":"asd"}' } ]
{
localAddress: '0.0.0.0',
localPort: 7400,
remoteAddress: '127.0.0.1',
remotePort: 7500
}
{
byteLength: 12,
parts: [
Uint8Array(12) [
47, 109, 101, 116, 97,
100, 97, 116, 97, 0,
0, 0
]
]
}
[ true ]
{
socket: <ref *1> WebSocket {
_events: [Object: null prototype] { close: [Array], message: [Function] },
_eventsCount: 2,
_maxListeners: undefined,
_binaryType: 'nodebuffer',
_closeCode: 1006,
_closeFrameReceived: false,
_closeFrameSent: false,
_closeMessage: '',
_closeTimer: null,
_extensions: {},
_protocol: '',
_readyState: 1,
_receiver: Receiver {
_writableState: [WritableState],
_events: [Object: null prototype],
_eventsCount: 6,
_maxListeners: undefined,
_binaryType: 'nodebuffer',
_extensions: {},
_isServer: true,
_maxPayload: 104857600,
_bufferedBytes: 0,
_buffers: [],
_compressed: false,
_payloadLength: 36,
_mask: <Buffer a5 03 1c a5>,
_fragmented: 0,
_masked: true,
_fin: true,
_opcode: 2,
_totalPayloadLength: 0,
_messageLength: 0,
_fragments: [],
_state: 0,
_loop: false,
[Symbol(kCapture)]: false,
[Symbol(websocket)]: [Circular *1]
},
_sender: Sender {
_extensions: {},
_socket: [Socket],
_firstFragment: true,
_compress: false,
_bufferedBytes: 0,
_deflating: false,
_queue: []
},
_socket: Socket {
connecting: false,
_hadError: false,
_parent: null,
_host: null,
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 4,
_maxListeners: undefined,
_writableState: [WritableState],
allowHalfOpen: true,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: [Server],
_server: [Server],
parser: null,
on: [Function (anonymous)],
addListener: [Function (anonymous)],
prependListener: [Function: prependListener],
_paused: false,
timeout: 0,
[Symbol(async_id_symbol)]: 11,
[Symbol(kHandle)]: [TCP],
[Symbol(kSetNoDelay)]: true,
[Symbol(lastWriteQueueSize)]: 0,
[Symbol(timeout)]: null,
[Symbol(kBuffer)]: null,
[Symbol(kBufferCb)]: null,
[Symbol(kBufferGen)]: null,
[Symbol(kCapture)]: false,
[Symbol(kBytesRead)]: 0,
[Symbol(kBytesWritten)]: 0,
[Symbol(RequestTimeout)]: undefined,
[Symbol(websocket)]: [Circular *1]
},
_isServer: true,
[Symbol(kCapture)]: false
},
metadata: true
}
{
byteLength: 20,
parts: [
Uint8Array(20) [
47, 101, 120, 97, 109, 112,
108, 101, 47, 56, 47, 98,
117, 116, 116, 111, 110, 47,
53, 0
]
]
}Here is with the same setup EXCEPT I'm using the udp-browser example node server. Interestingly enough, the Front-End message is interpreted differently. Might be the problem?
[ '{"username":"adasd"}' ]
{
localAddress: '0.0.0.0',
localPort: 7400,
remoteAddress: '127.0.0.1',
remotePort: 7500
}
{
byteLength: 12,
parts: [
Uint8Array(12) [
47, 109, 101, 116, 97,
100, 97, 116, 97, 0,
0, 0
]
]
}
[ false ]
{
socket: <ref *1> WebSocket {
_events: [Object: null prototype] { close: [Array], message: [Function] },
_eventsCount: 2,
_maxListeners: undefined,
_binaryType: 'nodebuffer',
_closeCode: 1006,
_closeFrameReceived: false,
_closeFrameSent: false,
_closeMessage: '',
_closeTimer: null,
_extensions: {},
_protocol: '',
_readyState: 1,
_receiver: Receiver {
_writableState: [WritableState],
_events: [Object: null prototype],
_eventsCount: 6,
_maxListeners: undefined,
_binaryType: 'nodebuffer',
_extensions: {},
_isServer: true,
_maxPayload: 104857600,
_bufferedBytes: 0,
_buffers: [],
_compressed: false,
_payloadLength: 40,
_mask: <Buffer 1d 5a a3 5a>,
_fragmented: 0,
_masked: true,
_fin: true,
_opcode: 2,
_totalPayloadLength: 0,
_messageLength: 0,
_fragments: [],
_state: 0,
_loop: false,
[Symbol(kCapture)]: false,
[Symbol(websocket)]: [Circular *1]
},
_sender: Sender {
_extensions: {},
_socket: [Socket],
_firstFragment: true,
_compress: false,
_bufferedBytes: 0,
_deflating: false,
_queue: []
},
_socket: Socket {
connecting: false,
_hadError: false,
_parent: null,
_host: null,
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 4,
_maxListeners: undefined,
_writableState: [WritableState],
allowHalfOpen: true,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: [Server],
_server: [Server],
parser: null,
on: [Function (anonymous)],
addListener: [Function (anonymous)],
prependListener: [Function: prependListener],
_paused: false,
timeout: 0,
[Symbol(async_id_symbol)]: 11,
[Symbol(kHandle)]: [TCP],
[Symbol(kSetNoDelay)]: true,
[Symbol(lastWriteQueueSize)]: 0,
[Symbol(timeout)]: null,
[Symbol(kBuffer)]: null,
[Symbol(kBufferCb)]: null,
[Symbol(kBufferGen)]: null,
[Symbol(kCapture)]: false,
[Symbol(kBytesRead)]: 0,
[Symbol(kBytesWritten)]: 0,
[Symbol(RequestTimeout)]: undefined,
[Symbol(websocket)]: [Circular *1]
},
_isServer: true,
[Symbol(kCapture)]: false
}
}
{
byteLength: 20,
parts: [
Uint8Array(20) [
47, 101, 120, 97, 109, 112,
108, 101, 47, 56, 47, 98,
117, 116, 116, 111, 110, 47,
53, 0
]
]
}Oh also,
"There's not much to the osc.Relay implementation, and I haven't extensively documented it because I haven't wanted to strongly promote its use. I may remove it in a future version of osc.js, since I think this sort of relaying logic can get complex quickly, and is best implemented in application code. For the use case you describe, I think you should be able to implement a custom relayer quite easily, but the osc.Relay object won't be of much help."
Considering I based most of my node logic on the UDP-Browser/Browser examples in the example repos, would you have a recommendation on what would be the best practice to do a similar thing (Bridge in the middle sort of approach instead of Point to point)?
ahfontaine
commented
3 years ago Hi @colinbdclark, I think I found where the issue stems from :
var socketPort = new osc.WebSocketPort({
socket: socket,
metadata: true
});if I remove the "true" and send a message from Unity to the back-end/websocket :
There you go. So Metadata will cause this issue.
colinbdclark
commented
3 years ago Hi @ahfontaine great detective work. So I think the issue is that your ports don't match; you've got your UDPPort set up to require type metadata, but your WebSocketPort does not. I expect that as long as you're using the same format for both Ports, it will work fine.
In general, you should always use type-annotated messages (metadata: true) on all Ports since there are a subtle type inference issues that often trip people up. I've tried to make sure all the documentation does too. But it sounds like I need to update the examples as well. I have been meaning to change the default for years but it's a breaking change so have been holding off. Let me know if changing all your Ports to use metadata: true fixes your issue.
There's still a bug here, regardless, which I will try to track down.
colinbdclark
commented
3 years ago I've added a unit test to the test suite for osc.Relay and have confirmed that this is indeed the issue—if either one of the Web Socket or UDP Ports are set to metadata: true and the other is not, the relay will fail. I'll track details about the fix on #187.
colinbdclark
commented
3 years ago I've also filed #186 to track the issue with the examples.
Hi,
I'm trying to get Unity and a Web App to be linked by OSC.Js I based myself on the UDP-Browser demo to make myself the webapp.
Transmitting Websocket data over to Unity works without any issues. Transmitting UDP data from Unity over to the Node server without any websocket connection will work as well.
However, transmitting UDP data from Unity over to the Node server while a single Websocket connection is connected will crash the node process with this error.
Is there something I'm misunderstanding in my use case?