Wires-X with YSF2xxx

[g4klx]

We need to modify the gateway to respond to a Wires-X command to link to a YSF2xxx program with a Wires-X Disconnect to the user and then a Wires-X Connect to the YSF2xxx. The former is easy to achieve, simply call m_wiresX->processDisconnect() but the connect is more complex. It needs to appear to come from the user and the gateway doesn't create Wires-X Connects as it only replies to them normally.

We need to get a dump of a Wires-X Connect and then create a new function for it, and remember to send it from the YSF Network to the correct YSF2xxx program rather than to the user. I may have such a dump at home but a new one would be nice. Failing that, reverse engineering my own Wires-X Connect handler is probably an easy option.

The individual reflectors have a new flag added to them to indicate that they are Wires-X capable, and this is only set for the YSF2xxx programs.

The branch YSF2xxx has been created for developing this capability.

[AndyTaylorTweet]

Does this help:

M: 2019-01-29 13:21:37.770 Wires-X Connect Request M: 2019-01-29 13:21:37.770 0000: 05 5D 71 5F 28 20 20 20 20 20 20 20 20 20 20 03 .]q_( . M: 2019-01-29 13:21:37.770 0010: 9D 00 00 00 .... M: 2019-01-29 13:21:38.775 Wires-X Connect Reply M: 2019-01-29 13:21:38.775 0000: 00 5D 51 5F 26 39 32 35 35 37 4D 57 30 4D 57 5A .]Q_&92557MW0MWZ M: 2019-01-29 13:21:38.775 0010: 2D 4E 44 20 22 42 6C 61 63 6B 77 6F 6F 64 2C 20 -ND "Blackwood, M: 2019-01-29 13:21:38.775 0020: 49 4F 31 32 20 20 20 20 20 20 20 20 20 20 20 20 IO12 M: 2019-01-29 13:21:38.775 0030: 20 20 20 20 20 20 20 20 20 30 30 30 20 20 20 20 000 M: 2019-01-29 13:21:38.776 0040: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 M: 2019-01-29 13:21:38.776 0050: 20 20 20 20 30 30 34 33 34 2E 30 30 30 30 30 30 00434.000000 M: 2019-01-29 13:21:38.776 0060: 2D 30 30 30 2E 30 30 30 30 30 30 20 20 20 20 20 -000.000000 M: 2019-01-29 13:21:38.776 0070: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 03 . M: 2019-01-29 13:21:38.776 0080: F6

[g4klx]

That's excellent. I won;t be able to do anything substantive while I think about it. The Connect request is suitably simple, thankfully. It still doesn't look like a complex problem but I am concerned that we may have state issues, let me explain. My software assumes that when you connect to something then you're data goes to them until disconnected, Now we need the system to remain connected even though we sent a Disconnect reply back. Now we have the YSF2xxx programs tagged in the system, it may be easier to apply a state bypass.

[AndyTaylorTweet]

I quite understand, we only want to send the RF back of a disconnect, without actually disconnecting, while passing a connect request to the YSF2xxx so that the subordinate gateway responds to tell your radio where you are.

Here is the same dump from the YSF2DMR gateway so that we can see what that looks like;

M: 2019-01-29 13:39:17.275 Wires-X connect M: 2019-01-29 13:39:17.275 0000: 15 5D 71 5F 28 20 20 20 20 20 20 20 20 20 20 03 .]q_( . M: 2019-01-29 13:39:17.276 0010: AD 00 00 00 .... M: 2019-01-29 13:39:18.380 Wires-X Connect Reply M: 2019-01-29 13:39:18.380 0000: 00 5D 51 5F 26 38 31 38 33 34 4D 57 30 4D 57 5A .]Q_&81834MW0MWZ M: 2019-01-29 13:39:18.380 0010: 2D 4E 44 20 57 61 6C 65 73 2C 20 55 4B 20 20 20 -ND Wales, UK M: 2019-01-29 13:39:18.380 0020: 20 20 31 35 30 30 30 30 39 4C 4F 43 41 4C 20 20 1500009LOCAL M: 2019-01-29 13:39:18.380 0030: 20 20 20 20 20 20 20 20 20 30 30 30 20 20 20 20 000 M: 2019-01-29 13:39:18.380 0040: 20 20 20 20 20 20 44 65 73 63 72 69 70 63 69 6F Descripcio M: 2019-01-29 13:39:18.380 0050: 6E 20 20 20 30 30 34 33 34 2E 30 30 30 30 30 30 n 00434.000000 M: 2019-01-29 13:39:18.380 0060: 2D 30 30 30 2E 30 30 30 30 30 30 20 20 20 20 20 -000.000000 M: 2019-01-29 13:39:18.380 0070: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 03 . M: 2019-01-29 13:39:18.380 0080: 18

[AndyTaylorTweet]

I think I spotted why this isn't working as I expected it to: from YSFGateway log: M: 2019-01-29 13:22:44.360 0000: 02 5D 51 5F 26 39 32 35 35 37 4D 57 30 4D 57 5A .]Q_&92557MW0MWZ from YSF2DMR log: M: 2019-01-29 13:39:18.380 0000: 00 5D 51 5F 26 38 31 38 33 34 4D 57 30 4D 57 5A .]Q_&81834MW0MWZ

The repeater ID's don't match... - this is why the radio is ignoring the WiresX back from the subordinate gateway....

[AndyTaylorTweet]

Aligning the fields used to create the ID in the YSFGateway and YSF2XXX helps greatly, no more need to worry so much about the WiresX on/off at the radio - I'll modify the Pi-Star dashboard to account for that so that we set those fields the same - this causes the repeater ID to match up and the radio stops ignoring the answers.

These fields must match perfectly, YSF2XXX does not include the speachmarks currently - so right now the field has to exclude them:

YSFGateway.ini [Info] Name=

YSF2DMR.ini [Info] Description=

Once those match - connecting to a YSF2xxx mode answers connected from the YSFGateway, radio shows linked to "YSF2DMR" for example, but now searches work (searches are passed to the YSF2DMR mode) and once a connection is chosen, the answers now work, radio shows connected TG.

Unlink also behaves like it should without having to WiresX off and back on.

I'm sorry I didn't work this out sooner :( but its a huge leap forwards, and a reduction in complexity! If would still be nice to forward a WiresX connect command to the subordinate gateway on connect, so that it replies with the current connected REF/TG etc, but that is the only thing I believe we are missing.

[g4klx]

I've implemented a sendConnect() function that requires the correct network object passed to it, m_ysfNetwork I think. Which should do the job.

[AndyTaylorTweet]

I need some time to sit and figure out exactly how/where I wan't to use that. We are now VERY close to having the functionality working well;

Once I modified the configs to mach up the fields, the need to stop/start WiresX on the radio is gone... so that is MUCH improved from a user perspective.

The disconnect command is currently processed locally and passed to subordinate gateway, so that all works perfectly (YSF2P25 will be improved when Andy (CA6JAU) fixes that up), the radio accurately shows disconnected and the subordinate YSF2xxx gateways are un-linked at the same time.

The last steps are to send the WiresX DX (connect) to the subordinate gateway, when you connect to the subordinate gateway with YSFGateway, I've read your code - but ran out of skills to implement it, I'll get there but it just might take me a while.

[AndyTaylorTweet]

OK I got it, PR submitted

[g4klx]

If you're happy with it, I'll merge it to master.

[AndyTaylorTweet]

Not quite - but its really close

[AndyTaylorTweet]

Something about line 270/271 cases YSFGateway to crash at the moment... (since the change from string compare to checking the reflector feature flag) - this is one of those times where understanding C++ means you are likely to understand the problem immediately where I am not..

for reference the lines from YSFGateway.cpp CYSFReflector* reflector = m_wiresX->getReflector(); if ( (wiresXCommandPassthrough) && (reflector->m_wiresX) ) {

I could use your help with that one - maybe swapping the name of the reflector flag would fix it.

[g4klx]

I wonder if reflector is NULL? Maybe sending the disconnect out cleared it. This could be the state problem I feared.

[g4klx]

I wonder if that new message type is some sort or error/reject message? All merged onto master.

[AndyTaylorTweet]

The new message (after digging through the Yaesu Manual) is supposed to take you back to the screen that get after connecting to the YSF Room, I didn't manage to get it to do that - the radio would ignore most of the answers I sent back; Not having a protocol specification or any more to go on than the dumps I can persuade out of my radio makes it a little tedious to get right (I'm sure you have been here before) - however dis-connect at least does something. I'll work on it some more and see if I can persuade it to do what the Manual suggests rather than dis-connect, but at least now it does somthing.

[g4klx]

Ideally we need to trace a real Wires-X system and find out what the response from their gateway is. I have none that I can hear, and of course it'd require a special setup with an MMDVM on RX only and full data dump enabled. That way we could do a full emulation. How about a request on the MMDVM group or the Pi-Star support forum for someone with the right nous to do so?

[AndyTaylorTweet]

I've been thinking along the same lines, I can get the radio commands from the radio easily enough, but without the answers - it does't help much.

I suspect the most useful thing will be the new FT-2D portable terminal mode thing - its similar to the Icom TA mode, but the Yaesu version allows you to use the FT-2D as an access point too, in place of the HRI and a full size radio.

I'll ask in the group - but also see if I can get hold of the cable set and make some headway myself.

[AndyTaylorTweet]

As you expected - the logic for when to pass WiresX and when not to, was complex, and I didn't quite get it right - it was close, but would never send the WiresX connect to the subordinate; See #133 and #134 for patches for that.

[m0vse]

Hi, I manage both a Yaesu Wires-X repeater (GB3LE) and an MMDVM one (GB3CF) and can hear them both from home and have a hotspot that I should be able to configure to dump the data from Wires-X. What exactly do you need? Phil

[AndyTaylorTweet]

Phil, I will let Jonathan answer on exactly what we need - but in essence - we need to make a custom binary for the hotspot, with some extra logging on, the hotspot is going to listen only - and I/we hope that it will be able to capture the WiresX commands from the radio, and then capture the answers from the WiresX repeater, the idea being that we (by we I really mean Jonathan :) ) can add in support for more of the WiresX commands, even if initially the gateway only answers with empty answers - that is a start.

One question I have on WiresX - can you use the WiresX mode in VW as well as DN? If that is possible I really want to capture the WiresX connect response (from when you press the WiresX button) in VW mode too - in the hope that YSF2P25 can be improved.

[m0vse]

That should be fairly straight-forward to do, I have both an FT2 and a FTM400 so can try them both and should be able to capture as much as needed. I am keen to get this working as well as the Wires-X based repeater does seem to have various extra features and am happy to work with you both to get them added to YSFGateway. I can enable whatever options and recompile the code on my hotspot as I am constantly rebuilding it anyway!

[m0vse]

I am fairly certain both modes work on the Yaesu repeater but I will need to check!

[AndyTaylorTweet]

I am sure they both work on the repeater... Currently when YSFGateway answers you (when you press the WiresX button) it always pops the radio into DN, what I want to be able to do is have it set the radio to VW, I bet the option is there (somewhere).

For the captures, incoming (RF) WiresX that is not processed - the standard binary will already log it: See WiresX.cpp, line 248 CUtils::dump("Unknown Wires-X command", m_command, cmd_len);

So long as the hotspot can hear the repeater and the radio (make really sure you are not linked to anything) then you should see the command form the radio and the reply from the repeater in the log.

Copy / Paste those into a text file - make good notes about what you pressed - what is the command from the radio and what is the response from the server.

I'll QRX now and give Jonathan chance to answer too.

[ChrisK9EQ]

VW or DN doesn't matter. WiRES-X commands/responses are sent using DW. According to the spec, the radios can slip into DW whenever they want. There are specific requirements on how to do this.

The radio will always move to the digital mode specified by the incoming signal. DN1, DN2, VW, or DW. YOU program the radio for which outgoling voice mode to use (DN, VW, or same as incoming). (The entry-level radios have less control over this.)

You are not really communicating with a repeater. You are communicating with the WiRES-X node. The repeater just sends signals along the way and is passive with regards to repeated transmissions (other than modifying call data).

The WiRES-X command/response is pretty simple especially with a good read of the Digital Specification. The biggest problem I see is the very high BER on these hotspots. Since there is no FEC on DW data, a single bit error can make a mess. Therefore it's very, very important to do sanity checking and dump commands where ANYTHING in the transmission is wrong (any invalid FICH data, checksum failures, etc.) In that case just don't do anything and it will be up to the radio to retry. In all cases THE DW INFORMATION SHOULD NEVER BE SENT ON THE NETWORK to a reflector.

One last thing, there are two WiRES-X modes you need to worry about: Advanced; and Introductory radios. The FTM-1/400, FT1/2 will operate the same. Only need to test with one of them. Likewise the FTM-3200/7,7250 and FT70 will behave the same and you will only need to test one of those. However you WILL need to deal with both families of radios. I'd start with the introductory radios as they are a much simpler case.

[AndyTaylorTweet]

@ChrisK9EQ Wires-X - Never passing to the network - Yes I agree, we've been working on a very special case here that is the exception to the rule, but WiresX will still never make it past the hotspot.

WiresX is always DW - yes I had noticed that, BUT, if I set the radio to VW, use the WiresX button, even when using YSF2P25 that now creates voice back in VW on link/unlink, the radio still reverts to DN (even in RX) no matter what - I suspect something in the WiresX replies is guiding that response, I could be wrong of course, but that is how it behaves.

Do you have a link to the specification you mention? Trying to find that kind of information has been quite impossible (for me at least), but who knows I may just have been looking for the wrong thing.

Before we get into too much of a tangent on this specific thread, would you start a new one for the suggested change / check for dumping any command frames with invalid FICH (this may already be the case).

[g4klx]

I've updated the gateway so that if you have debugging set to a level of 1 (debug) then all received Wires-X command, whether they are understood or not, are dumped to the log. The reply will also be logged. This will allow you to use an MMDVM to monitor a Wires-X system and hopefully make sense of the replies. Until we have some sort of Wires-X specification, we can only guess based on what we see.

[m0vse]

Thanks Jonathan, I have compiled the updated MMDVM on a duplex hotspot but it seems to be unable to decode the output from GB3LE (Yaesu Fusion repeater). I suspect it might be fairly deaf though so will try to get a 'proper' UHF transceiver setup with an MMDVM-Pi and see if that can decode it.

[AndyTaylorTweet]

I got the cable for the FT-2D - so I how have a live WiresX node to play with within earshot of the YSFGateway / MMDVMHost - setting up a debugging host for this at the moment - hope to start dumping what I see shortly...

[AndyTaylorTweet]

Jonathan - suggest we start with these and see where we want to go from here: http://wiki.pistar.uk/index.php?title=WiresX_Commands_/_Answers

[g4klx]

Hi Andy

I've finally managed to look at that data. It looks like the Wires-X responses have changed since my original data dumps, and we are using an old version apparently. Could you also trace the commands to and from Wires-X for the initial DX button as well as rooms lists, scrolling down, a search and a normal connect to a room and a disconnect. What you have is already very interesting, if you can add a full repertoire it'd be even better.

[AndyTaylorTweet]

Sure - I'll try and dump those tomorrow.

[g4klx]

H All. I've started a new branch for the update of the YSF Wires-X protocol (it's called bad_connect, a bit of a rubbish name, sorry) and added a couple already. If the reflector isn't found, it should reply in the same way as your Wires-X trace. Likewise clicking on an already connected room should bring up the same data packet as you show in your trace. An almost identical reply is sent after a request for International News. The Local News appears to return almost the same reply as for a request to list all rooms.

I'd love to see a Wires-X over-the-air protocol specification. We can get closer and closer to matching it I am sure. If it is possible to set the audio mode on the radio, is it possible to connect to a room that is VW only and have a look at the reply? There are so many unknown fields in the protocol. I suppose if I had a few spare days I could start doing tests and send all sorts of nonsense back to my radio and see what happens!

@ChrisK9EQ have you get any documentation on Wires-X or a link to same. I'd love to get my implementation as close as possible to the real thing.

[AndyTaylorTweet]

I want to say that I am sure I/we/you can flip the correct bits to set a room to VW, however I dont have the options in the WiresX software :( possible because I am stuck using the portable node setup using an FT-2D, this does some things differently, for example for a full Wires-X node, the network side uses UDP, this setup does not, running Wireshark it appears that ALL of the interaction to Wires-X happens via some Yaesu proxy servers over TCP via a single TCP port, there are at least two of these proxies that I have found so far - but this may turn out to be an interesting way into the Wires-X network for YSFGateway (or YSFReflector) at some point in the future - more on that another time.

I'll get setup here today and re-dump the Wires-X connect... I'll then see if I can test some of this.

There will be more commands after these too - but this should hoover up a load of available button presses... Thank-you

[AndyTaylorTweet]

See the new dumps at the bottom of: http://wiki.pistar.uk/index.php?title=WiresX_Commands_/_Answers Re-captured Wires-X button press, also captured the advertisement that happens when the GW is told to change rooms from the software (not the HT). Captured the Wires-X button press answer for already connected and not connected.

[AndyTaylorTweet]

More feedback here (scroll to the very bottom): http://wiki.pistar.uk/index.php?title=WiresX_Commands_/_Answers

Including a compile fail (and guess on how to fix it), the output from the Info Request that isnt quite right (and some why and a compare to Wires-X).

[g4klx]

I should have killed two birds with one stone in my latest commit. I think the 0x26 and 0x28 part is a protocol version number which I'll role through the rest of the responses in due course. I hope that this now compiles and the response is also correct.

[AndyTaylorTweet]

Compiles clean, the info reply is now good too! "Int News" is currently processed like an info request currently (shares the same bytes from byte 2 - byte 5) - so we'll need a way to handle that - but this is a great start. The Info reply is still short (missing the empty bytes) but the radio didn't seem to care.

[g4klx]

Those 0x00 bytes are just filler and AFAIK Wires-X stops processing after the checksum. I may spend time later getting the length exactly right, but it's lower in priority. The News sections are less interesting on the whole I think. I want to refresh the Wires-X interaction for the important aspects and then investigate if we can force a mode change to VW (and to DN where needed) so that the YSF2xxx work correctly always.

[AndyTaylorTweet]

Something about the answer currently does force DN, if the radio is in VW, it reverts to DN currently - so that part seems to be a done deal, it's the other direction (forcing VW for YSF2P25) that seems to be the golden goose at the moment.

There is a suggestion that may shine some light on it on page 7 here: https://www.tapr.org/pdf/DCC2017-Fusion_from_the_Inside_Chris_Petersen_K9EQ.pdf this may be of no use at all too, and I suspect you already know more than is in this presentation - but like you, I have found that finding information on any of this is pretty thin...

[g4klx]

If we accept that the connect reply is where the modes are set, if at all, then I would look at lines 762 and 763 of the file and play with those. I think it's possible that a similar reply from your node would have a '0' in the equivalent of line 762, which would leave line 763 to adjust for the mode. I've looked at the official Wires-X list from Yaesu and none have VW mode as a feature. Otherwise a link to that one would answer our questions. I look forward to more traces as time goes on.....

[g4klx]

I've done some more changes to bring the system more into line with your traces. I am sure there are still differences, for example on disconnections. Can you check the disconnection replies when it's local and at the computer end. I'll get around to the news bits soon once I have the basics working. Also we need to check the room lists and search replies. It sounds a lot, sorry. I don't think it's too far off as it is though.

[msraya]

Hello All!!

I'm replicating all the Wires X node functionality on YSF2DMR. I'm using reverse engineering techniques with an HRI200 setup. I have got working text message upload and download. I'm using local storage under /tmp folder. I'm working with Yaesu Block transfer limitation on mmdvmhost. I'm slowly advancing as I don't have much free time, only on weekend. I will put all the code when it work on my YSF2DMR fork repo. A Wires X RF commands manual can be a good contribution.

I don't know if I can help you with the Wires X Commands. Although It take time with trial and error, I think that in a couple of months I will have a setup with a server to share messages, pictures and voice messages over the Internet.

Manuel EA7EE

[g4klx]

I would be interested in your work. Is it possible that some of it would make more sense in the main YSF Gateway rather than in YSF2DMR? Anything to improve the Wires-X compatibility would be very much welcomed.

[msraya]

Yes that's the point. This code would be useful for YSF development, the fact is that I started working on the YSF WiresX code.

Ok, I can send you the WiresX.cpp and WiresX.h and the new Storage module. The basic dialog processing with the talky is near finished, but the module is not stable yet. So it can be modified and added new functions. Also CWiresX::process returns new codes so the main processing loop can store voice messages and process pictures in bulk data modes. It's not yet developed. The Storage Module initializes in the main program, so it can be used by the main program or the WiresX module.

I will send you the files so you can see. There are so many changes that the files are big. HI HI..

Manuel

[msraya]

I uploaded the files here. You must bear in mind that is work in progress..

Manuel

YSF2DMR.zip

[msraya]

Yep, I get the Picture uploading working right now. Now working on picture downloading...

Manuel

[msraya]

Finally get Picture Download working also.. only I try to reverse engineering HRI-200 packets. I hope soon I upload the code to my github..

Manuel

[g4klx]

I've added a new branch called msraya for you to push commits to. This will allow you to make pull requests which can be added to that branch, then eventually added to the master branch.

[msraya]

OK, I have a list of improvements. But, I'm new to Linux Coding and I think my code could be much improved, so I try to be specific with improvements. Please be free to make constructive criticism on the new code. I have no much free time, however I will try to contribute as soon as possible.

[msraya]

Hello!! I have been very busy with a Database project in python... Arggg!!! I'm enjoying the easter holidays, but i'm busy also.. So I'm sorry the project advances slowly.

I modified the pi-star dashboard because many people ask me to make more easy YSF2DMR installation on pi-star.. so now it is integrated into the pi-star code in my way. I'm sorry no to ask you for permission.. Also you can access photo, message and vocal amb files from the dashboard.. so I should write the php files to access the raw files. I've been a php programming teacher years ago...

I modified the MMDVMHOST code so it can pass picture data without header. It works, but I lost two headers. I think it is because the STM32 microcontroller code seems that lost synchronization, so it lost the header when the time between WIRESX packets is too short.

At the end I lost two packets 90% of time, but I can recover from data loss and get the pictures in disk file without problem. But sometimes I lost sections of the picture, specially with large pictures.... and I end with a wrong picture. It depends on how the HotSpot have been aligned. Also picture uploading does not work with duplex hotspot yet.

Picture downloading does not have any problem because block transfer was implemented long time ago in WiresX.cpp. Only I've had a hard time realizing the proper meaning of all byte fields in the data stream. But I think it is now complete.

I have improved other parts of the code. Only I need is add voice storage and improve the retry mechanism of the APRS-IS thread because with mobile hotSpot It stop APRS synchronization very often.

I have a hard time trying to compile MMDVMHost code. Because I have problems with ttyAMA0 right of access and I also lost OLED display and Logging to file. I will change to last Operating System in beta RC4 and recompile MMDVMHost... Because it does not work with my setup... It compiles, but does not work in daemon mode because lacks of right access to ttyAMA0, OLED not working, I compile the OLED library, but not luck.

So, I try to update the files soon... Regards Manuel

[msraya]

So, In the meantime I manage to compile MMDVMHost... I put the last pi-star beta version and now all is working 100%. Also, I improve the hotspot offset adjustment an I have picture both ways working again. I will try to finish the development now...
Manuel