WheezyE / Winelink

Installation scripts for running Winlink (RMS Express/Trimode & VARA) on non-Windows computers. Wine & Box86 make this project possible.
68 stars 18 forks source link

VARA 4.7.3 possibly no longer compatible with linux #66

Closed pe1rrr closed 1 year ago

pe1rrr commented 1 year ago

I have been doing some testing with José after he changed major parts of VARA to run through in-memory code obfuscation and anti-piracy systems.

This has added overhead which causes VARA to fail consistently when running on RPI4, causing ALSA underruns when used in a configuration sharing the audio devices with other TNCs such as ARDOP and packet, so just a heads-up: do not to upgrade to 4.7.3 if using dsnoop/dmix or relying on VARA working under linux.

4.7.2 is the last version without these (annoying) measures added by José (EA5HVK).

SpudGunMan commented 1 year ago

sheesh, if greed ruins this I hope FCC gets this app off the air - the https://github.com/Rhizomatica/mercury is a possible better solution some day

SpudGunMan commented 1 year ago

I just tested on a new build and with a digirig I see a waterfall!!! so .. you sure its dead? I haven't tried fully yet but will test more here ASAP -- ARDOP has its own issues.

pe1rrr commented 1 year ago

I just tested on a new build and with a digirig I see a waterfall!!! so .. you sure its dead? I haven't tried fully yet but will test more here ASAP -- ARDOP has its own issues.

Yes, I didn't mention that it is the link negotiation that fails when it attempts to connect to any other VARA system. I have done extensive testing with José and him releasing this version knowing it causes issues for us is a smack in the face.

I believe I have fixed the ARDOP issues, which is addressed on the ARDOP groups.io list.

This is a completely working setup: https://eindhoven.space/radio-experiments/packet-radio/vara-ardop/raspberry-pi4-trifecta-tnc-bank-image-companion-documentation/

SpudGunMan commented 1 year ago

Better to share the fix vs the binary but cool project.

pe1rrr commented 1 year ago

Better to share the fix vs the binary but cool project.

The fix is not in the binary (this statement confused me), it is a series of operating system adjustments that provide the solution. That document is literally it.

https://ardop.groups.io/g/users/topic/ardop_not_working_with_linux/95790391

SpudGunMan commented 1 year ago

I am confused ardop isnt part of this project and you said VARA is broken but you fixed it but not to update. links to projects without answers, it sounds like your suggesting this project is dead and you have a solution but its for ARDOP?

pe1rrr commented 1 year ago

I am confused ardop isnt part of this project and you said VARA is broken but you fixed it but not to update. links to projects without answers, it sounds like your suggesting this project is dead and you have a solution but its for ARDOP?

Urgh. Ok, crossed wires here-

I mentioned ARDOP because in my particular scenario, the audio interface is using a dsnoop/dmix setup in order to share the interface with VARA, Packet and of course ARDOP at the same time. This induces some small latency but has been working perfectly fine up until version 4.7.3 of VARA which has added realtime memory encryption/obfuscation which taxes the RPI4 too much in my setup-

I am not sure if VARA connection establishment is broken when accessing the audio interface directly/priority as this is something you will need to test for in Winelink.

The thing to look out for in VARA is stuttering audio in the 88bps and higher mode after the MFSK handshake has finished.

I forgot I had mentioned ARDOP in subsequent posts and saw your remark about 'ARDOP has its own issues' as an opportunity to respond that there is a workaround for ARDOP's known issue with newer Linux kernels (I assumed you referred to this because there aren't many other issues I am aware of). Ironically, the extra latency added to the audio interface through dsnoop/dmix mixing is what lets ARDOP work in 5.15 kernel builds.

Hope this helps.

SpudGunMan commented 1 year ago

got ya. I was able to connect with FM .. so it sounds like your testing reflects a dead project. this is sad news for sure.

have you mentioned this to the groups.io vara, I am very upset to hear this is dead because of shareware greed

SpudGunMan commented 1 year ago

I shot a dart out on groups for vara

SpudGunMan commented 1 year ago

I also shot a note to varac team

WheezyE commented 1 year ago

I have been doing some testing with José after he changed major parts of VARA to run through in-memory code obfuscation and anti-piracy systems.

This has added overhead which causes VARA to fail consistently when running on RPI4, causing ALSA underruns when used in a configuration sharing the audio devices with other TNCs such as ARDOP and packet, so just a heads-up: do not to upgrade to 4.7.3 if using dsnoop/dmix or relying on VARA working under linux.

4.7.2 is the last version without these (annoying) measures added by José (EA5HVK).

That is bad news. I’ll have to test the new version and see if things are totally unusable. On one hand, I’m not surprised - VARA was never intended for Linux in the first place and has a steep price tag for what it provides. On the other hand, maybe this will help wake up the Winlink community to not want to rely so much on pay-for services. I hope one day a newer generation will bring the hobby back to its open-source community roots.

Maybe this will inspire someone to write an open-source mode similar to VARA and push the field forward.

Thank you for the heads up!

SpudGunMan commented 1 year ago

88k limit is still acceptable for winlink emcomm

What I did experience is some possible glitching and lock ups. Around the inspection time for license. Not always but

I personally am very frustrated as I was working on some high speed experimental stuff. This stinks.

But 88k isn't horrible for sailmail(winlink). If stability can be corrected but on top of varac stability ooof. All for obfuscation and prevention of theft from dishonest? Lamenting sigh of pain.

Project mercury uses similar coding.

Yes thanks for raising the alarm

WheezyE commented 1 year ago

sheesh, if greed ruins this I hope FCC gets this app off the air - the https://github.com/Rhizomatica/mercury is a possible better solution some day

PACTOR is pay-for and has been on the air for a long time, but I appreciate the sentiment. I wish this communication hobby was more financially accessible so more people could communicate.

That Mercury project looks nice. I wish them the best.

SpudGunMan commented 1 year ago

Heard back the offending code may have been removed.

WheezyE commented 1 year ago

Interesting. The current version is still 4.7.3. When the new version comes out, @pe1rrr would you be able to check to see if you're finding the same slow-downs on RPi4?

In the meantime, we might all want to download these old VARA's while they're still online and hold onto the files: VARA HF v4.7.2 (High Performance HF Modem) VARA FM v4.2.8 (VARA for FM transceivers) VARA SAT v4.3.6 (VARA for QO-100 geostationary SAT) VARA Chat v1.3.4 (Easy Text and File transfer chat) VARA Terminal v1.2.0 (VARA dumb terminal for BBS’s) Official links found on internet archive - internet archive doesn't archive mega links though so download while you can

SpudGunMan commented 1 year ago

Yea it was, i am going to flash a pi today, license and test for lockup.

I was testing on free yesterday which didn't help matters much either.

WheezyE commented 1 year ago

Heard back the offending code may have been removed.

Oh great! I see your post on the vara groups.io. Just posting here to have everything in one place:

I did hear back from Jose on the matter and it seems a positive direction for the concerns and I do hope the mixed bag of user group continues to do wacky things with ham radio. We are all learning and thanks for tolerating. Sorry for the disruption. The response was..

That changes were discarded.

WheezyE commented 1 year ago

Also just for clarity: I was able to finally test-run VARA 4.7.3 today on my Pi4 (I'm still too lazy to get my general, so can't test OTA connections yet).

Running VARA 4.7.3 on RPi4 (through box86/wine), I can open the program, input my VARA license code, and still run the program without lockups. Like pe1rrr has already mentioned, I guess this new security code CPU overhead would create lockups when the system was being taxed, like while making connections / transferring data / talking to other TNC's running on the Pi4 at the same time.

SpudGunMan commented 1 year ago

something you and the XYL can get in a weekend! Technician and upgrade general!! New ham! Also don't forget just using sound card to sound card or speaker/mic combos.

Yes I suspect the BPQ use case (dual modems) is solved with the solutions provided and pe1rrr laid out.

(I would also be interested if wine/ardop.exe in this project would remedy some issues with ALSA?)

pe1rrr commented 1 year ago

This is a recording of the problem so you can recognise it:

https://youtu.be/xmjZ2t5fxE0

Please ignore the fact this screen shows 5 x RPI4 systems, the one RPI4 being recorded is a console server, the other 4 systems- one is EI2GYB in Ireland and the other three are my 10m/20m/40m systems.

The problem occurred only in VARA 4.7.3 (tainted) even with no other TNCs running, the other TNCs (Packet and ARDOP) did not impact it because VARA runs on a single core as do the others on their own core in the example shown. VARA 4.7.2 of course runs perfectly well with all of the TNCs running simultaneously.

However with initial 4.7.3 tests (and then subsequent tests with combinations of new files sent to me by José that decremented the obfuscation system bit by bit), the single core CPU usage of VARA increased from an average of 46% to 65% during an attempted VARA session.

Expanding on the bit about the extra files José sent: The binary and DAT files of 4.7.3 are incompatible with 4.7.2, so in order to test how far back he must roll the changes, I was sent new DAT and EXE files to modify my 4.7.2 installations incrementally.

Unfortunately, problems persisted regardless of how little of the obfuscation encryption was enabled (it is variable, apparently).

We got to the point where José sent me files with encryption disabled completely, and the problem persisted even still(!)

This indicates that something has changed but I don't know what because I can't physically see/audit José's changes. It could be his compiler, it could be the use of newer system libraries, it could be poor versioning control and he has forgotten to revert some code crucial to the build process. Who knows?!

It's impossible to tell.


Red, 

What i see is a chaos of delay. I have only added extra bytes to the files to v4.7.2, which are ignored in this version, so there is no any change. 

I suspect there are delay problems in your wine configuration and sometimes works and sometimes not. Impossible to get conclussions.

José 

The thing is, the 4.7.2 original installs worked sufficiently well with occasional problems that went away after VARA had been left running a little while (not stuttering audio, but failure to link on a first attempt after start up).

On the other hand, 4.7.3 issues cropped up and broke immediately off the bat, so to speak.

@WheezyE I will get around to testing the untainted version tonight.

Edit: Grammar and a bit of extra info.

Edit2: https://www.twitch.tv/videos/1740491204 (this will get auto deleted in a few days so I will try and archive it) - a longer test from last week showing the differences between VARA versions. Its a long boring video.

pe1rrr commented 1 year ago

@WheezyE @SpudGunMan I've updated the previous comment with additional info.

SpudGunMan commented 1 year ago

perfmon here is just opening the applications no use of app

SpudGunMan commented 1 year ago

The thing is, the 4.7.2 original installs worked sufficiently well with occasional problems that went away after VARA had been left running a little while (not stuttering audio, but failure to link on a first attempt after start up).

On the other hand, 4.7.3 issues cropped up and broke immediately off the bat, so to speak.

I share these observations. 4.7.3 is not ideal, I did respond to the email from Jose commenting this is not working as he suggested it should be.

WheezyE commented 1 year ago

Thank you @pe1rrr for the videos and explanations!

SpudGunMan commented 1 year ago

I haven't tested with 4.7.4 yet assuming the patch I provided feedback on is the same. I have not tested yet myself got side-tracked. anyone else?

pe1rrr commented 1 year ago

Hi @WheezyE @SpudGunMan

I have done extensive new tests fo VARA 4.7.4 and am happy to report that the functionality is fully restored to the same performance with 4.7.2. The ChangeLog doesn't mention anything but I presume this release has been gutted of the troublesome code.

Phew!

Here is a twitch stream I will archive later on YouTube of the entire process including installing on multiple systems and doing long distance RF as well as local (fancy pants BBS forwarding).

https://www.twitch.tv/videos/1752369892

OH2LAK commented 1 year ago

Exactly what has FCC to do with VARA-modem?

.. Erik OH2LAK TF3EY

On Thu 23. Feb 2023 at 1.44, Kelly @.***> wrote:

sheesh, if greed ruins this I hope FCC gets this app off the air - the https://github.com/Rhizomatica/mercury is a possible better solution some day

— Reply to this email directly, view it on GitHub https://github.com/WheezyE/Winelink/issues/66#issuecomment-1441010600, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEJADTG2NUKZS7LCZHDPSTWY2QFDANCNFSM6AAAAAAVE6PJAE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

SpudGunMan commented 1 year ago

There is a rule in laws in usa which have archaic language terms which complicate matters.

Bits baud bandwidth. Three B's needing clarification. Currently under submission for review.

All of which is out of scope for much banter on git but happy to pontificate pointlessly elsewhere!

de k7mhi via Qrz

http://www.arrl.org/files/file/QST/This%20Month%20in%20QST/September%202013/ItSeemsToUs.pdf

WheezyE commented 1 year ago

I have done extensive new tests fo VARA 4.7.4 and am happy to report that the functionality is fully restored to the same performance with 4.7.2. The ChangeLog doesn't mention anything but I presume this release has been gutted of the troublesome code.

Phew!

Really great work @pe1rrr ! I think it would be fair to say that you helped save VARA functionality on Linux. Thank you!

EDIT: Replying on my phone, I accidentally replied without comment and also closed this issue on accident. I think the issue can be close now though

WheezyE commented 1 year ago

Exactly what has FCC to do with VARA-modem? .. Erik OH2LAK TF3EY

That’s a good question since some people on forums have been frustratedly saying the FCC should ban VARA. As far as I know, the FCC has no control at all over VARA as software. They wouldn’t need to regulate any part of it, especially not any anti piracy features. If VARA was programmed to work on a frequency or at a speed that was not allowed in the USA, I believe that would be the radio operator’s fault for using the modem in a way that didn’t comply with FCC.

kyhwana commented 1 year ago

Exactly what has FCC to do with VARA-modem? .. Erik OH2LAK TF3EY

That’s a good question since some people on forums have been frustratedly saying the FCC should ban VARA. As far as I know, the FCC has no control at all over VARA as software. They wouldn’t need to regulate any part of it, especially not any anti piracy features. If VARA was programmed to work on a frequency or at a speed that was not allowed in the USA, I believe that would be the radio operator’s fault for using the modem in a way that didn’t comply with FCC.

VARA is a proprietary undocumented closed source mode, as such it's "obfuscation" and doesn't belong on the ham bands.

OH2LAK commented 1 year ago

So then PACTOR is OK to use on ham bands, being even more proprietary and $$$?

VARA is a rather nice modem and fits very well to ham bands. It is solely its developers decision to keep it closed-source. He is not making a lot of money with it with the one-time registration fee which includes free updates. It is a rather modest cost to support the developer in my opinion. How Jose has done the programming with inclusions of anti-piracy code and some sort of obfuscation is his choice and right to protect his copyright to the code. It is not doing any obfuscation on the air interface.

What comes to FCC and the bitrate/symbol/speed limits, every ham in the US should talk with their representatives that those limits should be lifted, they are not part of modern communication regulation anymore. Technology has made a huge leap in the last 20 years and limiting the data modes to sub-telephone line modem-era is no way reasonable or needed anymore.

What I have missed in this thread about VARA-HF not being compatible anymore with Wine or other windows emulators is that nobody seemed to contact Jose directly to present the issue to him? Few rumours through third parties yes but this project is already rather big so why not talk directly with the 'vendors' as well? Irad who makes VarAC has done a magnificent job by adding 'Linux compatibility' to the settings of VarAC.

73, Erik OH2LAK TF3EY

.. Erik OH2LAK TF3EY

On Wed, 1 Mar 2023 at 22:49, Kyhwana Pardus @.***> wrote:

Exactly what has FCC to do with VARA-modem? .. Erik OH2LAK TF3EY … <#m-6083191639177081969>

That’s a good question since some people on forums have been frustratedly saying the FCC should ban VARA. As far as I know, the FCC has no control at all over VARA as software. They wouldn’t need to regulate any part of it, especially not any anti piracy features. If VARA was programmed to work on a frequency or at a speed that was not allowed in the USA, I believe that would be the radio operator’s fault for using the modem in a way that didn’t comply with FCC.

VARA is a proprietary undocumented closed source mode, as such it's "obfuscation" and doesn't belong on the ham bands.

— Reply to this email directly, view it on GitHub https://github.com/WheezyE/Winelink/issues/66#issuecomment-1450825257, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEJADTWH6XPDNFMKFOJ46LWZ6Y63ANCNFSM6AAAAAAVE6PJAE . You are receiving this because you commented.Message ID: @.***>

kyhwana commented 1 year ago

So then PACTOR is OK to use on ham bands, being even more proprietary and $$$? No, PACTOR isn't OK either.

"Anti-piracy" code for a close sourced, undocumented protocol is an anathema to what ham radio is, or should be about.

The FCC 300 baud limit is strictly a US ham concern, and not concern of mine as a ZL ham.

If you read earlier comments you'll see that it's implied that someone already emailed him. Of course this wouldn't have been necessary if the vara modem was open source. "Anti-piracy" features wouldn't be breaking non-windows support of a modem used on the ham bands.

pe1rrr commented 1 year ago

What I have missed in this thread about VARA-HF not being compatible anymore with Wine or other windows emulators is that nobody seemed to contact Jose directly to present the issue to him? Few rumours through third parties yes but this project is already rather big so why not talk directly with the 'vendors' as well? Irad who makes VarAC has done a magnificent job by adding 'Linux compatibility' to the settings of VarAC.

73, Erik OH2LAK TF3EY

Read the very first line in this issue post. I worked directly with José on this.

OH2LAK commented 1 year ago

Sorry, I missed that. I only saw people posting to other VARA-groups and such to get attention to this issue.

Great work and thanks for the time & knowledge put into this project!

73, Erik OH2LAK TF3EY

On Fri, 3 Mar 2023 at 09:53, #PacketCat Red PE1RRR @.***> wrote:

What I have missed in this thread about VARA-HF not being compatible anymore with Wine or other windows emulators is that nobody seemed to contact Jose directly to present the issue to him? Few rumours through third parties yes but this project is already rather big so why not talk directly with the 'vendors' as well? Irad who makes VarAC has done a magnificent job by adding 'Linux compatibility' to the settings of VarAC.

73, Erik OH2LAK TF3EY

Read the very first line in this issue post. I worked directly with José on this.

— Reply to this email directly, view it on GitHub https://github.com/WheezyE/Winelink/issues/66#issuecomment-1453117720, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEJADUSBOL2ICIK5QWQQRTW2GPRLANCNFSM6AAAAAAVE6PJAE . You are receiving this because you commented.Message ID: @.***>