markjfine / nrsc5-dui

An enhanced, user-friendly version of nrsc5-gui that is not heavily dependent upon Python processing for audio generation.
GNU General Public License v3.0
139 stars 9 forks source link

Windows Support Thread #9

Closed andrewfer000 closed 1 year ago

andrewfer000 commented 3 years ago

Creating this as a thread for RTL-SDR Users on Microsoft Windows Operating Systems. Since neither myself or Mark use Windows, we will be relying on other users to test and give issues. Based on what I know, you just need to install python 3.8 or 3.9 and install the required modules from Pip, then install nrsc5 for Windows and try and run the application. You might have to change the /usr/local/bin/nrsc5 hard-coded path to the nrsc5.exe file installed on Windows or the one in the nrsc5-dui folder. We could really use some Windows testers, so if you are using that OS and are skilled enough with how python works then feel free to help out!

andrewfer000 commented 3 years ago

I'm working in a Windows VM as well to try and get results. It might take a few days since Windows and Python don't play together as nice as you may think.

markjfine commented 3 years ago

FYI: I added an argument when calling nrsc5-dui.py that lets you specify the path to nrsc5(.exe). Needs a final slash, but I think I should make it add it if it's not there.

Syntax is: python3 nrsc5-dui.py <path to nrsc5.exe>

andrewfer000 commented 3 years ago

So it turns out right now due to the dependency on termios (and maybe tty as well) this program will only work on Linux, macOS, and Maybe on Windows with WSL.

comat0se commented 3 years ago

I was able to the get GUI up and running under WSL1 Ubuntu 20.04, however I think there's a problem with nrsc5 on WSL. I initially tried getting this going using python 3.9 but eventually ran into an error importing 'gi'. It was probably some sort of version mismatch, so I installed everything using python 3.8 and then I'm able to pull up the GUI, but the audio decode doesn't do anything. So I tried nrsc5 standalone from the command line. I get: "shared memfd open() failed: Function not implemented" which seems like it just might not work under WSL1. It seems that WSL2 doesn't support USB devices, so... maybe use rtl_tcp?? I haven't tried WSL2 yet, but perhaps that's the next step.

I was able to get nrsc5.exe to run under MSYS2... so if there's a way to get nrsc5-DUI running in MSYS2, that potentially might be another way. I started that path but it became difficult quickly.

comat0se commented 3 years ago

A follow up... maybe the issue isn't the memfd error but rather getting pulseaudio working within WSL. There's a few webpages talking about how to set it up, and I have it halfway working using Pulseaudio v1.1. but it seems that largely I can't get it to find my audiocard currently. I can test it using pacat < /dev/urandom and I hear things, but when I type aplay -l I get no soundcard... if I run nrsc5, I get Open Device failed. So maybe the real monster here is pulseaudio on WSL.

markjfine commented 3 years ago

'Open Device Failed' is usually the message nrsc5 generates when it can't open a usb channel to the radio. That may not necessarily mean the usb is bad, just that the radio is non-responsive, so can also mean the radio is jammed up. That happens when it gets confused. Reconnecting it usually fixes that.

markjfine commented 3 years ago

Having said all that, if you can get nrsc5 running from a command window, and generate status messages without nrsc5-dui or even audio, then you're most of the way there. Once you get that, then get the audio operational, then nrsc5-dui.

comat0se commented 3 years ago

Ok, that was a helpful comment. I don't think WSL sees the USB rtl-sdr, however it can communicate via rtl_tcp. I was able to get nrsc5 working from the command line... pulseaudio server running windows-side. I got all of the normal status messages but the audio was slow and severely stuttered. Perhaps a result of pulseaudio being version 1.1. I can't seem to figure out how to use rtl_tcp from nrsc5_dui though...

markjfine commented 3 years ago

rtl_tcp has to be running in the background and listening for a specific IP address, for example: rtl_tcp -a 127.0.0.1

In nrsc5-dui's settings, there should be the RTL_TCP IP. Enter the IP you set using the -a switch in that box, then click the Enabled button to the right of that. The next time nrsc5 is spawned by nrsc5-dui, it will command nrsc5 to use the radio found at that IP (using the default port of 1234).

comat0se commented 3 years ago

Success, kind of. I edited the config.json file and input 127.0.0.1 as DevIP and UseIP=true. This successfully connects to rtl_tcp and DUI starts getting the stream. However, it's plagued by high BER of up to 27% and when it is able to decode audio it's extremely stuttered and slow. The signal is quite strong and shouldn't have trouble decoding, but my computer is fairly old. I will say this is possible, but it was a struggle to get it this far. I may try again with WSL2 to see if it has any improvement.

markjfine commented 3 years ago

Heh... that's cheating, but those two things are basically the equivalent of what's in Settings.

comat0se commented 3 years ago

Yea, I was blind to that button for some reason...

signals42 commented 3 years ago

Success on WSL2 (Ubuntu)... Sort of.

This needs some cleanup and a lot of documenting.. And I'm still having some difficulty keeping it stable, but I've now heard multiple alternate digital channels, and seen album artwork and weather/traffic maps. The GUI can tune the radio, and it works great until one of the many, many hacks and background cheats I'm using fails.

I have the Windows version of rtl_tcp.exe running on the WSL2 network address of the windows host in one window. You have to set the (WSL2 network 172.*) IP address of the Windows host in the cfg/config.json file to make nrsc5-dui use it, but it seems to work just fine.

I am running PulseAudio for Windows in another window and listening for a TCP stream on the same WSL2 interface. I have the WSL2 Ubuntu instance configured to forward audio to it over the network.

I was unable to get nrsc5 to play audio across the network back to Windows without significant choppiness. I think this may have to do with sample rate, and a lack of resampling, but was unable to fix it. So, I created a fifo on /tmp/stream and have mplayer running in the background playing it through PulseAudio back to Windows. This takes care of the audio quality issues I had. Then I modified nrsc5-dui.py to add "-o /tmp/stream" to the nrsc5 command line when it calls the program.

I also installed VcXsrv and did all of the network forwarding work to get a GUI from WSL2 up in it.

So now, assuming I have VcXsrv, Pulse, and rtl_tcp.exe running on Windows, and I have an "mplayer -cache 2048 /tmp/stream" running in WSL2, I can just call "nrsc5-dui.py" from my WSL2 Ubuntu instance and the window opens up on the desktop, it uses the local RTL-SDR I have on this Windows machine, and everything works!

I'm going to try to clean this process up a bit in the coming days, and will attempt to do a better job of documenting it, but I figured I should let everyone know that it is possible to get this working well enough to use, and let you know the ingredients, even if I haven't written up a recipe yet. nrsc5-dui-windows

markjfine commented 3 years ago

Interesting solution... wondering if anyone's gone and rebuilt the nrsc5 and sox packages under mingw to make it all more native rather than go through some *nix hoops.

Not sure where 105.7 is, but was testing a little on a Philly station (106.1 WISX) that also has a smooth jazz stream and at 9:37 ET played this: Paul_Hardcastle_-_Welcome_To_The_Beach

signals42 commented 3 years ago

West Michigan. Apparently there's not enough of a market to populate the sub-channels with bespoke content. Looks like this is simulcast on iHeart Media stations everywhere.

I actually thought about trying mingw when I hit about the 6 hour mark on this. I was already pot-committed on WSL2, though. I hadn't actually used WSL2 before now, so it was a good learning exercise. I'm not sure what the USB device support looks like with mingw, but even if you still have to use rtl_tcp, eliminating all of the pulseaudio/X11/mplayer stuff would be great.

Although, Microsoft is promising a native solution to the WSL2 GUI/Audio problem too. So, hopefully, it will just run without issue after a Windows update... Eventually.

markjfine commented 3 years ago

There may be more clues on effie's site, since he started making this more Win-compatible and I believe that's where the dlls came from.

Edit: Zefie... not effie.🤦🏻‍♂️

pib commented 3 years ago

I tried to get it up and running properly on Windows. I got nrsc5 itself built using MSYS2 as documented, and I got all the dependencies installed using conda, which included building musicbrainzngs and getting sox from someone's random build.

The one remaining issue (so far 😬) is that Python for Windows doesn't have the "termios" module, which is required by tty, which is required by the pty module. Recent versions of Windows actually have PTY support built in, and there's a library that takes advantage of that: https://github.com/spyder-ide/pywinpty

Unfortunately the interface is different so it would replace both the pty and the subprocess modules. Wrapping pty/subprocess and pywinpty in classes with the same interface would probably be a pretty straightforward way of supporting both.

I suppose using https://github.com/theori-io/nrsc5/blob/master/support/nrsc5.py to call libnrsc5 directly would maybe be a better solution overall, though. They even have what seems to be a pretty similar implementation of the main nrsc5 in Python https://github.com/theori-io/nrsc5/blob/master/support/cli.py

markjfine commented 3 years ago

Unfortunately, Python really doesn't have the horsepower to run using the lib and do any other processing, including processing the audio, which was really problematic. Kind of how we got to letting nrsc5(.exe) do what it does, and letting the Python frontend just interpret the messages being sent to it.

andrewfer000 commented 3 years ago

The whole reason this fork came into existence is because of the problems that nrsc5.py caused in the old nrsc5-gui lol. I tested nrsc5's cli.py in the past and it had many audio issues and slowdowns.

DarkAlchy commented 3 years ago

I tried to get it up and running properly on Windows. I got nrsc5 itself built using MSYS2 as documented, and I got all the dependencies installed using conda, which included building musicbrainzngs and getting sox from someone's random build.

The one remaining issue (so far 😬) is that Python for Windows doesn't have the "termios" module, which is required by tty, which is required by the pty module. Recent versions of Windows actually have PTY support built in, and there's a library that takes advantage of that: https://github.com/spyder-ide/pywinpty

Unfortunately the interface is different so it would replace both the pty and the subprocess modules. Wrapping pty/subprocess and pywinpty in classes with the same interface would probably be a pretty straightforward way of supporting both.

I suppose using https://github.com/theori-io/nrsc5/blob/master/support/nrsc5.py to call libnrsc5 directly would maybe be a better solution overall, though. They even have what seems to be a pretty similar implementation of the main nrsc5 in Python https://github.com/theori-io/nrsc5/blob/master/support/cli.py

You got further along than I did. I got all the way to this point

Windows Go to http://www.msys2.org/ and download the x86_64 installer Follow the instructions on the page for setting up the basic environment Run C:\msys64\mingw64.exe - a terminal window should pop up Execute pacman -Suy Execute pacman -S mingw-w64-x86_64-gtk3 mingw-w64-x86_64-python3 mingw-w64-x86_64-python3-gobject To test that GTK 3 is working you can run gtk3-demo

Command not found. No other issues so I thought it might be a path issue, but when I went looking I could not find anything about that, or what to add.

markjfine commented 3 years ago

Seems a lot of people are having a lot of trouble with this. I was under the impression that zefie's build worked in Windows. Maybe I was mistaken.

I grabbed MSYS2 and am going to give some of this a shot in my Win10 VM before I roll this up into a release. Hopefully I can find a solution, perhaps build-in Win10 alternatives to the termios problem as well.

Just wondering if anyone has run setup.py to set the Win10 environment, or if it even works...

markjfine commented 3 years ago

Got nrsc5.exe built under MSYS2's 64-bit environment using the instructions on theori-io/nrsc5. I have to say this is pretty involved. It ran the sample stream just fine, producing audio and everything

It kind of runs using the receiver, which is to say that it tries to do what it's supposed to do. However, it doesn't want to keep synch let alone decode a station that I know happens to be pretty strong in this area. BER is fine, but MER is dismal, so something ain't quite right.

Or maybe not... you have to move the decimal over two places on the BER status messages. 🤦🏻‍♂️

markjfine commented 3 years ago

SDR# works fine in the Win10 VM. No issues with USB. But the MSYS2 build of nrsc5 is showing a BER that's about 100+ times what it should be. Very strange behavior.

Going to try cross-compiling from macOS and see if that version works any better.

markjfine commented 3 years ago

Cross-compiling generated the same ugly results. Then I tried capturing an IQ stream using rtl_sdr.exe and playing it back through nrsc5.exe... same ugly results. Should point out that none of this happens on the Mac side, just in the Windows VM.

So I took a cue from @comat0se and tried using rtl_tcp on the Mac side, with the IP address shown in System Preferences. On the Windows side, I ran nrsc5 -H <mac IP address> <frequency in MHz> <stream#> (just like nrsc5-dui would call it) and it worked like a charm. Worthwhile to note that the VM uses a Bridged Network using the default adapter and I haven't tried the other settings (Host-only or Shared).

So the first part of the answer looks to be that the processing in librtlsdr.dll is a majority of the problem here. So there's that. Next I'll tackle the Python part.

Edit: I realize that doesn't really solve the problem for Win-only. I'll try to work that too in time.

markjfine commented 3 years ago

Got Python and most of the dependencies installed outside of MSYS2. Of course that one outstanding dependency would happen to be pygobject. It's trying to build something that probably doesn't need to be built and isn't recognizing the version of C++ Build Tools. Had already pulled down and built gsvbuild from github, which in theory should have been the Win10 gtk+ stack.

It's been at least two decades since I did any substantive Windows development and the PTSD is very deep.

markjfine commented 3 years ago

Been diverted to other things the past couple of days. But have been thinking about this.

First, I'm bothered that there appears to be no way to make this work natively in Win10. Three things prevent this:

  1. RTL_SDR code doesn't work right at all, inhibiting signal detection.
  2. PyGObject won't build right unless you use very old compilers.
  3. Lastly, Win10 is not Posix compliant, so there is no good pty solution without going through hoops to spawn and control nrsc5.

This leaves the solution everyone's been using: Running under MSYS2 or an equivalent MingW environment. To me that's slightly problematic because there's two ways to do this: using things like pacman or pip to manage Python modules... and the two are not the same (I come to find out). And, it still doesn't solve the RTL_SDR reception/detection issues under Win10.

The impact to this release is that I will write some kind of disclaimer and some minimal recommendations in the Readme of how to possibly get this to work, using what some of you have done as an example.

Lesson learned is that it's likely what zefie had done may have only partially worked to begin with, or I broke it (which is very possible). I guess that's be expected when dealing with other people's code on Github and Sourceforge.

DarkAlchy commented 3 years ago

Oh, well, at least you tried. For me I will delete this as I had Linux for a couple of years, but I did not like it.

Thanks again for trying, but you can only do what you can do and Windows 10 being the bitch that it is says no you can't.

pib commented 3 years ago

Not that it helps the other problems, but installing PyGObject via conda (https://anaconda.org/conda-forge/pygobject) was pretty easy and didn't involve actually compiling it myself. Miniconda seems to be a pretty decent starting point for someone without a Python install on Windows.

I used MSYS2 to build nrsc5 and miniconda for everything else. I had to build a couple of missing dependencies, but it was just a matter of running conda skeleton pypi ... and conda build ... (https://conda.io/projects/conda-build/en/latest/user-guide/tutorials/build-pkgs-skeleton.html)

Also, this may or may not help, but I haven't been able to get nrsc5 itself to work at all under Windows or Linux. I don't have a proper antenna for my rtl-sdr dongle yet so I'm not sure if the issue is that I am not getting a strong enough signal or if there's something wrong with my builds on both machines. I built from the latest on the master branch on both systems. nrsc5 gives little to no output:

(base) PS C:\msys64\home\pib\nrsc5\build\src> .\nrsc5.exe 93.3 0
Found Elonics E4000 tuner
Exact sample rate is: 1488375.071248 Hz
Disabled direct sampling mode

That's all it ever prints out. Trying to use rtl_tcp doesn't help either (I can see from the rtl_tcp output that it sets the gain up to "420" and then doesn't do anything else). gqrx and SDR# are both able to pick up that FM station even with the stock ezcap antenna.

pib commented 3 years ago

Oh, I should also mention that I am able to play back the sample file with nrsc5 on both Windows and Linux, so it definitely seems to true that it's something with the rtlsdr part.

markjfine commented 3 years ago

I used MSYS2 and even cross-compiled my Darwin (Mac) build of nrsc5. It runs, but for some reason doesn't calculate the BER of the signal on something that should be near-perfect. So your problem wasn't likely having enough signal... there's something else at play.

The only way I was able to get nrsc5 to work even semi-correctly was to have the receiver run using rtl-tcp on the Mac side, and have the Windows side run nrsc5 with the -H switch using the Mac's IP address.

That's 'workable', but not optimum for someone just wanting to run it on Win10 stand-alone. Also seems like way too many hoops to jump through for the average person that might see this as a novelty... just trying to be realistic.

Also not totally giving up. I'm just not going to spend more time on it for the purposes of this release. Eventually I'll find a solution.

DarkAlchy commented 3 years ago

I am picking up on a vibe here but I have an issue with my rtl-sdr and figured it must be broken, but now I am not so sure. With W10 I can't scan for frequency as I did previously. If I scan it stays stuck on the same frequency (window on the screen moves/changes frequency but the dongle stays at its current frequency), but if I stop then restart the dongle will be where it is supposed to be.

I bring the above up because it sounds like this is something happening with the driver being so old?

markjfine commented 3 years ago

Not sure what you mean by 'driver'... libusb? librtlsdr?

I would think the usb driver is fine. In fact I was just using whatever worked for SDR# and didn't even mess with that.

Librtlsdr is a different story and has nothing to do with age. Same code runs correctly on Mac and Linux, just not on Windows.

DarkAlchy commented 3 years ago

Well, I find it ironic that suddenly SDR# will not change the frequency (in real time) on my dongle but if I stop it and restart it (the play like button) the frequency does change. To get the rtl-sdr to run in SDR# you have to have a driver installed from Zadig. Not sure where the issue is but I find it weird that if the dongle broke why does it work when starting SDR#? 100.0Mhz. Start SDR# and there it is. Scroll to 108.00MHZ and the window scrolls, but the dongle is stuck on 100.0Mhz. Hit the stop button on SDR# then hit the play button and my dongle is at 108.00Mhz.

See what I am talking about? It updates but when I scroll something is not updating the dongle as it once did only when it first is ran (play).

markjfine commented 3 years ago

Ah, ok. The thing zadig does is fine. But I noticed SDR# did some weird things after trying to run nrsc5 direct (meaning without rtl_tcp). More than likely the version of rtlsdr that it's trying to use conflicts with the way SDR# operates. I'll bet if you restart Win10 and not use nrsc5, SDR# will begin to work correctly again.

In fact, I recall SDR# crashed on me a couple of times when trying to tune with it after using nrsc5.

DarkAlchy commented 3 years ago

You would lose that bet. I have had this issue for over a year but I can't find any old versions of SDR# to try.

nrsc5 I only tried recently when I posted here so the two may be connected but it isn't the cause.

DarkAlchy commented 3 years ago

Well, I was right. I went looking to see if there was another program for my SDR but I found nothing except a web page from 2014. On that webpage was a mirror of sdrsharp-5-19-2014.zip. I downloaded it, then I unzipped it. I hit run and everything ran fine. Something they did does not work, so I wonder if something they changed, or added, may be linked to the same thing that made this not work?

At least I was right and my dongle is not to blame and works perfectly just not on the newer versions (means Zadig is fine since it is the same version as before).

markjfine commented 3 years ago

You may have been right about your particular dongle having an issue with the current version of SDR#.

I'm talking about a conflict between a Windows-built rtl_sdr library and likely anything that tries to communicate with an RTL_SDR V3 dongle, which also exists. I've had to reset my dongle and Windows several times in order to get things back to normal after attempting to run nrsc5 with anything other than via rtl_tcp.

This appears to be completely separate from your issue, which you never said predated this project by more than a year. So I apologize if I tried to link the two thinking this was recently discovered when trying to run this project. That is what this thread is meant to discuss.

markjfine commented 3 years ago

Ok, so I've updated the README. Would appreciate it if I can get a second set of eyes on it to make sure I've accurately captured the pertinent parts of others' experiences.

DarkAlchy commented 3 years ago

You may have been right about your particular dongle having an issue with the current version of SDR#.

I'm talking about a conflict between a Windows-built rtl_sdr library and likely anything that tries to communicate with an RTL_SDR V3 dongle, which also exists. I've had to reset my dongle and Windows several times in order to get things back to normal after attempting to run nrsc5 with anything other than via rtl_tcp.

This appears to be completely separate from your issue, which you never said predated this project by more than a year. So I apologize if I tried to link the two thinking this was recently discovered when trying to run this project. That is what this thread is meant to discuss.

Then there may be the problem it very well could be SDR# is now being built with the same rtl_sdr library you used and that is the problem child? Oh, well it is just a thought.

Good luck.

markjfine commented 3 years ago

Again, my SDR# worked well until I ran nrsc5 in Windows, so no.

DarkAlchy commented 3 years ago

Well, didn't work on mine no matter what I used until I fell back to an earlier build. I guess it is a big fuck it on Windows for this, or a new version of SDR#. Not really worth all the hassle for this as one of the previous comments said.

As I previously said thanks for trying but there is something here that you are missing, or overlooking when 1 doesn't work but did work on prior builds and your project doesn't work for shit on W10 (iow at all). Anything common between the two? Who knows.

markjfine commented 3 years ago

Rethinking the Windows-specific issues:

  1. RTL-SDR code: Wondering if the RTL-SDR thing isn't a 64-bit vs 32-bit thing. Will try to investigate the possibility of 32-bit.

  2. PyGobject: Will try to rebuild for standalone, but this may still be problematic. Edit: Another option is to go back to the original gtk and gobject modules (just for Windows), but that may produce deprecation warnings.

  3. Posix compliance: Got the idea to look at zefie's original code, which didn't include the pty requirement - something I added in order to make switching streams smoother. It occurred to me that I could feasibly bypass the master/slave functionality of pty for only Windows installations. It would make stream switching on Windows kind of clunky again (will have to stop/start rather than simply pipe a keypress to the nrsc5 process), but at least there would be a chance of it working.

Edit: I should specify that any changes I make would be Windows-only with zero impact to Linux/Mac.

pib commented 3 years ago

Not Windows-specific but I found the reason nrsc5 wasn't working for me: the direct sampling option. Even with direct-sampling turned off (the default), the call to specifically turn it off in the main function would cause my receiver to stop working altogether.

This is all it took to make it work for me:

    //if (nrsc5_set_direct_sampling(radio, st->direct_sampling) != 0)
    //{
    //    log_fatal("Set direct sampling failed.");
    //    return 1;
    //}

Doing that under Linux fixed nrsc5 for me. It didn't work on Windows still (but this time there was zero output so who knows what it's doing?), but it's possible I did something wrong in my rebuild.]

Edit: I realize this isn't specific to nrsc5-dui, but figured I'd put it here in case it helps someone else trying to get this working.

markjfine commented 3 years ago

Had some time so rebuilt nrsc5 and rtl_sdr under MSYS2 32 bit. No joy. Same issue.

Edit: Again, it's possible although not really probable that this could be due a conflict in the way my Win10 VM does USB.

markjfine commented 2 years ago

Based on some recent code changes to nrsc5 (albeit for NEON code, and unrelated to this), I decided to cross-compile the whole mess again for Windows under mingw-w64. After dragging all the rebuilt pieces over to Win11, it seemed to work! Still some stability issues (cb errors and finally some rtlsdr read errors), but I actually got audio out of it and it was reading keypresses now. That's more than I got the last time so maybe this isn't a complete lost cause after all.

markjfine commented 2 years ago

Quick update on this: Still some stability issues, however I think these are related to USB stability issues within the Windows VM I am using (Parallels). These are annoying, vis-a-vis nrsc5, but not show-stoppers.

TheRealBanana commented 1 year ago

I'm having trouble getting this to run under Windows 10 with the latest msys2. Not sure what I'm doing wrong but whenever I try to run nrsc5-dui.py I get the error: ModuleNotFoundError: No module named 'termios'. I've installed all the dependencies listed and double checked the instructions were followed correctly. Googling for solutions to missing termios hasn't been fruitful. I couldn't figure out how to install python 3.9 under MSYS2 so this is the version I have installed: mingw64/mingw-w64-x86_64-python 3.10.8-2. Is this the issue?

markjfine commented 1 year ago

Read the readme. There are several issues with trying to get this to work properly in Msys2. What you've run into is the fact that Windows isn't Posix compliant. IOW: There is no termios available for Windows, but it's required to spawn nrsc5 in a shell. I'm looking to replace it for Windows but it's not a simple fix.

TheRealBanana commented 1 year ago

I saw the readme but figured I could find a way. Cygwin supports termios but I had trouble with out of date glib, otherwise I think it would work there. I did eventually get it working using Windows Subsystem for Linux 2 and Ubuntu 22.04 LTS (20.04 had issues). I just compiled everything like I was under linux except I used the cross-compile option for nrsc5 and Im listening successfully. I also had to update the max length for the rtl_tcp ip address as it was too short for port numbers.

markjfine commented 1 year ago

d'oh... forgot the port numbers.