Testing Notes

[andrewfer000]

Thank you for doing this! The audio works very well.

My System- Ubuntu 20.04

I do have only one issue I see right now and it's this

Mar 27 17:22:02 : Error: Unable to load station logo database
Mar 27 17:22:02 : Error: Unable to load config
Mar 27 17:22:43 : Error: Unable to save config

Not sure if this is a WIP still but I cannot save favorites or save the station logos. The album art works fine though. Do you know any fixes? besides that it works awesome!

[andrewfer000]

Also in the map viewer separate window, when traffic is clicked it does not show the traffic map.

[markjfine]

Yeah, I've seen this before, The config is not 100% compatible with the last one. There's a procedure I used.:

1. Kill the app if it's running.
2. Delete the config.json it created, which is probably corrupted.
3. Run the app and then kill it to create a new config.json.
4. Copy just the guts of the bookmarks array (without the outer brackets) that you want to use from an old config.json into the outer brackets of the new, empty one - then save it.
5. Run the app again, it should be ok.

One of the major differences between these two are the naming conventions. One used underlines, the other did what I call Pascal variable notation, every word in caps... That was all over the place, including the config file that won't match at all.

[andrewfer000]

it does not create the files.

[markjfine]

I'll have to look at that. It does on mac... maybe you have to create a cfg directory underneath first (it's 'supposed' to do that for you, but...)

BTW: I thought about what was going on with the PyAudio one versus this one during a run today. The PyAudio one built essentially a whole new nrsc5 around the nrsc5 library, so it was dependent upon the process load and interpreter time to keep things moving. Not very efficient, and as you can see, it backfires in some instances. The new one just spawns nrsc5 and communicates with it via stdin/stout pipes - I will need to create a new stdin pipe to get it to change streams without restarting the whole synch process.

[andrewfer000]

nvm. I was not running it in python3. Once running it by using the python3 command it created the files.

I take that back - I created the files myself. I deleted them again and it did not work even in python3

[markjfine]

Yeah, that's actually the problem. I just noticed my cfg ws missing in Linux as well. I created the cfg, started it, and now the two files are there. When I get a chance, I'll add a cfg directory in the repo so it creates it on a clone.

[andrewfer000]

I fixed it. Just had to create the cfg directory.

[markjfine]

Yup. There's also a bug with album art showing after the song is over. I fixed it in the last interface, I just need to remember to add the code I used there on this one. There's a lot of things I fixed that need to be re-fixed. Lol

[andrewfer000]

It's not too bad in my testing. Once again thank your for doing this.

If it wasn't for this pandemic, I'd treat you to a coffee xD.

[markjfine]

Lol... cheers.

[markjfine]

As far as the map viewer... yeah, I noticed that yesterday. The radio buttons are dead. Should be a simple fix to synch the code and mapForm.glade.

[andrewfer000]

Oh yeah, forgot to mention that sub-channels logos do not work but it seems like their album artwork does.

Edit: nvm for some reason kinda works now.

I don't know what you have for this programs "TODO List" but I would like to know what you plan on fixing so I can see if I can do anything.

[markjfine]

There's a weird latency to the album art. It's downloading the images, just not displaying them properly. And you're really right about the logos. Only the first stream's logo displays automatically. The others seem to need to have a XHDR to trigger it.

[markjfine]

There were a bunch of problems the way the XHDR and LOT messages were being handled, the least of which mime strings being treated as either a 1 or 0 (the regex wasn't picking up the right thing). Was another part where images were being downloaded, but not cached as the stream logo. I think I fixed most of these.

Next thing to work on I think is getting the stream changes to not require a restart, and to fix the separate map display. The may display fix will likely be an easier task.

[markjfine]

Plowed through the hard part and fixed the stream switching... now onto the easy tasks...

[andrewfer000]

Stream switching works well. I like this new version a lot better. I just wish there were more people out there who were into SDR and Radio. For all we know, you and I may be the only two people to ever use this program.

Also, nrsc5 has a slightly annoying audio click but I think that has more to do with the bitrate of the radio station than anything and I only notice it when I use my earbuds. I do not have a high-bitrate station (like WHYY) near me to test at the moment but I think that that's out of your control since it's an issue in nrsc5's C code and for all I know it could just be PulseAudio acting stupid like usual.

[markjfine]

I always thought the clicking was more to do with the issues with the radio more than anything. The information is in the sidebands. All you need is a strong signal on either side to modulate funny and it blows the whole thing up. That's one reason why Ibiquity is great if you want backwards compatibility, but really awful for digital radio. DRM is all in-band and you only need 10MHz. It's magic that it works at all on shortwave.

BTW: The map issue was a one-line problem. I'll post another fix, but somehow there was a line missing under builder.add_from_file in the NRSC5_Map class:

builder.add_from_file(os.path.join(resDir,"mapForm.glade")) builder.connect_signals(self) ` self.parent = parent # parent class`

The callbacks were never set!

Testing right now, since there seems to be an issue with animating the maps. But I think it's there.

[andrewfer000]

Ah good, Also there is a rare strange bug that happens when I start the audio. Sometimes the GUI will crash and nrsc5's audio will run and i need to run killall nrsc5 and then run the program again where it wont have any issue.

Also after about 15-20 minutes the audio pauses (not underrun skips) and sometimes have glitches such as large skips as well as out-of-order playback. The GUI also crashes with this message

nrsc5-dui.py:733: Warning: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
  self.imgSynch.set_visible(state == 1)
Segmentation fault (core dumped)

Also the clicking stopped for me so its defiantly the signal. Also processing the audio through Pulse Effects helps clean up the sound.

[markjfine]

Sounds like a synchronization issue. It's probably losing synch and reacquiring too quickly for the gui to process. Might have been related to the nasty storm that breezed through. All you need is one of those sidebands to drop out and pop back up very quickly to mess everything up.

[markjfine]

After nearly breaking the entire weather map generation, I managed to fix the animation feature. Regex string was truncated, causing the map list to not be created properly: "^map.WeatherMap_([a-zA-Z0-9]+)_([0-9]+).png" This would have been fine if mapDir was the entire path from root, not just map. Add a utility wildcard before map fixed it: "^.*map.WeatherMap_([a-zA-Z0-9]+)_([0-9]+).png"

Reposting and taking a break... lol

[andrewfer000]

Alright sounds good.The crash happend a few times and it's not a synch issue since both sidebands were coming in strong at around 11 dB each. Also my laptop's CPU usage shoots up so it might be a python bug.

Also, feel free to take as long as a break as you need. You've done more than I can ever ask.

[markjfine]

I'll take a look at what's going on in my F33 VM. The error seemed to be caused by trying to set the synch pilot to green, but it might very well be something else completely - like a resource issue.

[markjfine]

Oh... before I forget... I changed the default path to nrsc5 for Mac/*nix to be /usr/local/bin. Do you think that's a reasonable assumption?

[andrewfer000]

for nrsc5? In that case yes. When I followed the nrsc5 install instructions it installed to /usr/local/bin/. It would be a nice feature for testing if the path can be quickly changed.

Also did you hear about Fedora 34 replacing PulseAudio with PipeWire? I hope that does not cause any issues!

[andrewfer000]

Welp I pulled down your newest code and tested it for a while and it's pretty much perfect. The only thing left I see that can be done is fix the gain stuff and whatever else you think can be improved.

[markjfine]

Had not heard about PipeWire. Seems like an odd thing to do, but it wouldn't surprise me.

Things on my TODO for now:

[Items moved to new TODO issue]

[andrewfer000]

Those features sound pretty cool. I mean the MPS/SPS1/SPS2 is kinda the stations fault. The Navteq/HERE feature would be a great addition for when iHeart stations are not available. The timestamp on the traffic might be harder since traffic is downloaded in a grid of 3x3. I guess a way you can do that is timestamp the image on the bottom right only. As for the AirSpy support, You can probably find some C/C++ code for it somewhere and implement it in nrsc5. I do not have an AirSpy so I cannot help. Here is a FM Radio project that has airspy support https://github.com/JvanKatwijk/sdr-j-fm The developer is nice and active so maybe he can help.

[markjfine]

The traffic is in 9 pieces, but they assemble it into a single image that I can add the timestamp to prior to display. The weather map also has to be constructed before display. The station only provides the overlay which is merged with a background map image that created from a much larger map image (map.png).

I have the nrsc5 and airspy api code. I started looking at it, and the trick is making sure the interface lashes up properly.

[markjfine]

Can cross 6 off the list

[andrewfer000]

Can confirm it works. The stamp is a little large though.

[andrewfer000]

At this point this version of the program exceeds the features and functionality of the old version. The only thing this version does not have is a Windows package (but is that really important?). Hats off to you for doing what I thought would of been impossible in just a few days!

While I am satisfied at the program as it is, I will continue to test new features and give feedback as you add more. I think the best thing that can be done now is share this version with others in the community and hope they show interest and test it on more platforms and areas (Since we're both in NJ, We cannot be 100% sure how well it performs in certain conditions).

Edit: I also plan to test this program on a PinePhone and RPi when I get one. I want to send a message to Purism and Pine64 that it could be a cool idea to build-in a RTL-SDR (even if it's one of those little $12 dongles) in to their future phones. (Having a cheap GNU/Linux mobile RTL-SDR + Smartphone would be the coolest thing ever!). This program is one of the steps in my testing plan. I also would like to have an easy FM Radio player with RDS support like what my LG V30 has. But no one has developed one yet for RTL-SDR & Linux so I am trying to look for jumping off points such as sdr-j-fm where I can just transform it into a simple GUI without the graphs or anything.

[markjfine]

I think I fixed the sizing. The overlay was designed for a 981x981 weather image and the traffic images are 600x600. So I just make the overlay for a weather image and shrink it to fit the traffic image. This way when both are reduced to be the same size visually, the timestamp will be resized proportionally. I'll test that tomorrow and repost.

I could swear my old Droid X had something like an RTL for it's built-in FM receiver.

I've often wondered why there were all these specialty apps. Maybe there's something to adding a straight FM Stereo capability in this app for when HD Radio isn't available. If I'm thinking like a user, HD Radio should really be just another mode in a bigger app. Should also be AM and C-QUAM too, though C-QUAM AM Stereo may be a dead mode. Maybe add this to the long-term project goal.

[andrewfer000]

I mean if you can implement FM in this app that would be awesome. I was going to mention AM as well but I forgot. If you can do it and you have the time, by all means do it. I think the SDR community would like a program like this especially once Linux phones like the Librem and PinePhone become more popular. I do not know how easy it will be to implement FM with RDS (RBDS) functionality. It will be a challenge, but code for it exists in programs like sdr-j-fm so if you don't want to spend hours on end researching the technology know that most of the work has been done for you already in those projects.

Also. I have just tested this with my worst SDR hooked up to my attic/outdoor TV antenna and it works 1000x better than the old version.

[markjfine]

Traffic fix is up there and looks pretty consistent with weather now.

Shouldn't be too hard to implement RDS. The trick is getting a working RDS. I have the code from gqrx, but it doesn't work very well and seems to need a near-perfect signal. The best I've seen implemented is in SdrDx. Think I have the code for that somewhere. It also has C-QUAM, but have never seen it work on an RTL-SDR (or Airspy for that matter). Because SdrDx requires an intermediary TCP server, I've never been able to get SdrDx to work with an RTL-SDR in direct sampling mode. That makes doing anything below 28MHz very difficult. lol

The real shame is the only all-digital AM station using HD Radio is in the Maryland side of DC (820 WWFD the Gamut). It's one of those stations that plays literally everything (think that used to be a format called 'Jack' for 'Jack of all trades'). I can see it clearly on an Airspy when I'm home in Virginia, but the noise level is too high on an RTL-SDR. I can even see it on my JRC 535D and could play it if I had a baseband decoder. So I've never been able to really test nrsc5 in AM mode. They're a stream on both WTOP's FM HD Radio channels (103.5, 107.9), so I can hear them there, but not on the AM side.

[markjfine]

Did a little testing. 100.3 has no adjacent channels to interfere and should be easily decoded by nrsc5. It's not a perfect signal here, yet nrsc5 struggles with it.

Looked at the spectrum on gqrx using the same sampling rate (1.44MHz) and agc value (20) I use on nrsc5-dui. Both sidebands are clearly defined with zero adjacent interference. However, the RTL is generating a few poorly positioned spikes in the sidebands that are enough to disrupt processing (rhythmic dropouts, etc.)... yet another reason why I'd like to move off of RTL onto an AirSpy.

[andrewfer000]

Interesting. I do not have any issues on my RTL-SDRs. While Airspy is likely a lot better, the main benefit of RTL-SDRs is that they use the highly available and low-cost RTL2832U+R820T/2 or RTL2832P+R282D chips (I have devices with both). I mean you can only make hardware work so well but I'm sure there are still some software and driver optimizations that can be made to improve RTL-SDR.

[markjfine]

The difference is in the amount of filtering. The RTL was originally designed for very wideband signals, which is why you get up to 2.5MHz instantaneous bandwidth. It presents a problem when you're only looking at signals that are 10% of that. Without proper filtering you get intermodulation products (spurs) from strong signals mixing with each other within the receiver's front end. It can also artificially raise the noise floor. If you've used applications like gqrx or sdr#, you see these things as spikes and fake signals in the spectrum, as well as other products due to mirroring in the fft algorithm.

[andrewfer000]

Feature Request- Album Art Database (low priority / long term)

After doing some thinking, I came up with an idea for an album art database that, like the station logo database would be able to match song + artist up with with the song that is currently playing allowing for a even more accurate album art display: Here is an example structure -

{
  "10303_WRFF01283f.jpg": [    //File name
    "10303_WRFF01283"  // ID
    "Mr. Brightside", // Song Title
    "The Killers",   // Artist
    " ",   //Album (If provided)
  ],
}

It's like the reverse logic of the station art. The only issue I see with implementing this is when the next album art comes in early. One solution I see is add a counter, so a song cannot have the next song's art overwrite the correct one if it comes in during the current song.

Also, A feature like this can make it easier for users to use their own higher quality album art if desired or maybe allow the possibility of using an optional internet database for higher-quality album art.

I'll look at the code and see if I can implement parts of it myself.

[markjfine]

I like the idea, and there's even another reason for doing it. Don't know if you noticed, but the aas directory gets pretty large. A lot of that is from duplicate images for the same song, or even songs from the same album. Do I really need three copies of the same image with different file names from the same Dua Lipa album? Seems a little wasteful.

There's a station in the DC area that uses Gracenote for the images, with the song name and a serial number already embedded within the filename, (e.g., %2Fgracenote200%2FBohemian_Rhapsody_142.jpeg). Makes things easier in that case.

One way of solving the duplication is setting up a folder hierarchy for the images: artist, album, song. Then it becomes easy to index, provided the spelling and capitalization is similar. It also moves them out of the top level aas directory, which could solely be used for logos. I want to discriminate better between how logos and album art is used anyway.

[markjfine]

In fact, if the user wants to correct the album art, I may have found a way to automate getting it from Discogs by just sending the Artist and Title info. You just create a string with "Artist-Title" then replace each space with a '+'. Send it to https://www.discogs.com/search/?q=&type=all, then look for "card card_large float_fix", then "<img data-src=" and the album art is url between the following quote marks...

[andrewfer000]

Interesting, but I have not tried it myself yet. If it won't be a hassle to implement maybe that can make for a good online album art database.

If you are interested in implementing such a feature you should add it to the TODO list so we do not forget it.

[markjfine]

Consider it added. :)

Edit: I actually thought I added it when I wrote that. Must have bailed on the commit button.

[andrewfer000]

The new Track Info implementation does not allow window resizing. I personally preferred when the text overflowed and I can use the textbox to scroll to see the rest of the text.

A nice feature (if GTK supports it) would be if the overflowed text scrolled.

[andrewfer000]

For some reason, some resent changes are preventing the station logos from HD sub-channels from downloading.

I need to do more testing before confirming this issue. Maybe it's just really slow.

So it seems that this is only happening on some HD sub stations...

Alright, there are some station logo bugs that need to be ironed out. I'll try and figure out what's going on and update.

Somtimes, Station logos will be downloaded but not configured. It looks like 95.7 Ben FM just did that.

So in conclusion, for some stations and substations, the logos will download but will not be set in the config files. It works if set manually.

I think the album-art rework should include the station logos as well since that seems to have some strange quirks as well.

[markjfine]

Track Info: Let me check on the Track Info. Wouldn't be hard to change it back, if needed. It just looks cleaner this way. Another alternative is to make them ellipsize if too long and add tooltips. Would that be more helpful? Might be easier than scrolling.

Station logos: Some station logos are actually sent as covers. I noticed this on one station. The station logo has to start with a serial number, underline char, 'SL', then the call, eg, "7378_SLWPEN$$010007.jpg" for WPEN. I've noticed another station will send 1004_sl855af729e9055fc2012689f7e776e214.jpg, which is actually a cover that looks like the logo.

But there are quirks, as I've said, where the logos will download for every stream, but won't 'take' unless you're actually tuned to that stream. Has to do with lashing up lot numbers, etc, to make sure it attaches to the right stream. This, even though it's already cached that information for every stream. Needs fixing for sure. It's a silly bug.

And yes, the handling of station logos and covers needs to be separated. Right now they're using the same routines and it causes problems like this.

[markjfine]

Ok... added tooltips to the Track Info. Let me know if this also works. I just think it looks cleaner than using an entry box. Labels are supposed to static, whereas entry boxes are for user entry... it's a minor style point, but I have OCDs about keeping things uniform. lol

I also added your suggestion about adding a static path to nrsc5 as an argument, just in case that was Casey's problem. Seems to work when I add it to the Apple Script, and leave it out elsewhere. Tried automating the path by using which nrsc5 and piping the result, but remembered if it can't see the PATH variable there's no point. the argv approach is good enough.

[markjfine]

Just added using the bookmark names (if available) when a station uses sloppy and nebulous ways to name the streams. This is the last semi-easy thing on the TODO list for now. Might focus on getting nrsc5 to work with my AirSpy before proceeding, unless there's a new bug that crops up.

[markjfine]

Just because it was on my mind as something cool and needed to do something about it as a personal challenge: I now have an operational prototype that senses when the ID3 info changes, searches Discogs, downloads the album art (if it can find it), save it in the aas directory, then display it. It also knows to use that image the next time the title comes up.

One I get it playing nice with the logo and the station-provided cover images I'll post the change. Also finding Discogs can be a little touchy when it comes to anything esoteric, so the accuracy may be questionable in those cases.

That said, it means two new things to pip3 install (if not already): urllib3 for the internet interface, and pyOpenSSL to handle the 's' part of 'https'.

Having a little difficulty because reception is absolutely awful today (one of the reasons why I wanted to do the AirSpy thing next). Must be a truck parked somewhere messing me up.

[andrewfer000]

That's Awesome! I only ask that using Discogs can be toggled in the GUI just in case the station uses custom or not available album art (such as for a new album or studio recording).

[markjfine]

Or in this case, Discogs sometimes returns something off the wall