SoapySDRPlay support

[ghost]

I am trying to get the SDRplay to work on a mac using osmosdr and it just doesn't work. Also i don't expect too much Mac support from the SDRplay guys. They are mainly focussing on Windows. Given that the SDRplay API is public, how much effort would it be to add native support for SDRplay in CubicSDR? I think native support for the trio RTL-SDR (done), AirSpy and SDRplay would make CubicSDR the first SDR application for the "new generation" SDR devices.

In this issue you can find (will be copied to wiki eventually):

Build instructions (OSX): https://github.com/cjcliffe/CubicSDR/issues/144#issuecomment-140906180 Update instructions: https://github.com/cjcliffe/CubicSDR/issues/144#issuecomment-141237861

[cjcliffe]

@dc1rdb @Toontje @SDRplay I've updated my SoapySDRPlay branch with an experimental fix for the 'deaf' issue; If the frequency change is more than half the current bandwidth it'll force a re-init.

I'm guessing @Toontje isn't often tuned near 100Mhz on startup but @dc1rdb might often be; so he's not having as many issues since there isn't a huge frequency jump on startup -- the latest patch should solve this issue by not changing the Rf frequency in large increments but instead re-initializing at the new frequency.

Edit: I've also just updated the soapysdr-support branch with experimental attempt to request the ideal amount of sample elements from SoapySDR to balance the visual framerate cap at 60fps. It should help CubicSDR use larger buffers at higher sample rates as well.

[ghost]

Still deaf. I noticed something when starting CubicSDR, maybe this gives you a hint. Immediately after starting CubicSDR, before the automatic ACG kicks in, i have one blip on the waterfall.

[cjcliffe]

@Toontje It looks like it's working but the input is flat -- there's actually no AGC control on the SoapySDRPlay module at the moment; it just sets it to '40' once at the start and that's the end of it -- I'm starting to think perhaps the lack of gain control at the moment is actually the problem.

I'll try implementing the AGC outlined in the SDRPlay documents later today and we'll see if that makes any difference.

[ghost]

What do you mean with “the input is flat”? The screenshot shows 105MHz which is where a super loud WFM broadcast station sits. Here are the RTL (0.1.4) and the SDRplay (0.1.8) side my side on 105MHz:

[dc1rdb]

Looking excellent now - latest commit 626d557f84 has solved the "deaf" issue here. Thanks a lot!

[cjcliffe]

@dc1rdb good to hear; more progress!

@Toontje your screenshot above my last comment just shows a blip and flat spectrum is what i was referring to -- it looks like there's data streaming in but the data contains little or no signal (since the waterfall keeps moving after the blip)

The comparison shot looks like bad gain control -- the dB ceiling on the SDRPlay is nearly as low as the noise floor on the RTL-SDR; but the automatic visual scaling on the spectrum has stretched out the noise since there were no defined peaks.

I think if I add the SDRPlay suggested gain control in there we'll see a better dB range and some signals might emerge from that noise..

[ghost]

Wait a minute, i have to get things clear here. SDRplay is connected to a outdoor antenna. RTL is connected to its standard magnet "thingy". Are you telling me that the SDRplay cannot receive the WFM stations and the RTL can? I think my SDRplay is currently in "deaf" mode and that for this the spectrum scaling looks the way it looks. There is nothing coming in to the SDRplay. Same location, better antenna, hence SDRplay "deaf".

Since @dc1rdb seems to have success i will build SoapySDRplay and SoapySDR again from the last commits. Let's see what it brings.

[ghost]

My SDRplay is working again. I had to go into Windows (Bootcamp) and launch HDSDR where my SDRplay works fine, then back to OSX and launch CubicSDR. Now it works. Apparently CubicSDR/SoapySDR/SoapySDRplay is capable of putting the SDRplay in a state where it gets deaf and nothing, not even unplugging, can get the SDRplay going again. Maybe @SDRplay can say something about this.

[cjcliffe]

@Toontje not quite; I was suggesting that my SoapySDRPlay is leaving the SDRPlay gain control in a bad / invalid state -- and since I haven't implemented gain support yet it's likely that the RF input levels could be anywhere between "no signal" ("flat", data is all zeros) and completely overdriven. If the SDRPlay had completely stopped sending data I would expect to see nothing in the waterfall areas at all.

I'm not suggesting anything is wrong with the SDRPlay; it should definitely look like the RTL device output -- I'm suggesting that I haven't implemented any Gain or Auto Gain Control yet so the actual internal gain values might just be left wherever they left off at (including possibly invalid values).

The up/down scaling movement of the spectrum display has nothing to do with Gain or AGC; that's just basic re-adjustment of the visible floor/ceiling to fit the data in view.

[cjcliffe]

@Toontje @dc1rdb I've implemented the AGC routine that's outlined in the documentation @SDRplay referred me to and pushed an update to the SoapySDRPlay repository.

Let me know what that does; if anything :)

[ghost]

I don’t know if it’s the new SoapySDRplay or if it’s the reset-by-HDSDR, but i don’t have the “deaf” problem anymore. All working fine here now.

[dc1rdb]

Just did some quick tests of latest commit c650caa1ef (AGC). Honestly, I can not tell any difference in performance and dB range, compared to the previous version.

[cjcliffe]

@dc1rdb: yeah I have no idea if that actually does anything unless you change the gain :) I think I'm supposed to actually trigger it on a frequency change or something so it may be dormant; I'll take another look and push an experiment or two.

[cjcliffe]

@Toontje good to hear; I was guessing that it was overloading the SDRPlay with frequency changes and if that fixes it then that was likely the case (or something related).

Anyone have enough CPU left over while it's running to record some action videos? :)

[ghost]

Let's see if my Macbook Pro has sufficient cycles to record a video. My SDRplay is working, but i am still a bit worried about the state Soapy left my SDRplay in. I solved it by using HDSDR. I don't think we can ask users to install Windows and HDSDR when this happens to them.

[ghost]

Ok, i think i am short on cycles. Check this video: https://www.dropbox.com/s/9yoi7b5q4pnh17k/DemoSDRplay.mov?dl=0 If you cannot use it, just let me know and i'll remove it. This is recorded using QuickTime Player.

[cjcliffe]

@Toontje got the video, looks like it's working reasonably -- I should be able to optimize the CPU usage a fair bit more once I have a unit handy.

If the latest fixes aren't breaking your SDRPlay anymore I'm hoping that's the last we'll see -- the documentation makes it clear that large frequency changes should initiate a reset or they might overload the unit -- that's what fixed the sample rate issues but I didn't do this for the Rf tuner itself and it was throwing arbitrary changes of any size at it without consideration -- now it's handled in both cases and is hopefully much more gentle with the hardware.

[cjcliffe]

@Toontje also I note a feature you might have missed in recent upates -- if you hover the bandwidth digits and press space it will now let you enter the bandwidth manually (i.e. 200khz to snap to a WBFM)

[ghost]

Missed that one, yes. I do it for the Center and LO, but not for the bandwidth. BTW, CubicSDR is not the only one suffering from the "deaf" issue. This sound similar: https://www.facebook.com/groups/sdrplay/permalink/1658485501055708/ but for SDR Console.

[cjcliffe]

SDRPlay test unit has arrived! I'll start making fixes and optimizations shortly :)

[cjcliffe]

Ok; so I've made a hilarious error from misinterpreting the filter bandwidth documentation that I've noticed right away -- I will have a fix up shortly :)

[cjcliffe]

Fix has been pushed to my SoapySDRPlay repository and pothosware master; filter bandwidth now properly matches up with the sample rate requested.

[ghost]

I had a "deaf" issue again yesterday when starting CubicSDR. Unplugging and re-plugging solved the issue this time, but i think this issue should be looked after. Do you want me to make a separate issue out of this?

[cjcliffe]

@Toontje start a new issue describing the steps and current state of the "Deaf" issue; I have a unit here now so if I can reproduce it I should be able to find it quickly.

[cjcliffe]

Please Note!

I've removed my fork of the SoapySDRPlay repository, if you need to update use the official repository now which I'll be committing updates to.

You can re-orient your existing checkouts with the following command:

SoapySDRPlay/$ git remote set-url origin https://github.com/pothosware/SoapySDRPlay.git

Cheers!

[cjcliffe]

@Toontje @dc1rdb I've updated SoapySDRPlay with automatic gain control; this makes it much more usable here.

I was experiencing extreme aliasing and noise all over the bands and the SDRPlay appeared to be overloading and losing some sort of sync -- adding proper AGC appears to resolve this issue.

I've also updated CubicSDR to support hardware DC correction option; still tweaking but it gains another 5-30% CPU here depending on bandwidth.

[ghost]

That last update was not a successful one:

mir_sdr_SetGr: GR->37[13,0,0,24] gRset->0xCD DCCALmode=4 DCCALspd=1 GrToggle->1
mir_sdr_ReadPacket: Gain update confirmed: Gr=37dB GrToggle=1 gset=0xda
[DEBUG] Gain change acknowledged from device. packet: 108
mir_sdr_ResetUpdateFlags: resetGainUpdate=1 resetRfUpdate=0 resetFsUpdate=0
[DEBUG] AGC: Gain reduction changed from 37 to 42
mir_sdr_2500_SetRegister(0x09, 0x030d21)

mir_sdr_SetGr: GR->42[18,0,0,24] gRset->0xD2 DCCALmode=4 DCCALspd=1 GrToggle->1
mir_sdr_ReadPacket: Gain update confirmed: Gr=42dB GrToggle=1 gset=0xd2
[DEBUG] Gain change acknowledged from device. packet: 216
mir_sdr_ResetUpdateFlags: resetGainUpdate=1 resetRfUpdate=0 resetFsUpdate=0
[DEBUG] AGC: Gain reduction changed from 42 to 45
mir_sdr_2500_SetRegister(0x09, 0x030d51)

mir_sdr_SetGr: GR->45[21,0,0,24] gRset->0xD5 DCCALmode=4 DCCALspd=1 GrToggle->1
mir_sdr_ReadPacket: Gain update confirmed: Gr=45dB GrToggle=1 gset=0xd2
[DEBUG] Gain change acknowledged from device. packet: 0
mir_sdr_ResetUpdateFlags: resetGainUpdate=1 resetRfUpdate=0 resetFsUpdate=0
[DEBUG] AGC: Gain reduction changed from 45 to 41
mir_sdr_2500_SetRegister(0x09, 0x030d11)

mir_sdr_SetGr: GR->41[17,0,0,24] gRset->0xD1 DCCALmode=4 DCCALspd=1 GrToggle->1
mir_sdr_ReadPacket: Gain update confirmed: Gr=41dB GrToggle=1 gset=0xd2
[DEBUG] Gain change acknowledged from device. packet: 0
mir_sdr_ResetUpdateFlags: resetGainUpdate=1 resetRfUpdate=0 resetFsUpdate=0
[DEBUG] AGC: Gain reduction changed from 41 to -2147483648
mir_sdr_2500_SetRegister(0x09, 0x030001)

mir_sdr_SetGr: GR->0[0,0,0,0] gRset->0x0 DCCALmode=4 DCCALspd=1 GrToggle->1
mir_sdr_ReadPacket: Gain update confirmed: Gr=0dB GrToggle=1 gset=0xd1
[DEBUG] Gain change acknowledged from device. packet: 0
mir_sdr_ResetUpdateFlags: resetGainUpdate=1 resetRfUpdate=0 resetFsUpdate=0
[DEBUG] AGC: Gain reduction changed from -2147483648 to -2147483648
[DEBUG] AGC: Gain reduction changed from -2147483648 to 54
mir_sdr_2500_SetRegister(0x09, 0x030de1)

Without touching the application....

[ghost]

CubicSDR also crashes:

See here: https://gist.github.com/cjcliffe/0c088ecf121b9671f5e6

[cjcliffe]

@Toontje that looks like an overflow of some sort for sure; I'll have to add some logging to figure out where it goes wrong, thanks.

[dc1rdb]

For some reason, latest commit 7960a671a0 works on my machine with no overflow. However, I noticed that the AGC is very slow and changes gains up and down quite a bit. See waterfall in attached screenshot.

[dc1rdb]

Similar results on shortwave AM:

[cjcliffe]

@dc1rdb looks like gain value might be sort-of overflowing for you as well; when it does here it's usually flipping back and forth between an invalid and valid value in the log -- it should be an easy fix I'll have a look this evening.

Edit: found a missing gain update buffer offset increment; this would cause the AGC to re-evaluate a mix of current/previous samples which would toggle a fair bit. Fix coming shortly!

[cjcliffe]

@Toontje @dc1rdb I've updated SoapySDRPlay with some fixes for my hacked up AGC; it should now be in-sync and updates should hopefully be smooth and seamless (although frequent if the spectrum is busy!)

I may need to figure out how to disable mir_sdr logging :-)

[cjcliffe]

I've had CubicSDR and SoapySDRPlay with the latest updates running for a few hours consecutively now doing random things at around 7Msps on my late-2010 macbook pro. It's maintained clear audio and the gain has only been sketchy a few times.

I can't use more than one demodulator at a time at anything much above 4Msps and I can use the waterfall at rates up to 10-12Msps but demodulation isn't effective on my current environment (which I don't want to change as incentive to make it work effectively on here :). -- But I have some experiments/ideas and helpful suggestions provided that should alleviate that in the near future.

*don't know what's up with my menu bar at the moment and in that pic, I managed to tank the Nvidia driver and it recovered that way ;)

[SDRplay]

We will be releasing the Mac API with the gap coverage functions. I will also email you some information on how they are used. We will also get a lot of the debug messages removed. Too many cycles are spent producing them.

[ghost]

Much better, yes!

On Fri, Oct 2, 2015 at 2:07 AM, Charles J. Cliffe notifications@github.com wrote:

@Toontje https://github.com/Toontje @dc1rdb https://github.com/dc1rdb I've updated SoapySDRPlay with some fixes for my hacked up AGC; it should now be in-sync and updates should hopefully be smooth and seamless (although frequent if the spectrum is busy!)

I may need to figure out how to disable mir_sdr logging :-)

— Reply to this email directly or view it on GitHub https://github.com/cjcliffe/CubicSDR/issues/144#issuecomment-144881665.

[dc1rdb]

Much better here, too. Only occasional gain hickups.

[ghost]

Smashing up and down a lot today. Impossible to read any signals like this. BTW, i started CubicSDR with a >>/dev/null to suppress the debug messages.

[cjcliffe]

@Toontje yeah there's still a few scenarios that the AGC doesn't handle well; need to add some basic filtering of sharp gain changes

[ghost]

Just let me know when you update your code again. I couldn't resists bringing my SDRplay on my travel. ;-)

[cjcliffe]

@Toontje will do -- I'll probably have some updates shortly; doing a bit of work to cleanup SoapySDR integration and setup -- current status:

[cjcliffe]

@Toontje @dc1rdb I've added some basic filtering to the SoapySDRPlay AGC gain reduction changes; it should help average out the AGC response to sudden changes in signal.

[ghost]

Every time better. No more jumping of the AGC. What i do notice is that the visual gain now is not as responsive as it was before. Before there was more contrast in the waterfall. Now the FM broadcast band looks like this: Top waterfall correct, Main waterfall not enough contrast.

[ghost]

New errors:

mir_sdr_SetRf: f->100918192.000Hz (int=21 frac=77e afc=312) fSynth:3229382144.000
mir_sdr_SetRf: Rf->RfNomHz+dRfHz+LifHz+Lif1Hz=100918192.0+0.0+0.0Hz+0.0Hz=100918192.0Hz RfToggle->1
[DEBUG] Changed center frequency
mir_sdr_ReadPacket: Rf update confirmed: Rf=100918192.0Hz RfToggle=1
[DEBUG] Center frequency change acknowledged from device. packet: 523
mir_sdr_ResetUpdateFlags: resetGainUpdate=0 resetRfUpdate=1 resetFsUpdate=0
Error: USB buffer overflow

Error: USB buffer overflow

Error: USB buffer overflow

Error: USB buffer overflow

Error: USB buffer overflow

Error: USB buffer overflow

Error: USB buffer overflow

Error: USB buffer overflow

Error: USB buffer overflow

Error: USB buffer overflow

[dc1rdb]

AGC response is very good here now, too. Unlike @Toontje, I'm not experiencing waterfall contrast issues, though.

However, some occasional lockups resulting in the following log entry:

libusb: warning [darwin_transfer_status] transfer error: data overrun
Warning: DriverCallback() ISOCH transfer did not complete status = 6

[cjcliffe]

@Toontje I think I have a possible fix for your remaining gain issues; I've pushed an update that adds a watchdog to check if the gain updates have stalled and kicks them back in. I had something similar before but it didn't reset the gain flag properly so I don't think it was effective.

I was having the AGC stall several times here while undergoing sharp changes in signal level and would have to switch the sample rate or jump around to get it going again; the latest update should keep it from doing this.

Search your log for 'kick' and see if it's happening to you more often, I've got about 129 kicks in the past 15 minutes here.

[cjcliffe]

@Toontje @dc1rdb I think the data over-run issues are just indicative of more optimization required; that's what happens here when I try to do too much zooming at high bandwidth which rapidly rebuilds filters and peaks my MacBook CPU too much thereby killing the stream.

[SDRplay]

The latest API is now out (v1.7) www.sdrplay.com/mac.html - @cjcliffe I'll send you the notes we have on the new functions to close the frequency gap we had. Contacted Josh about supporting the binary builds on our website.

[dc1rdb]

Just rebuilt latest commit bdc40b7573 with API v1.7 and did some testing. The overall stability seems to be improved - I could not reproduce the data overrun issue anymore...

[ghost]

Halfway the waterfall i disconnected the antenna. Still a completely yellow waterfall. An i missing something? Compared to HDSDR on Windows: