Possibility of SpyServer Client Support?

[darksidelemm]

Hi!

This looks like a super useful addition to a remote HF site I am involved with, which has (along with multiple KiwiSDRs - see kiwisdr.areg.org.au:8073 ) a pile of Airspy HF+ Spyservers - one for each HF amateur radio band. We run these because they allow more parallel users than the KiwiSDRs, and they provide much less audio latency than the KiwiSDR, which is great because a lot of people use them as an alternate RX for their home station.

I am still a bit unhappy with the closed nature of the SpyServer system, but there is an open-source spyserver client here: https://github.com/miweber67/spyserver_client which can stream IQ to stdout. There's also now another open-source spyserver client in the form of SDR++, so I'm no longer locked in to using clients like SDR#.

It would be awesome if I could use spyserver_client to run WSPR decoders on the Airspys - this would avoid tying up channels of the KiwiSDRs, leaving them open for other listeners.

I was able to proof-of-concept decoding WSPR via it using the following script (running it manually just before the start time of the WSPR transmissions) and a fork of WSPRd which accepts data from stdin.

FREQ=14095600

# SpyServer Details
HOST=localhost
PORT=5020

# SpyServer Connection, requesting 12 kHz of IQ bandwidth, and using lots of buffering.
ss_iq -a 1200 -r $HOST -q $PORT -f $FREQ -s 12000 -b 16 - | \
# Convert the incoming signed 16-bit IQ to floating point
csdr convert_s16_f |\
# SSB demodulation using a tight bandpass filter (0-3600 Hz), and then taking the real part.
# (I could probably make this a bit narrower) Also add on some AGC and limiting.
csdr bandpass_fir_fft_cc 0 0.3 0.05 | csdr realpart_cf | csdr agc_ff | csdr limit_ff | \
# Convert back into signed 16-bit format and pass into sox, then to wsprd
csdr convert_f_s16 |
sox -t raw -r 12000 -e s -b 16 -c 1 - -c 1 -r 12000 -t wav - trim 0 114 |
./k9an-wsprd -wf "14.0956" /dev/stdin

There's some information on how I compile and use the spyserver client utility for decoding HF APRS here: https://gist.github.com/darksidelemm/6b60767714295962771bca7b728b343c#file-spyserver_hf_aprs-sh-L26

Is this something that could be considered for a future release?

[rrobinett]

Thanks for your interest. One of the goals of WD 3.0 is to broaden the SDR support beyond the KiwiSDR to include other devices like the AirSpy. To that end, I have added support for WD to obtain wav file audio streams through the use of the SoapyAPI which has drivers for many SDRs. My experiments with the use of Soapy have so far been limited to the SDRPlay and RTL-SDRs, and I was al little concerned about Soapy's CPU and bandwidth costs.
But the major restructuring of WD in V 3.0 has made it much easier to add device-specific support to WD, and your posting has given me a very clear blueprint for doing that for the AirSpy, although I don't own one to use in developing the code. Your receive site is quite impressive and the WSPR community desperately needs a multi-band WSPR receive site in your part of the world. But on a per-band basis, in 8 channel mode the Kiwi is a much less expensive WSPR receiver than the AirSpy which when used with a system like WebSDR might offer better ham band listening performance than the Kiwis?

Please join us on our forum at https://groups.io/g/wsprdaemon for further discussions of this and other topics.
We have a WD contributor adding Red Pitaya support in a way analogous to your's and I would very much like to add the AirSPy as well.

[darksidelemm]

Since in this case i'm talking specifically about airspy spyserver support, there's heaps of spyservers online that you can test against - no need to actually own an Airspy. Happy to email you some details of ours (they are not on the public spyserver list until we know the sites internet connection can handle the outbound bandwidth).

Another reason behind going with the spyserver route for each band has been that I can fairly easily attach other 'listeners' onto each spyserver, without much penalty in CPU cost. I can happy have maybe 20 users attached to each spyserver at the moment.

Right now i'm running HF APRS receivers up there, and more recently a club member has written a SSTV Skimmer which connects to the spyserver (https://github.com/xssfox/sstv-skimmer output visible on https://botsin.space/@aregsstv ) - if we were doing this on the KiwiSDR we'd nee a lot more kiwisdrs at the site!

Getting an all-band WSPR receiver going at the site is certainly the aim here! We already have airspys set up for 160/80/40/30/20/15m, and are planning on adding 17/12/10m soon. Later on we are also intending on adding omnidirectional horizontally polarised antennas for 6m, 2m and 70cm as well.

[rrobinett]

Hi Mark,

In this morning's (for me in Californai) WD Zoom call we had an extensive discussion of your site and desired configuration. It makes complete sense that you would want to use the SpyServers as the source for your WD and the code you supplied seems relatively easy to integrate into WD. But as I am the only really active developer it may be some time before I can work on it. I'll post to the WD group when I have time to look into it.

73,

Rob

On Sun, Jan 30, 2022 at 1:15 PM Mark Jessop @.***> wrote:

Since in this case i'm talking specifically about airspy spyserver support, there's heaps of spyservers online that you can test against - no need to actually own an Airspy.

Another reason behind going with the spyserver route for each band has been that I can fairly easily attach other 'listeners' onto each spyserver, without much penalty in CPU cost. I can happy have maybe 20 users attached to each spyserver at the moment.

Right now i'm running HF APRS receivers up there, and more recently a club member has written a SSTV Skimmer which connects to the spyserver ( https://github.com/xssfox/sstv-skimmer output visible on @.*** ) - if we were doing this on the KiwiSDR we'd nee a lot more kiwisdrs at the site!

Getting an all-band WSPR receiver going at the site is certainly the aim here! We already have airspys set up for 160/80/40/30/20/15m, and are planning on adding 17/12/10m soon. Later on we are also intending on adding omnidirectional horizontally polarised antennas for 6m, 2m and 70cm as well.

— Reply to this email directly, view it on GitHub https://github.com/rrobinett/wsprdaemon/issues/17#issuecomment-1025232936, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIFAQZBSMHABXSMXVNS7SV3UYWTA5ANCNFSM5NEDDQJQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896

[rrobinett]

Hi Mark,

Now that WD 3.0 has been released and deployed, I can look into this. I see the biggest challenge will be to create a direwolf-like utility which outputs a continuous series of one minute long wav files. The long chain of commands in your example suggests that it might be a challenge to determin which audio sample matches the beginning of the 2 minute WSPR cycle .

rob

[darksidelemm]

Yes, there will likely be some latency through that processing chain. I suspect the latency should be deterministic however, and may just require a 'fudge factor' somewhere in the processing.

I think it's still something that would be highly valuable however... We now have spyservers running on all HF amateur radio bands, and just recently added one on the 2m band.

[darksidelemm]

Could you explain a bit more about what kind of output is required? You mention multiple one-minute long wav files (Should this be 2 minutes long, corresponding with the WSPR cycle time?). Do these just written out to a directory from which some other process grabs and decodes them?

[darksidelemm]

On this note, I've started a repo here which will take input from spyserver_client, and drop out wave files every 2 minutes: https://github.com/darksidelemm/spyserver_wspr_skimmer

[rrobinett]

Hi Mark,

WD spawns kiwirecorder.py (KWR) job for every receive 'job' (i.e. kiwi+wspr rx band). KWR is instructed to tune to the the WSPR band in upper sideband mode with the lower audio passband cutoff at 1340 hz and the upper passband cutoff at 1660 hz. KWR then outputs a series of 12000 sample per second sample-contiguous 1 minute long wav which have names which include the UTC date+time, the tuning frequency, and the upper sideband mode.
So if your program could create a series of similar wav files, it would be easy for me to incorporate it into WD.

Rob

=============

wd_client@KFS-WD3:/dev/shm/wsprdaemon/recording.d/KIWI_Omni_A/20$ ls -lrt *wav -rw-rw-r-- 1 wd_client wd_client 1439788 Oct 9 14:15 20221009T141500Z_14095600_usb.wav -rw-rw-r-- 1 wd_client wd_client 1439788 Oct 9 14:16 20221009T141600Z_14095600_usb.wav -rw-rw-r-- 1 wd_client wd_client 1439788 Oct 9 14:17 20221009T141700Z_14095600_usb.wav -rw-rw-r-- 1 wd_client wd_client 1441836 Oct 9 14:18 20221009T141800Z_14095600_usb.wav -rw-rw-r-- 1 wd_client wd_client 1439788 Oct 9 14:19 20221009T141900Z_14095600_usb.wav -rw-rw-r-- 1 wd_client wd_client 532524 Oct 9 14:20 20221009T142000Z_14095600_usb.wav wd_client@KFS-WD3:/dev/shm/wsprdaemon/recording.d/KIWI_Omni_A/20$

[darksidelemm]

Hmm, I'm pretty sure now I can reliably produce a just-under-2-minute (~116 seconds) recording covering a WSPR transmit period without worries about sample slips and drifting off time, though I'm not 100% sure i can produce perfectly contiguous 60-second recordings at this point, at least not continuously. At the moment I'm using the gap at the end of the WSPR transmit period to ensure my recordings are back in sync with UTC.

I'm pretty sure I would be able to split my 116 second recording into a 60 second and a 56 second recording safely. Would this work well enough?

[rrobinett]

I don't think your split would work well. KWT runs continuously after startup and so it doesn't drop any audio samples. Any gaps will compromise the ability of wsprd and jt9 to extract spots.

On Sun, Oct 9, 2022 at 2:28 PM Mark Jessop @.***> wrote:

Hmm, I'm pretty sure now I can reliably produce a just-under-2-minute (~116 seconds) recording covering a WSPR transmit period without worries about sample slips and drifting off time, though I'm not 100% sure i can produce perfectly contiguous 60-second recordings at this point, at least not continuously. At the moment I'm using the gap at the end of the WSPR transmit period to ensure my recordings are back in sync with UTC.

I'm pretty sure I would be able to split my 116 second recording into a 60 second and a 56 second recording safely. Would this work well enough?

— Reply to this email directly, view it on GitHub https://github.com/rrobinett/wsprdaemon/issues/17#issuecomment-1272631625, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIFAQZDVISPT55HOWBREFZ3WCM2I5ANCNFSM5NEDDQJQ . You are receiving this because you commented.Message ID: @.***>

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896

[darksidelemm]

OK, in that case I think I'll withdraw this issue. I suspect that there will be far too many changes required to wsprdaemon to support anything other than KiwiSDR and it's timestamped samples.

I'll go my own way with this one.

[rrobinett]

I understand. Another approach would be to communicate with the AirSpy using a SoapyAPI. The author of SDRGlut ( https://github.com/righthalfplane/SdrGlut ) enhanced the SDRtest utility included in that package to duplicate the 'record a series of one minute wav files' feature of KiwRecorder. Although I only tested SDRTest with an SDRPlay, in theory any of the many SDRs for which there is a SoapyAPI driver should work with WD using that interface.

On Mon, Oct 10, 2022 at 2:55 AM Mark Jessop @.***> wrote:

OK, in that case I think I'll withdraw this issue. I suspect that there will be far too many changes required to wsprdaemon to support anything other than KiwiSDR and it's timestamped samples.

I'll go my own way with this one.

— Reply to this email directly, view it on GitHub https://github.com/rrobinett/wsprdaemon/issues/17#issuecomment-1273065127, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIFAQZAT4M5RCI3UX4YFNATWCPR2BANCNFSM5NEDDQJQ . You are receiving this because you commented.Message ID: @.***>

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896

[darksidelemm]

There does appear to be a (very new) Spyserver client for SoapySDR, so that may be an option to investigate. A bit of a heavyweight solution though.