Whiting et al. proposes that real time clock synchronization among ADCs in a distributed network can be achieved on the software level without a dedicated hardware solution such as GPS-disciplined oscillators. This would be extremely useful to our current transmitter/receiver architecture which currently does not have the dedicated hardware necessary to synchronize unconnected ground nodes. There are still many questions that remain on the specific paper, which proposes solutions to bulk delays, clock timing drift, and frequency/phase offsets.

Questions:

In the Bulk Delays section of the paper, no reference signal or beacon is mentioned. Our initial understanding was that an external reference signal in a multichannel coherent receiver or a beacon in a distributed network would be needed during cross correlation. This is because a cross correlation of mobile node transmissions would include the phase offsets of the received signal at multiple antennas (which would differ based on direction of arrival). I can see how this may not be an issue in clock timing drift solutions, in which the stream that has the faster sampling rate is interpolated to the rate of the stream with the slower rate.
How are frequency and phase offsets not incorporated into bulk delays and clock timing drift? My initial understanding is that clock timing drift includes frequency differences among receivers. There seems to be a difference between receiver frequency difference and ADC frequency difference. Maybe a hint to this answer would be in the Bulk Delays section, where "even in systems with two ADCs sharing a clock, sampling does not typically start at the same time" which may indicate a multichannel coherent receiver, and not distributed unconnected receivers for this section. Also, why would we want to correct phase offsets at receivers? Are phase offsets not necessary for calculating AoAs at coherent receivers in conventional beamforming/MUSIC and phase differences between ground nodes in PDOA?
It is not clear how information is transmitted between unconnected receivers in the distributed network. Wireless transmissions from multiple receivers to a central basestation could potentially create huge offsets between sampling streams, which would require extremely long cross correlations (which would yield inaccurate results). Are transmissions to a central basestation timestamped to avoid this problem?
If bulk timing delay, clock drift, and phase/frequency offsets are reliably corrected among distributed ground nodes using the methods entailed in this paper, will we still need PRN codes to synchronize ground nodes and collect accurate measurements in PDOA?

Response to Silva's June 21st questions

1. Reference signals

In the Bulk Delays section of the paper, no reference signal or beacon is mentioned...

In Whiting's paper, the reference signal he is using is the signal of an FM radio station. I'll admit I do not understand exactly why this is sufficient for his purposes. I would think that there would be minute timing (as well as phase differences) in the received signals from his two receivers; it is unlikely that the receivers are spaced exactly equidistantly from the source FM antenna. That said, perhaps the difference is so small it can be ignored.

I suspect that this setup may simply be a proof-of-concept, and one would want to set up a much more controlled noise source for cross correlation.

(cont.) I can see how this may not be an issue in clock timing drift solutions, in which the stream that has the faster sampling rate is interpolated to the rate of the stream with the slower rate.

I don't think I understand how this relates. If you want to calibrate for clock drift and bulk offsets (both are sampling time issues), I would imagine you would need a good signal to cross correlate.

2. Are frequency and phase offsets really not incorporated by bulk delay and timing corrections?

My initial understanding is that clock timing drift includes frequency differences among receivers. There seems to be a difference between receiver frequency difference and ADC frequency difference.

I'm afraid I am somewhat puzzled by this as well, but I think I have a vague idea of why this is a problem. Bulk timing offsets occur even in coherent receivers where one receiver begins program execution with a delay with respect to the other receiver (note that this a problem even when there is only one clock driving the system). Now, with non-coherent receivers, there is a clock drift component not present in coherent receivers. That means that the ADCs (driven by different clocks) are going to drift away from each other due to manufacturing error and environmental effects, etc. Correcting for this drift adequately allows one to calibrate one channel's received signal with another.

My vague understanding of the reason this calibration doesn't adequately deal with the frequency and phase offsets of the two distributed receivers' clocks, however, as Whiting vaguely alludes to, is related to your comment about receiver frequency versus ADC frequency. In order to actually record any samples at the ADC the RF signal must first be mixed by the RTL-SDR's circuitry's local oscillator (LO). That LO has a frequency that is not exactly known (it drifts because it is driven by the same clock that drives the ADC), and that means that the frequency differences between the LOs of the two RTL-SDRs is also not known. What's more, the phase of each of the LOs at a given time is also unknown. So the timing corrections for bulk offset and sampling drift don't help with LO frequency or LO phase. One thing I'm still confused about is why the ADC clock timing drift calibrations don't help in the calculation of LO frequency drift, since they ought to be related (the drifts in both cases are caused by the crystal oscillator).

Maybe a hint to this answer would be in the Bulk Delays section, where "even in systems with two ADCs sharing a clock, sampling does not typically start at the same time" which may indicate a multichannel coherent receiver, and not distributed unconnected receivers for this section.

I don't think that's right. This quotation is referring to the fact that even coherent receivers need to have calibration for bulk offsets even though there is no clock drift. Whiting is mentioning that his discussion of timing calibrations are potentially useful for coherent receivers in addition to not instead of non-coherent receivers.

Also, why would we want to correct phase offsets at receivers? Are phase offsets not necessary for calculating AoAs at coherent receivers in conventional beamforming/MUSIC and phase differences between ground nodes in PDOA?

The key is that we need to calibrate the "heartbeat" of the distributed receivers, not the received signal. The "heartbeat" here is the LO. The LO is mixed with the RF signal, and it's that mixing stage that allows us to actually sample the RF wave. Without the LO we can't measure anything [unfortunately, I do not fully understand exactly how this works, and I don't even know if the RTL-SDR is a super-heterodyne, heterodyne or homodyne receiver, which all mix the LO with the signal in importantly different ways]. Because the mixer is mixing the true RF signal with the LO, the phase of the LO (which is initially unknown) makes the true phase of the RF signal unknown as well. It comes down to this, when we start the LOs of the two receivers we don't know what the initial phases of either of the LOs were. Even if the frequencies of the two LOs were exactly the same, and didn't drift, they would still have different initial phases (i.e. they might have started [and then remained] 23 degrees out of phase). If we pretended that the phase offset between the LOs' waveforms was zero, then when we mixed our received signal, and measured a 80 degree phase offset between the two signals, we'd be miscalculating the true phase difference between them. The reason is that we didn't calibrate for that initial 23 degree offset of the LOs. The received signals, with a measured phase offset of 80 degrees, would actually have a phase difference of 57 (or 103) degrees.

It's vitally important to understand this concept before we continue, since it's relevant to all of the calibrations we need to do. I'd recommend reviewing a basic introduction to hetero/homo-dyne receivers.

3. Inter-ground-node communication

It is not clear how information is transmitted between unconnected receivers in the distributed network... Are transmissions to a central basestation timestamped to avoid this problem?

Absolutely. For distributed nodes the data has to be timestamped and sent to a central processing station where the DSP happens. The millisecond delays of logging the data in a buffer before sending it, much less the additional delays of sending it to an output buffer, and retransmitting the data, would obliterate the minuscule timing information. Studying Kruger's TDOA system (as well as McCurdy and Gabrielson's Cornell system, described in the Handbook of Radio Direction Finding) should help clarify how this is achieved. To be sure, distributed nodes require a lot of data to be sent to a central node, and that will be an important constraint.

4. Do we need PRN codes if we have a calibrated system?

If bulk timing delay, clock drift, and phase/frequency offsets are reliably corrected among distributed ground nodes using the methods entailed in this paper, will we still need PRN codes to synchronize ground nodes and collect accurate measurements in PDOA?

Yes.

Regarding calibration:

The PRN codes are not only useful for assisting in cross correlating the actual signals, they are likely to be extremely useful in calibration, for precisely the same reason. Whereas the use of white noise within our coherent receivers (i.e. between the channels of the same ground node) will likely be sufficient to calibrate the 3-4 channels, the calibration of the different ground nodes will require a signal that can be used in cross correlation, and the PRN codes excel in this.

Regarding the signal design:

The fully calibrated and synchronized system will still require signals that are themselves easily timestamped (i.e. that can be cross correlated to establish an ~exact time and phase of arrival). The unambiguously best signal for that is a PRN code, though we still need to better understand the requirements for the measurement of PDOA, and whether PRN codes generated with OOK are really compatible (I think they are).

Summary

These are all really important questions you've brought up, and we're reaching (or have reached) the limits of my confidence with this material. I'd strongly urge you to try to find someone who is competent in RF electronics (e.g. Rich Gabrielson, Hunter Adams, etc.) who can help you (and indirectly help me) to understand this. They might be able to recommend introductory reading materials that we can peruse as well.

My vague understanding of the reason this calibration doesn't adequately deal with the frequency and phase offsets of the two distributed receivers' clocks, however, as Whiting vaguely alludes to, is related to your comment about receiver frequency versus ADC frequency. In order to actually record any samples at the ADC the RF signal must first be mixed by the RTL-SDR's circuitry's local oscillator (LO). That LO has a frequency that is not exactly known (it drifts because it is driven by the same clock that drives the ADC), and that means that the frequency differences between the LOs of the two RTL-SDRs is also not known. What's more, the phase of each of the LOs at a given time is also unknown. So the timing corrections for bulk offset and sampling drift don't help with LO frequency or LO phase. One thing I'm still confused about is why the ADC clock timing drift calibrations don't help in the calculation of LO frequency drift, since they ought to be related (the drifts in both cases are caused by the crystal oscillator).

As of now, I cannot think of any other component inside the RTL SDR that contributes to phase/frequency differences between distributed receivers. Furthermore, frequency/phase offsets among unconnected RTL-SDR local oscillators is not only possible, but extremely likely. The ADC clock timing drift calibrations do not help in the calculation of LO frequency drift because the I/Q values themselves are being compared in calibrating the LO frequencies, and not the delays of the samples. Referring to Figure 6 in the paper, the delays between the samples themselves (faster/slower data) is calibrated in the clock timing drift procedure. Now think about the I/Q values inside those circles. These I/Q values of data streams (the reference signal at multiple channels) will have different fluctuation rates, and an offset, if the local oscillators between the distributed nodes are not synchronized.

I don't even know if the RTL-SDR is a super-heterodyne, heterodyne or homodyne receiver, which all mix the LO with the signal in importantly different ways].

The RTL-SDR is a heterodyne receiver. This hyperlink contains a useful overview of the RTL-SDR, including an explanation of the R820T tuner and a frequency domain representation of the local oscillator acting on the amplified antenna signal.

The PRN codes are not only useful for assisting in cross correlating the actual signals, they are likely to be extremely useful in calibration, for precisely the same reason.

I completely agree. Just to make sure we are on the same page, the alignment of a PRN code with a local template code at a receiver improves the accuracy of arrival times at separate receivers and naturally establishes a common time base among receivers (each receiver will determine a signal arrival time at the same location in the signal transmissions (i.e. the location of the 1,0,1,1,0 code in the signal)).

Furthermore, the time and frequency corrections proposed by Whiting et al. primarily aid with the aggregation of arrival times from different receivers at a central basestation. The problems that occur with aggregating samples at a central basestation include:

Bulk Delays - What is the difference of sampling start times between the ADC's of separate receivers? This is solved by a long cross correlation between the channels.
Clock Timing Drift - How separated are the sampling frequencies between multiple receivers? This is determined by interpolating the stream with the fast sampling rate to the stream with the slower sampling rate using fractional delay accumulations between the streams.
Phase Offset - What is the phase offset of a common reference signal between receivers? Again, this does not refer to a sampling offset (how far behind an ADC is from another ADC), but an offset related to the I/Q samples (how far behind one local oscillator is from another local oscillator). This phase offset is determined by conventional beamforming (also known as Bartlett beamforming).
Frequency Offset - How separated are the local oscillator frequencies between multiple receivers? The procedure used to to determine the fractional phase accumulations used in interpolating the faster stream to the slower stream is analogous to the clock timing drift procedure. Instead of using fractional delays, fractional phase differences are used to determine the frequency correction term.

Excellent! Thanks for this information. I have a few follow-up questions:

As of now, I cannot think of any other component inside the RTL SDR that contributes to phase/frequency differences between distributed receivers.

Is this in reference to my explanation of the LO as a source of phase/frequency offsets? I can't quite tell if you are in agreement with that explanation or not. Are you saying that you agree that the LO is likely the source of these offsets?

Furthermore, frequency/phase offsets among unconnected RTL-SDR local oscillators is not only possible, but extremely likely.

Just for the sake of accuracy, we should upgrade this from "extremely likely" to "certain." Frequency/phase offsets will occur, and they have to be compensated for.

The ADC clock timing drift calibrations do not help in the calculation of LO frequency drift because the I/Q values themselves are being compared in calibrating the LO frequencies, and not the delays of the samples. ... These I/Q values of data streams (the reference signal at multiple channels) will have different fluctuation rates, and an offset, if the local oscillators between the distributed nodes are not synchronized.

I think I understand you here, and although I don't fully understand the underlying math I believe I agree with you. The reason I thought the clock drift and bulk offset calibrations might help in frequency/phase offsets is that the LO frequency is determined (through multiplication or division) from the underlying clock signal that determines the ADC, so calibrations dealing with the clock ought to provide information that could be used in calibrating the LOs, but this information is clearly not sufficient for full calibration.

Just to make sure we are on the same page, the alignment of a PRN code with a local template code at a receiver improves the accuracy of arrival times at separate receivers and naturally establishes a common time base among receivers (each receiver will determine a signal arrival time at the same location in the signal transmissions (i.e. the location of the 1,0,1,1,0 code in the signal)).

Without the PRN code calibration signal there would be no timebase (unless the receivers are "GPS disciplined") so I think it's more accurate to say that the alignment of the PRN codes enables the measurement of arrival times rather than that it "improves" it. Regarding the establishment of a common time-base, I think more is necessary than the simple cross-correlation of the signal with the template (for distributed receivers). The calibration signals will have to travel different distances to different receivers, so we cannot conclude that all the receivers have a common time base once they've all aligned their clocks to the calibration signal. With a very high precision estimate of the location of the timing beacon the hypothetical travel times to different receivers should be calculable; perhaps this is how timing beacons work?

I believe a similar problem applies for GPS disciplined receivers (different receivers will receive the same GPS signal at different times). I think the difference, however, is that the GPS signal itself contains an accurate timestamp, which further assists in calibrating distributed receivers.

Furthermore, the time and frequency corrections proposed by Whiting et al. primarily aid with the aggregation of arrival times from different receivers at a central basestation.

I am not certain I understand what you're saying here. Are you suggesting that I/Q samples are being streamed to a central base station from multiple receivers, or timestamp and calibration information is being streamed, and then further DSP is conducted on the central hub? I believe Whiting's work assists in calibration with the latter setup, and that the former setup cannot work (simply given the bandwidth that would be required for a hub to receive and process multiple data streams in real time). I'll admit I still don't understand fully how this can be done efficiently, but to be sure Whiting's paper is primarily geared toward calibrating distributed receivers that are not driven by the same clock, and (I'm assuming) also not sharing the same ADC or DSP.

Regarding your list of 4 calibration types:

Bulk Delays Agreed, though what do you mean by channels? If we are talking about the different channels of a coherent ground-node, the whole cross correlation process will occur between different within-node channels, however for distributed nodes, the cross correlation of the PRN calibration signal with the template will occur at each distributed node, the results sent to the central hub, and then further calibration done at the hub. Correct?
Clock drift Agreed, though, again, how this is accomplished with distributed nodes is still unclear to me; we cannot send I/Q samples to a central hub in realtime, so some pre-processing has to occur beforehand.
Phase offset I'll have to look into Bartlett beamforming; I have no knowledge of this. So, the phase offset of the LO is calibrated using Bartlett beamforming on a calibration signal, and then this offset is subtracted from the measured phase difference of a wildlife transmitter signal?
Frequency drift I have the same question about this as above: how do we achieve this synchronization without distributed ground-nodes sharing a DSP that has access to all of the I/Q data from the different receivers?

jakapoor / AMRUPT

Questions concerning time and frequency corrections in a distributed network #16