Auto fine tune with new driver

[davidwkerr]

This is a suggestion for Romemfr when he writes a new, more table based and specifically targetted driver.

The current calibration function works well and is a great feature. It delivers a CF within about 3kHz of the required CF.

The calibration routine of dwarning which sets the clb values and henceforth FSCTL0/FSCTRL1 and other things is useful. However it is specific to one set of hardware and correct at one point in time.

I have 4 Micronet devices (Dual digital display, analogue wind, wind transducer, Triducer). I have found that they all have different CFs ranging over 25kHz (about 30ppm). This is not surprising because the best small SMD crystal I can buy is 10ppm tolerance with a total variation of +-20ppm over temperature and time. Also, because of the use of an early Ti chip, the Micronet system does not seem to have implemented any sort of frequency adaptation. Indeed, it could be somewhat challenging to do that with the CC1000.

The CC1101 provides a frequency offset estimate through register FREQEST. This value is fully compatible with register FSCTRL0 and can be used to tune the CC1101 in real time. Therefore, in theory, you can use Rodemfr's tuning algorithm to set the approximate CF which will apply to one device on the network. You can then use the FREQEST register to select the optimum CF for all the instruments. So, will it work? I believe the answer is "YES"!

I have hacked the ELECHouse driver to remove all the clb code. So, the default initialisation is with FSCTRL0=0 and FSCTRL1 set to the value recommended by the Rf Studio. As this is the IF frequency, this is totally correct. I have made a new function which gathers the FREQEST after each successful receive and puts it into an array of 40 values. I started with 10 values but found that with 4 devices, you got bias depending how many packets were seen from each device in a set of 10. So now there are 30 values and I calculate a running average of all these and apply it to the FSCTRL0 register to fine tune the RF dynamically. The resolution of this is +-1.6kHz which means the CC1101 CF is set at the arithmetic mean of the 4 devices. I can see the CF changing by small steps on the spectrum analyser and the range performance is the best I have had so far. I am putting the instruments into a sealed stainless steel pressure cooker (<>) one at a time and can observe the CF changing to focus only on the remaining devices. There is a small amount of frequency change in a static environment probably due to the end cases of 4 frequencies in the array of 40 and bit errors in the FREQEST, but it is small (+- one bit). One could choose a median rather than average or some other statistical function but I doubt there is any point.

The above mechanism must be disabled when doing the Menu item 6 calibration, which should be done with FSCTRL0=0. I put in a flag to handle this.

I hope something like this can be part of the reworked driver when it comes. I'll continue testing it and refine it further if necessary.

[Rodemfr]

Tuning dynamically CF with FREQEST is a great idea, let's do it. I hope to start it soon.

[dwarning]

I am sure you read the attached DN015 to use accumulated offset values. DN015_Permanent_Frequency_Offset_Compensation.pdf

[davidwkerr]

Yes- thank you. I read it just after I did the work. It doesn't change anything. I had thought of storing the average into EEPROM but I prefer to start with FSCTRL0=zero every restart in case something went wrong. I also thought about discarding "noisy values". However with a circular buffer of 40 values (I started with 10, then 30, then 40), an occasional noisy value does not do much harm. The time constant is of the order of 10 to 20secs, depending upon number of devices. But I might screen out values that would pull the CF more than (for example) 50kHz. But a lot of that sort of experimentation & filtering could be done after implementation of a new driver. Having read the Datasheet, I wanted to try the concept. The good news is that the concept works, saves effort and makes it more likely that implementors will succeed. I do the changes after the code which gets the RSSI. So another thing could be to only do the freq change if the RSSI is above a threshold. But, in practice, I have found the FREQEST does not jump around much.

[davidwkerr]

I've done quite a lot more testing and observation of FREQEST by deliberately moving devices further away and changing the CFREQ in the code away from optimal. It all converges very nicely to within +-1.6kHz and there is almost no "jitter" of the CF. I can start up to about 60kHz off frequency and still converge. So as long as a menu 6 scan has been done, all is good or even if not done and the coded frequency is within +-60kHz.

[dwarning]

@davidwkerr: Can you provide a simple testcase which you are using to test the FREQEST approach. Thanks.

[davidwkerr]

As I said, I hacked the code somewhat. I did not want to make a new branch because I saw these changes as beneficial to include in the new driver. However, if you want to try it out, here are the instructions which are quite simple. I have not provided diff files because I have changed other things like the programming of DEVIATN (described eleswhere) and sorting through various other changes might cause confusion:

1. Create new function in the ELECHOUSE driver:

define NUMBER_SMOOTHS 40

uint8_t smooth[NUMBER_SMOOTHS]={0}; //DK- Smoothing table for centre freq int8_t smoothp=0; //pointer for smoothing bool frequency_flag=true; //flag to disable offsetting during tuning

// //Reads and saves the estimated frequency deviation //after each read, into a circular buffer. //We take the arithmetic mean for our Centre Freq but the median //could also be used to give less weight towards the majority of //clumped Micronet transmitters. // void ELECHOUSE_CC1101::Squeeze(void) { int8_t i; int16_t estim; //Total of values in circular buffer uint8_t j,k; //In case of non-optimizing compilers if(!frequency_flag) { //We accumulate no data during tuning SpiWriteReg(CC1101_FSCTRL0,0x00); //end ensure no offset return; } smooth[smoothp]=SpiReadStatus(CC1101_FREQEST); //Get the frequency error estimate

if(++smoothp >= NUMBER_SMOOTHS)smoothp=0;      //circular buffer
estim=0;
for(i=0;i<NUMBER_SMOOTHS;i++)                               //Sum them all
    {
    j=smooth[i];                                                                 //We could use a Union instead....
    k=~j+1
    if(j >= 0x80)estim-=(int16_t)k;    //convert our twos complement
    else estim+=(int16_t)j;
    }

    SpiWriteReg(CC1101_FSCTRL0,(uint8_t)(estim/NUMBER_SMOOTHS));       //Program the offset in 2s complement
    return;

}

2. New prototype in ELECHOUSE header:

void Squeeze(void);

3. Around line 178 of RfDriver, add call to the new function, immediately after call to get RSSI:

cc1101Driver.Squeeze(); //DK addition. Get and process FREQEST

4. In void ELECHOUSE_CC1101::Calibrate(void)

Comment out three lines in the section that refers to your CF (presumed 868MHz, mine is 916MHz):

else if (MHz >= 900 && MHz <= 928){ //SpiWriteReg(CC1101_FSCTRL0, map(MHz, 900, 928, clb4[0], clb4[1])); //DK removed SpiWriteReg(CC1101_TEST0,0x09); //int s = ELECHOUSE_cc1101.SpiReadStatus(CC1101_FSCAL2); //DK removed //if (s<32){SpiWriteReg(CC1101_FSCAL2, s+32);} //DK removed if (last_pa != 4){setPA(pa);} //FSCAL2 remains the default programmed from RfStudio }

5. In MicronetToNMEA cpp file, add:

extern bool frequency_flag; //Added DK- our global to enable/disable the tuning algorithm

then around line 862 immediately after the console print that tuning is starting insert:

       frequency_flag=false;      //Turn off the auto calibration   

then around line 937, immediately before each of the two "exitTuneLoop = true;" lines, then insert:

            frequency_flag=true;      //Turn on the auto calibration

****Note that these could be moved saving one line of code.

[dwarning]

I made a check with receipt above. Everything works well, I made similar observations like you by shifting the MICRONET_RF_CENTER_FREQUENCY_MHZ and looking for estim/NUMBER_SMOOTHS*1.59 kHz. I am afraid that the long smooth filter will bring problems in the time schedule. In my opinion lower values of NUMBER_SMOOTHS brings not worthier results. Still open for me is the statement in the design note DN015 to use accumulated values in the update of the FSCTRL0 register. Accumulated value in the meaning of update value before and actual.

[davidwkerr]

Hi, Thanks.

1. Yes, you could store the derived FSCAL0 value in EEPROM. There are some good reasons and some less good reasons. But I really do not mind one way or the other.
2. The time constant I have come to (40 values) is a reasonable compromise. I have not seen any negative effects. Shorter is not so good and I am not sure there is benefit in longer. I have two displays, one wind anenometer, one Triducer. Some people will have more devices and some less. Some will have more disparity of CFs and some have less.
3. There are some arguments for using median, not mean. But either is probably more than satisfactory.

[dwarning]

Sorry, you meant FSCAL0? Or FSCTRL0?

[davidwkerr]

Sorry, typo, I meant FSCTRL0.

[dwarning]

OK, only a recommendation: I would not comment out the clb code in the Calibrate routine. Your approach is working well also if the FSCTRL0 is set by the user of the lib. Sure, the FSCTRL0 value will be different but this doesn't matter.

[davidwkerr]

I see no point in keeping the clb code. Just makes the new driver more complicated than it needs to be. Rodemfr can choose but I would leave it out and make things simpler. No need for users to do any separate calibration to obtain clb values. Getting rid of that stuff was one of my motivations for trying the FREQEST approach.

[Rodemfr]

Hi @davidwkerr , I finally implemented the FREQEST algorithm. Just as you mentioned with a small difference : I kept the FSCTRL0 calculation of CC1101 driver because I noticed a too wrong initial value can kill reception of Micronet packets, making the FREQEST algorithm dead also since it only runs on packet reception. The implementation is in branch "continuous_freq_correction". Let me know if this runs properly on your setup.

[Rodemfr]

By the way, I derivated this branch from the "low_power_cc1101" branch so you will also get a large number of other modifications which might cause troubles...

[davidwkerr]

Hi, I hope you are safe from the fires and canicule. I ran my modification (hack) with four different cc1101 modules (two cheap crystals and two replaced good ones). After MicronettoNMEA ran perfectly for several weeks, I made a whole lot of personal modifications to also handle differential GPS in and out as well as to output UDP packets and merge incoming packets over UDP. So quite a few changes plus I also made a backplane board to fit into a box for my boat. Attached is a poor photo of the working board. I also installed the MHU unit up the top of the mast. I do not think my recent changes will be useful to many people other than myself. So, it is a bit hard to test the new branches at the moment. When I get time, I'll make up a new system and try to do some testing. The great news is that your project has been running perfectly on my system for some weeks now without stopping! So I think I shall be happy with it on the boat. Best Regards, David

On Monday 18/07/2022 at 1:39 am, Rodemfr wrote:

Hi @davidwkerr , I finally implemented the FREQEST algorithm. Just as you mentioned with a small difference : I kept the FSCTRL0 calculation of CC1101 driver because I noticed a too wrong initial value can kill reception of Micronet packets, making the FREQEST algorithm dead also since it only runs on packet reception. The implementation is in branch "continuous_freq_correction". Let me know if this runs properly on your setup. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[Rodemfr]

I'm far from fires and I'm now somewhat accustomed to extreme temperatures in Toulouse. We are passing 40° each year. This is however more frequent and longer each year. France will look like more and more Australia... A sign that the end of the world is near : there will be 40° and a shiny sun in Brest today !

No problem with FREQEST testing, I will use my RTL-SDR to test it on my side. In the recent changes, there is one thing that could interest you : power consumption reduction. My setup switched from ~1000mW to ~600mW with these changes. Depending on the CPU power requested by your own changes, you could be able to reduce CPU speed to 24MHz instead of 120, it will already significantly reduce current draw without any SW modification.

By the way, Github dropped your attachement.