Implementing DCF77 library DHT22 reading.

[Avien169]

Hello, I would like to thank you for your work on DCF77 libraries. They work brilliantly! I have replaced resonator on my Arduino Uno board with an oscillator, tweaked yours examples and it is working perfectly.

In my hobby project I am trying to make a VFD clock. There would be additional features like showing temperature and humidity. To that I am using DHT22 sensor based on one-wire communication.

From what I have noticed, after starting DCF77 decoding process, delay() function (and more timebase) change it's actual execution time. I was trying to use this library https://github.com/niesteszeck/idDHTLib and other one which can be found inside Adruino IDE.

Could you guide me a little into the solution? I think I have tried anything, but I cannot overcome time based problems :/

Best regards and thank you for your time, Avien169

[udoklein]

How much do they change their actual execution time? Until you tell me I have to guess what is going on. Here is my preliminary analysis. In wiring.c I find:

void delay(unsigned long ms)
{
    uint32_t start = micros();

    while (ms > 0) {
        yield();
        while ( ms > 0 && (micros() - start) >= 1000) {
            ms--;
            start += 1000;
        }
    }
}

and in wiring.c I find

unsigned long micros() {
    unsigned long m;
    uint8_t oldSREG = SREG, t;

    cli();
    m = timer0_overflow_count;
#if defined(TCNT0)
    t = TCNT0;
#elif defined(TCNT0L)
    t = TCNT0L;
#else
    #error TIMER 0 not defined
#endif

#ifdef TIFR0
    if ((TIFR0 & _BV(TOV0)) && (t < 255))
        m++;
#else
    if ((TIFR & _BV(TOV0)) && (t < 255))
        m++;
#endif

    SREG = oldSREG;

    return ((m << 8) + t) * (64 / clockCyclesPerMicrosecond());
}

Obviously delay() is busy waiting for micros to advance far enough. So I would assume it appears as if micros() deliver "longer" microseconds. This would explain the issue. But why would this happen?

In wiring.c I also find

// the prescaler is set so that timer0 ticks every 64 clock cycles, and the
// the overflow handler is called every 256 ticks.
#define MICROSECONDS_PER_TIMER0_OVERFLOW (clockCyclesToMicroseconds(64 * 256))

So 64 * 256 ticks per timer 0 cycle (or 1.024 milliseconds). Now my clock library may sometimes use more than 1 ms in one cycle. This adds negligible jitter to the sampling. However it may happen that in such cases timer0 overflows twice while interrupts are blocked. As a consequence one of the timer0 interrupts gets lost. It follows that delay and friends will slow down.

Now suppose my library would not cause this behaviour but would block only n microseconds with n below 1000. You would still experience timing jitter of up to n microseconds. What I am after is: delay() and friends are not suitable for anything that requires any kind of exact timing.

Conclusion: do not use delay(). The question is: what are you trying to achieve with delay() that could not be better handles differently? Tell me why you need delay() and we can find a better solution for you that does not require delay() in the first place. The proposed solution will go in the direction of my library providing precision ticks anyway.

[udoklein]

One more comment. In your library I find

        // REQUEST SAMPLE
        pinMode(pin, OUTPUT);
        digitalWrite(pin, LOW);
        delay(18);
        digitalWrite(pin, HIGH);
        delayMicroseconds(25);
        pinMode(pin, INPUT);

        // Analize the data in an interrupt
us = micros();

So I know the piece of code that is probably affected. However I do not understand why you need delay() for it and why it is crucial that delay() does not exceed the desired timing. Can you elaborate on this detail a little bit?

[udoklein]

I had a look into the datasheet. I think the occasional increased lenght of the delay() should not be an issue because the datasheet says to pull down for AT LEAST 18 ms. However the delayMicroseconds part is critical because it might get interrupted for several 100 microseconds, sometimes more than a millisecond. You could block interrupts but then this might throw my library of the tracks. The bottomline is that my librarie applies advanced signal processing and this takes its toll in the form of heavy CPU utilization.

I suggest to implement a mechanism where you find out after the sampling if you properly got the desired values and discard any readings where an interrupt messed with your results. I think this would be the easiest solution. That is least complex and least side effects.

Another option would be to go for my superfilter approach and dedicate a controller for the dcf77 decoder. This would offload the processing to another controller. Then you could use one of the dumb dcf77 decoders which does not create such timing issues.

One more option is to avoid AVRs for such processing heavy stuff and switch to the ARM platform. This has so much more power that these effects can be avoided. In particular it features leveled interrupts / interupt priorities which makes handling this whole set of challenges a lot easier.

[Avien169]

After having DCF77 synchronize I was trying to make one readout from DHT22 each minute (setup ISR), then after couple seconds I am making "check status". Every time there is a time-out error :/ (there is no valid data) which probably comes from micros(); I have checked signal line between Arduino and DHT22 with an oscilloscope, and with every ISR setup there is a communication handshake. I am thinking about disabling DCF77 synchronization every 20-30 seconds (or more up to minutes) to make a readout from DHT22. Will it easily synchronize? Or maybe make it synchronize each hour?

Please let me know what do you think about this solution.

Clearly now I understand why ARM should be used, but I couldn't drive myself out from buying cheap arduino nano.... ;)

Again thank you very much for your help.

[udoklein]

The algorithm relies on a precise local clock. Suspending it has the same impact as slowing down the clock. So if you suspend it for maybe a millisecond each minute this would have no impact. Suspending it for a millisecond each second would be the equivalent of a crystal with more than 1000 ppm deviation. This is outside the tuning range and would not work.

If you need the noise tolerance and do not want to switch to ARM I suggest the "superfilter" approach.

[Jo-Achim]

Hello Avien169, here an additional idea to get a delay without delay(). If you look in Adafruits NeoPixel library, you will see inside the "Adafruit_NeoPixel.cpp" a lot of NOPs to create the correct timings to program the pixel-controller inside the LEDs. I don't know if it helps with your DHT22, but if the needed delays are independent minimum values (that can be interrupted)...

Hello Udo, if I understand you're DCF77-project and your accomplishments above correct, then simplified, your Super_Filter creates a 1-Hz-PLL to synchronize with the DCF77-1-Hz-signal. Therefore, to adjust your PLL, you use timings divided from Arduinos 16 MHz crystal clock. Correct? The spontaneously question was, if the filter get work with only one precision external 32-kHz-signal, too? If this was possible, the benefit should be, that your Super_Filter can also work on 'non-crystal-based' Arduinos and projects with your Super_Filter have additionally a power backup for date and time – the needed accurate 32-kHz can come from an RTC like DS3231 (ZS-042); the internal RTC oscillator can be stopped and started to synchronize with the PLL / DCF77-signal. Is that right or an error of reasoning?

Best regards, Joachim.

[udoklein]

Yes, it would be possible to work with an external 32768 Hz reference like the DS3231. The following would need to be done:

1) Remove the autotune / frequency control stuff as it is not needed. 2) change the 1 Khz generator to be driven by the external timer interrupt and adapt its timing constants to 32768 Hz operation.

However it adds another part (the DS3231) and is both more complex to setup. Also it is more expensive than buying a cheap crystal based Arduino clone in the first place. Thus I did not implement this.

[Jo-Achim]

Ok and thank you. The idea behind was, that a lot of used Arduinos (Uno, ..., MEGA2560) are 'Ceramic-based-only'.

[udoklein]

Of course. But they lack oboard DS3231 so you either need external components or go for a suitable board in the first place. I am fully aware of this cost cutting design decision of Arduino. IMHO pretty bad given the fact that quite a lot of the knock offs still provide a crystal.

[brettoliver]

Here is a quick mod for Ceramic based UNO. Shows how to replace the resonator with a crystal. http://home.btconnect.com/brettoliver1/Master_Clock_MK2/Master_Clock_MK2.htm#Mod

http://home.btconnect.com/brettoliver1/Master_Clock_MK2/Master_Clock_MK2.htm#Mod

[udoklein]

Definitely easier and cheaper than adding a DS3231. Also does not eat up an IO pin. In my opinion this is the proper way to go.

[udoklein]

Or while you are at it - even better - replace the resonator by a 16 MHz TCXO or OCXO.

[Jo-Achim]

Hello Udo, hello Oliver, the background of my question about 32-KHz and MEAG2560 was the following. I use a crystal-based MiniPro with the 'Simple Clock with Timezone Support' for my project. One step before, I used an additional Nano to do the rest, like handling an OLED-Display, control my LED-Stripe with Adafruits library and so on. Interesting to check if it was possible to load all parts onto one MiniPro, I find out, that it was possible when using the Digi-Dot-Booster instead of Adafruits NeoPixel library to control the LED-Stripes. But in first view only. The second view shows a 'Synced'-/'Locked'-problem; after first DCF77-sync, I get 'locked' only. Why? Was it a possible bad DCF77-signal quality from Conrad-receiver?

I think my MiniPro is overloaded. If I use a second crystal-based MiniPro with your 'Simple Clock with Timezone Support' adding some digitalwrites to show the signal quality with LEDs and connecting the second MiniPro at the same DCF77-receiver, I will see most of time 'synced'; parallel the 'overloaded MiniPro' shows a 'locked' instead.

So I was interested, if it was a better solution to use e.g. one MEGA2560 (with more RAM) or to go back and split the program for two Arduinos again (possible prevent trouble with other libraries).

At this point, I think that one crystal-based MiniPro (for the DCF77-library) and a second Arduino (Uno with prototype shield or so on) for the rest are the best way.

So thanks - and best regards, Joachim.

[Jo-Achim]

Please look in conjunction with my Post above in "Good Hardware, bad Hardware": https://github.com/udoklein/dcf77/issues/20.