Tiny85 delays very late issue

I am developing an IO process which can switch on LED by Hall effect sensor and 
delay 500ms to turn off after I un-press it just like 555 monostable can do. I 
am using UNO R3, Arduino 1.0.5, Board is Tiny 8Mhz internal BOD off to program 
a tiny85. But issue occurs as below.

1. Where is a very huge delay at the 1st time around 16-20 seconds. Next click 
is around 8-10 secs. After few time to activate the process it becomes normal 
(500ms).
2. Then I leave it alone for a while (let say 30 secs), the 1st delay time is 
back to 16-20 secs.

I am just thinking if the sleep function is enable by default cause this false, 
or if it is just a stupid fault by myself. If so, can anyone help to solve this 
problem. Thank you very much.

Chris

//////////////////////////////
const int ledSensorPin = 3;
const int led_mosfet = 2;
int ledHallState = 0;
int ledLastHallState = 0;
long previousMillis = 0;
long interval = 500;
int startCountDown = 0;
//////////////////////////////
void setup()
    {
    pinMode(led_mosfet, OUTPUT);
    pinMode(ledSensorPin, INPUT); //pulled up by hardware
    }
//////////////////////////////
void loop()
    {
    unsigned long currentMillis = millis();

    if (digitalRead(ledSensorPin) == 0) ledHallState = 1;
    if (digitalRead(ledSensorPin) == 1) ledHallState = 0;
////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////
    //LED MOSFET SWITCH ON
    if (ledHallState != ledLastHallState && ledHallState == 1) // Save current state as the last state
        {
        ledLastHallState = ledHallState;  // Save current state as the last state
        startCountDown = 0;
        digitalWrite(led_mosfet, HIGH);   // LED on
        }
    //LED MOSFET SWITCH OFF
    if (ledHallState != ledLastHallState && ledHallState == 0)                  // Save current state as the last state
        {
        ledLastHallState = ledHallState;
        previousMillis = currentMillis;
        startCountDown = 1;
        }
    if (startCountDown == 1)
        {
        if(currentMillis - previousMillis > interval)
            {
            ledLastHallState =ledHallState;
            digitalWrite(led_mosfet, LOW);
            startCountDown = 0;
            }
        }
    }
Original issue reported on code.google.com by taichauc...@gmail.com on 12 Feb 2014 at 5:55
adityanagal / arduino

Tiny85 delays very late issue #1134
