LIN_ADVANCE freeze

[nzinov]

I am running d3cb1a86 on delta. When I try to print with LIN_ADVANCE uncommented, printer moves to the start point, then extruder produces some noise for ~1sec and then printing completely stops. Hotend led is blinking at changing rate as usual - so firmware is not crashed at that moment. However, it only outputs busy: processing into console and is not responding to any commands. Any tips on debugging that?

[psavva]

Can you lower your print speed, say to 35mm/s…?

[mottihoresh]

I think the print/move speed may cause it, I am experiencing the same issue. Not sure what's going on. It's not happening with RC8BugFix branch

[Sebastianv650]

@mottihoresh there is no branch "RC8BugFix" - do you mean the code base tagged with "1.1.0-RC8"?

@nzinov can you post the gcode section +- some lines around the start point where it freezes? A test with a low speed as the noted 35mm/s can't hurt also, maybe it helps to find the reason for it. You might also try RCBugFix at the state without my latest LIN_ADVANCE change.

If somebody knows other issues mentioning such a behaviour, please give me a hint on them.

[mottihoresh]

@Sebastianv650 Meant to write RCBugFix (https://github.com/MarlinFirmware/Marlin/tree/RCBugFix)

I actually spend couple of hours yesterday reading through the issue log. my best guest of what is going on is that the ATMega MCU just chokes since it's trying to process data too fast.

I got non print moves at 160mm/s, and 32 steps per minute, using the new bilinear bed leveling, print speed are at 60 or 80mm/s. When reducing the global feedrate to 50% i noticed no issues, however as soon as I switched it back to 100% the MCU entered panic mode and restarted.

I think this issue may be related: https://github.com/MarlinFirmware/Marlin/issues/4931

My configuration file: https://gist.github.com/mottihoresh/5f51c1d6c07d75c918992125b4e952ea

[nzinov]

@Sebastianv650 here is the begining of my g-code

M104 S235 ; set temperature
G28
G29
M420 S1 Z10
M109 S235 ; wait for temperature to be reached
G21 ; set units to millimeters
G90 ; use absolute coordinates
M82 ; use absolute distances for extrusion
G92 E0
M106 S178.5
;layer 0.4 0
G1 Z0.400 F12000.000
G1 X36.622 Y6.495 F12000.000
G1 X36.622 Y18.274 E0.72007 F1440.000
G1 X35.880 Y21.412 E0.91717
G1 X33.222 Y26.716 E1.27982

I am going to test lowering print speed a bit later. At this moment my issue is not quite like @mottihoresh. He says the MCU is restarted while in my case it apparently is doing a good job of maintaining temperature even after freeze.

[Sebastianv650]

I'm also thinking we have two independent problems here. So your printer is freezing after this line: G1 X36.622 Y6.495 F12000.000

Then I don't think it's the print speed, F1440 is 24mm/s which is already really low.

I have two problems: No knowledge about Marlin code related to bed leveling (as far as I know, your G29 means you are using it) and no knowledge about delta code handling.

What happens if you try a print without bed leveling, is this possible?

If yes, it would be interesting to know if it works without bed leveling and if yes, does it also work with LIN_ADVANCE enabled. This way it should be possible to get the problematic feature.

[nzinov]

@Sebastianv650 I've tried printing with bed-leveling turned off. Now it makes two circles of skirt with extruder producing strange noise (can't tell for sure but maybe step skip) and then stops in the same manner. Freezes outputting "busy: processing" not responding to commands from host (unfortunately I don't own an LCD)

[psavva]

@nzinov If you turn off LIN_ADVANCE, I guess you don't have the issue? right? Can you provide you config files in a pastebin, for both Configuration.h, and Configuration_adv.h? Lastly, If you're using 1/32 step drivers, I suggest to configure your control board to 1/8 or max 1/16 steps, and try the test again.

If lowering the step resolution resolves this issue, then I think it certainly has to do with draining the CPU, and thus failure at some point....

[nzinov]

@psavva Yes, without LIN_ADVANCE I have no issues (multi-hour prints went OK). I use Polulu drivers, so 1/16 steps max. Here lies my configuration: https://gist.github.com/nzinov/ea6544ce3df78a9438c81bf9b1a435a3.

Besides, I really doubt that CPU would fail to calculate moves but still happily chat to console and handle temperature

[Sebastianv650]

I will try to provide you a modified stepper or planner file in the next 1-2 days that outputs some debug data for each move. That should help to find the error.

[nzinov]

@Sebastianv650 OK, looking forward to it.)

[hexane360]

I'm having a very similar problem. Sending M905 K0 fixes the problem. I am also using ABL (linear).

I don't have a LCD, so I'm not quite sure what is happening with the firmware. Help debugging would be appreciated.

Additionally, my extruder motor gets very hot during the resulting fugue state, and sending M112 doesn't seem to have an effect.

[hexane360]

I've been playing around with various things, and the problem seems to be VERY weird.

Some general printer information: -Lulzbot Mini -Cartesian -ABL Linear -1/8 Microstepping (down from 1/16) -Latest RCBugFix

And here is the start G-Code that caused the problem:

G21                          ; metric values
G90                          ; absolute positioning
M82                          ; set extruder to absolute mode
;M200 D2.9175 T0 ; set filament diameter
M200 D0 T0; cancel volumetric extrusion
M107                         ; start with the fan off
G92 E0                       ; set extruder position to 0
M140 S55                     ; get bed heating up
G28                               ; home all
M109 S150                     ; set to cleaning temp and wait
G1 Z150 E-10 F75           ; suck up XXmm of filament
M109 S170                     ; heat up rest of way
G12  P1 S2                      ; wipe
G01 Z20 F700              ; raise off wipe pad
M109 S170                    ; set to probing temp
M204 S300                    ; set accel for probing
G29                          ; Probe
M204 S2000                   ; set accel back to normal
G1 X5 Y15 Z10 F5000          ; get out the way
M400                         ; clear buffer
G4 S1                        ; pause
M109 S190    ; set extruder temp and wait
M190 S55 ; set bed temp and wait
G1 Z2 E0 F75                 ; extrude filament back into nozzle
G21 ; set units to millimeters
G90 ; use absolute coordinates
M82 ; use absolute distances for extrusion
G92 E0
;layer 0, z:0.4
G1 E-1.00000 F600.00000
G92 E0
G1 Z0.200 F10800.000
G1 Z0.400 F10800.000
G1 Z0.600 F10800.000
G1 X64.958 Y65.166 F10800.000
G1 Z0.400 F10800.000
G1 E1.00000 F600.00000
M204 S500
G1 F1800

No amount of manually entering commands allowed me to recreate the problem. However, if I started the print with M905 K0 and then paused, entered M905 K25, and resumed, the print had no trouble. Eventually, I started stripping out extruder moves until I got it working. One of the following lines caused it to fail:

G1 Z2 E0 F75
G92 E0
G1 E-1 F600
G1 E1 F600

It seems like the culprit could be one of the following:

• Slow feedrates
• Sign of extrusion (- to + or + to -)
• Something else

Disabling retraction in Slic3r also removes the problem code. However, I don't think that's the root cause.

[hexane360]

@Sebastianv650 I've been poking through planner.cpp getting an idea how LIN_ADVANCE works. The method Planner::_set_position_mm seems to reset the position[] array. However, position_float is only updated after a move has taken place, using memcpy(position_float, target_float, sizeof(position_float));. Is it possible that the first planner move after _set_position_mm is called will use a stale position_float[]?

void Planner::_set_position_mm(const float &a, const float &b, const float &c, const float &e) {
  #if ENABLED(DISTINCT_E_FACTORS)
    #define _EINDEX (E_AXIS + active_extruder)
    last_extruder = active_extruder;
  #else
    #define _EINDEX E_AXIS
  #endif
  long na = position[X_AXIS] = lround(a * axis_steps_per_mm[X_AXIS]),
       nb = position[Y_AXIS] = lround(b * axis_steps_per_mm[Y_AXIS]),
       nc = position[Z_AXIS] = lround(c * axis_steps_per_mm[Z_AXIS]),
       ne = position[E_AXIS] = lround(e * axis_steps_per_mm[_EINDEX]);
  stepper.set_position(na, nb, nc, ne);
  previous_nominal_speed = 0.0; // Resets planner junction speeds. Assumes start from rest.
  ZERO(previous_speed);
}

There may also be other gaps similar to this. Looking through the file, there's a lot of places where position[] is set without position_float[].

[Sebastianv650]

I don't think that's a problem, but I can't say that for 100% because I don't know where all that set_position things are called. According to some comments, they are used for homing, probing and such things. That would mean they will not be active during the main print. But all freezes I know of happen during a print, not during homing?

I'm still having no idea where this problem comes from. Even with a gcode from another TAZ 5 that produces this issue, I can't reproduce it on my TAZ 5. It seems to be a very specific problem that happens only with specific configs combined with specific numbers in the gcode. At the moment my only hope is that some day somebody can provide a gcode snipplet that lets me reproduce the error.. :-(

[billyd60]

Sebastian, I have solved most of my issues with RCBUGFIX and it was mostly hardware related. I had a bad enclosure connector on my heated bed cable and the silicone heater was going bad as well. Crazy to have two problems so closely related. Anyway I have a new silicone heater and I am running the cables (power and thermistor for the bed) direct into the RAMBO with no breaks. This is actually amazingly good as I think I was getting some serious losses in power and accuracy with the two standard breaks (Enclosure connector and at the bed) that the Taz 5 has in this line. My bed has never heated so fast or been this stable. So I may have had a balky enclosure connector from day1

Anyway long story short I still get a freeze in printing from time to time (pretty rare though). Other than that lin advance and Rcbugfix are working great and producing the best prints I've ever gotten. The freeze is bizarre though, the printer just sits there, heat still going to the extruder and bed, lcd screen reading normally, just zero movement. Just happened again last night for the first time in weeks, printing in PLA. It seems to happen on large complex models. Maybe just because they take so long. Never have any trouble with small prints.

[Sebastianv650]

Glad to hear that at least some problems were hardware dependent!

When it freezes, is the LCD menu also functional, meaning it's possible to enter and navigate through it? If yes I will continue searching around the stepper ISR handler.

[billyd60]

I had to cycle power to get the extruder off the part (to use the prepare menu move z axis) so the LCD screen was not responsive to the dial control. The extruder and heat bed were still at temperature though, and the LCD screen looked normal in every respect and it was still reading the thermistor temps as well as x y and z coordinates which lined up with where the extruder was approximately (I didn't actually measure just by eye).

[Sebastianv650]

I still have no real answer, but you might try the following with a code that usualy crashes:

Inside void Stepper::ISRhandler(), comment out the lines containing sei() and cli(). Inside void Temperature::isr(), comment out the line containing sei().

This will undo some ISR handling changes I did to Marlin. You might see a lot of serial communication errors this way, so you might print at a lower serial baud rate for this test.

Second option: Undo the changes from above, but add a cli(); just above SBI(TIMSK0, OCIE0B); at the end of void Temperature::isr().

Both changes are "a shot in the dark" if my translator was able to translate what I mean ;-) But at least it's better than nothing..

[billyd60]

I only print from SD card never from USB.

[billyd60]

Sebastian I believe I have isolated the freeze problem. If I stop a print prematurely from the LCD panel and restart the job without cycling power to the RAMBO, the printer will ultimately freeze mid print at some point. It also cause odd sudden drops in the extruder temp and bed temp readings. None of this behavior occurs if I simply start the printer and print a job. Stopping the print from the LCD panel seems to cause a whole host of problems with RCBUGFIX.

All of this is just what I have observed in using the firmware, I can't prove any of it since I can't code.

I hope this puts you on the right path to find out why lin_adv freezes a print job sometimes.

[Sebastianv650]

Thanks, i will try to reproduce that!

[Sebastianv650]

I started a print from SD card, let it run for a minute or so and then aborted it. Restarted it, let it run for 1-2 minutes. No crash.

How long should it take until the freeze? As there is nothing "counting up" within the stepper or advance code, I have no idea why it should behave different when I let it run longer.. I had a quick look at the issue page (I wasn't activly reading it for some weeks) and there seems to be quite some freezing, temp. jumps and other issues with the recent code page. That make me believe the error might not be in LIN_ADVANCE specific because they are not mentioning it. Maybe advance only increases the probability the error shows up?

[thinkyhead]

I think there's probably some merit to the point that position_float and other additions for LIN_ADVANCE are not being set properly. Note that position itself is never updated until the end of _buffer_line. However the LIN_ADVANCE code updates it right away, at the top of _buffer_line. Likewise the E movement is skipped when in dry-run mode, but this is after the LIN_ADVANCE code is using it. So I think the LIN_ADVANCE code in _buffer_line needs some tweaks to make sure position_float[] has full parity with position[].

[Sebastianv650]

I thought I only doubled the position array by the position_float thing. It looks like I have to dig into the code again and compare what both of them are doing once more.

[Sebastianv650]

@billyd60 Can you try this branch and see if it solves for example your print from SD -> abort & restart problem? It's RCBugFix + a commit that should cleanup all position <> position_float things. https://github.com/Sebastianv650/Marlin/tree/cleanup-position_float

If it works better now, I would create a PR.

[billyd60]

Hi Sebastian been away from my computer for awhile. Yes I will try it tonight or tomorrow.

Regarding the sudden drops in the measured thermistor temps (after an lcd panel stop print command and restart) it appears as though the code is "losing track" of the actual readings from the thermistors (Bed and extruder) periodically. Ultimately I believe the temperature drops so low that it may trigger cold extrusion protection. This is just a guess though.

Btw this behavior sometimes takes a few hours to show up (both the odd temperature drops and print freeze). So it is quite difficult to reproduce. This suggests a very slow build up of the problem. Perhaps some kind of additive floating point problem that takes tens of thousands of iterations to show up? Again this behavior never happens on my printer without an LCD stop print command and then restart without cycling power. As long as I turn on the printer, and execute a print job without cancelling it, I have never had a print fail. I just completed a 9.5 hour print job with Rcbugfix and it was flawless.

[Sebastianv650]

@billyd60 please read the last posts from issue #5698, this might be another possible reason for the freezes you have.

[billyd60]

Do I need to use your config and adv.h files and change them for the Taz 5 or can I just use my current config and _adv.h files I am currently using for Rcbugfix? I really don't feel like doing all that editing again...

[Sebastianv650]

My two PR are not changing the config files. Depending on how old your current ones are, you might be able to copy and paste them. I would open your current one and the maybe newer RCBugFix one in notepad++ and use the compare function to see if there are structural changes. If not, just use the old ones.

[psavva]

I just simulated the same problem. Printer stopped mid print, and froze.

I'm using LIN_Advanced as well. K=400. The LCD screen RRD Full Graphics Smart Controller became extremely slow... The reset button would not respond. The LCD menu items didn't have an effect.

I'm trying to print the same thing again, and see if its reproducible...

I'll merge your PR in a local branch and try again to see if its fixed (if I manage to replicate it)

[psavva]

I replicated the issue again. I had K=300, and had no problem. I set K=400, and the printer froze before it finished a 10cm scirt.

I upped the the FR to 150%, and instantly froze...

[Sebastianv650]

With the pr to prevent temperatur isr reentry?

[psavva]

Not yet, now I know how to trigger, I'll try again with the PR

[Sebastianv650]

Ah, I was afraid it was with the PR :-) The behaviour is logical: When you fire the stepper ISR more often, which is true with a high K value and/or a high FR, there is a higher chance to trigger a stepper ISR within a temperature ISR. Now let's hope the best it's now gone!

[psavva]

@Sebastianv650, The problem still exists, even when applying https://github.com/Sebastianv650/Marlin/tree/cleanup-position_float

I just simulated in a matter of seconds.

I'm using: DRV-8825's at 1/32 steps. Lin_Advance; K=400 Bilinear Bed Levelling 50 mm/s Print Speed, but set FR to 300% to simulate the issue.

Update: I just turned Lin_Advanced off, and printed the exact same gcode, at 600% FR, with no freezing...

[Sebastianv650]

You need another PR, this one: https://github.com/Sebastianv650/Marlin/tree/Protect_tempISR The one you applied can also prevent problems, but only in rare situations I think.

[psavva]

Hi @Sebastianv650 I'm sorry to report that the problem still exists after merging Protect_tempISR.

I went to 250% FR, and Froze immediately.

I turned off LIN_ADVANCE, and no issue at 600% FR.

Let me know if I can do anything else to help diagnose the issue. I'm going to retire for tonight, early next week, or maybe over the weekend, I will be able to test it again, unless @billyd60 can try test before then :)

[thinkyhead]

the problem still exists after merging Protect_tempISR

If that doesn't address this, then probably #5829 doesn't either. But give it a check. I have a query in about the line cli(); at the end of Temperature::isr — maybe it's connected, but I can't confirm.

[Sebastianv650]

@psavva can you give me the speed your print is running (at 100% FR) and your steps/mm setting for x,y and e axis? I want to see what step rate we speek about when it's freezing. I think your problem might be something different than re-entry of temperature ISR, as your freeze isn't giving you a thermal error message and it doesn't happen with LIN_ADVANCE disabled. But that's also an advantage, this limits the areas where I have to search.

I will be blocked until friday, but I should find some free minutes in between to read through the code and think about it anyway.

[psavva]

@Sebastianv650 Print Speed: 70mm/s @ 100% FR Configuration.h http://pastebin.com/XBmAGqzq

Configuration_Adv.h http://pastebin.com/2D2uvRE1

When it freezes, I do not get a Thermal Error Message. The Steppers just stop. The LCD is extremely slow, but still functional (somewhat)

[Sebastianv650]

The LCD is extremely slow, but still functional (somewhat)

That's good to know, and also what I expected. It's clearly something with the stepper ISRs with LIN_ADVANCE at high step rates.

With your settings, your leading axis has 11200steps/second, which already means double stepping to get it done. Just for information, at 600% FR (67.2ksteps/s) you will most likely don't see 6x the speed of 100% FR because Marlin is already just handling the steps as fast as the CPU can do it. I will have a look what's going wrong with LIN_ADVANCE..

[Sebastianv650]

@psavva a quick check for someone that can reproduce the error - I can't, just tested it at 600% FR and no crash :-/

Have a look into stepper.cpp, search for the void Stepper::ISRhandler() block. At the moment, it should say:

    if (!nextMainISRrun) {
      isr();
    }
    if (!nextAdvanceISRrun) {
      advance_isr();
    }

    if (nextAdvanceISRrun <= nextMainISRrun) {
      OCR1A = nextAdvanceISRrun;
      if (nextMainISRrun)
        nextMainISRrun -= nextAdvanceISRrun;
      nextAdvanceISRrun = 0;
    }

change it to:

    if (!nextMainISRrun) {
      isr();
    }
    if (!nextAdvanceISRrun) {
      advance_isr();
    }

    //Start Debugging Code
    if (!nextMainISRrun) {
      SERIAL_ECHO_START;
      SERIAL_ECHOLNPGM("nMainISRrun=0");
      nextMainISRrun = 65535;
    }
    if (!nextAdvanceISRrun) {
      SERIAL_ECHO_START;
      SERIAL_ECHOLNPGM("nAdvanceISRrun=0");
      nextAdvanceISRrun = 65535;
    }
    //End Debugging Code

    if (nextAdvanceISRrun <= nextMainISRrun) {
      OCR1A = nextAdvanceISRrun;
      if (nextMainISRrun)
        nextMainISRrun -= nextAdvanceISRrun;
      nextAdvanceISRrun = 0;
    }

This is not for printing, but if one of the two values become 0 that might be a reason for the behaviour of your printer. If it is, you should get a message over serial when it's crashing. Please test this when you find the time and report if it gives you one of the messages and which one.

[psavva]

I'll report back by tomorrow :)

[billyd60]

Sorry I have not been able to do any testing. Too many print jobs to mess with it at the moment. Actually I am going to bow out of this because Rcbugfix is working perfectly for me now, all I have to do is not cancel a print job from the lcd panel or if I do, make sure to power cycle the Rambo before running a print and I get perfect prints every time. So I will remain a spectator and see what you guys come up with to solve it.

[psavva]

@Sebastianv650 the source code doesn't exist in either stepper.cpp, or anywhere else in the codebase in RCBugFix to debug as instructed.

Please double check what I should do to debug, and i'll do so.

[Sebastianv650]

You are right, the code has changed (renamed) in RCBugFix. Goto stepper.cpp, the code block is now "void Stepper::advance_isr_scheduler()".

Current Code:

    // Run main stepping ISR if flagged
    if (!nextMainISR) isr();

    // Run Advance stepping ISR if flagged
    if (!nextAdvanceISR) advance_isr();

    // Is the next advance ISR scheduled before the next main ISR?
    if (nextAdvanceISR <= nextMainISR) {
      // Set up the next interrupt
      OCR1A = nextAdvanceISR;
      // New interval for the next main ISR
      if (nextMainISR) nextMainISR -= nextAdvanceISR;
      // Will call Stepper::advance_isr on the next interrupt
      nextAdvanceISR = 0;
    }

change to:

    // Run main stepping ISR if flagged
    if (!nextMainISR) isr();

    // Run Advance stepping ISR if flagged
    if (!nextAdvanceISR) advance_isr();

    //Start Debugging Code
    if (!nextMainISR) {
      SERIAL_ECHO_START;
      SERIAL_ECHOLNPGM("nextMainISR=0");
      nextMainISR= 65535;
    }
    if (!nextAdvanceISR) {
      SERIAL_ECHO_START;
      SERIAL_ECHOLNPGM("nextAdvanceISR=0");
      nextAdvanceISR= 65535;
    }
    //End Debugging Code

    // Is the next advance ISR scheduled before the next main ISR?
    if (nextAdvanceISR <= nextMainISR) {
      // Set up the next interrupt
      OCR1A = nextAdvanceISR;
      // New interval for the next main ISR
      if (nextMainISR) nextMainISR -= nextAdvanceISR;
      // Will call Stepper::advance_isr on the next interrupt
      nextAdvanceISR = 0;
    }

[psavva]

It's not freezing anymore!!! http://pastebin.com/y4zJ4ake

I don't get it. From being able to consistently getting it to freeze going to FR250%, now it doesn't

I don't see what could have changed.

It also froze on me without LIN_ADVANCE enabled this morning.

Only difference now, is that the serial monitor is running, whereas before, I did not have a laptop connected, printed directly from the SD Card, without the Serial Monitor.

[psavva]

The printer just froze again, but this time with a Thermal Runaway on the LCD. No laptop connected, and LIN_ADVANCE enabled

[Sebastianv650]

I just recognised you are running a dual extruder setup. But on a first look, the code looks OK with that. The reaction is as I expected it. Your printer is not freezing because this line

nextAdvanceISR= 65535;

is disabling the advance ISR before it will crash your printer. If nextAdvanceISR becomes 0, this code block will "freeze" the time for the step execution because it subtracts 0 from nextMainISR.

    // Is the next advance ISR scheduled before the next main ISR?
    if (nextAdvanceISR <= nextMainISR) {
      // Set up the next interrupt
      OCR1A = nextAdvanceISR;
      // New interval for the next main ISR
      if (nextMainISR) nextMainISR -= nextAdvanceISR;
      // Will call Stepper::advance_isr on the next interrupt
      nextAdvanceISR = 0;
    }

So the good news is, we now know why it freezes printing and we also know why the display is still function, but slow: The printer is flooded with extruder ISRs at the max. possible rate. The bad news is, nextAdvanceISR shouldn't be able to become 0. When I did the coding for that, I had such a debuging code running on my printer and never got a message. The extruder stepper rate is calculated with

#define ADV_RATE(T, L) (e_steps[TOOL_E_INDEX] ? (T) * (L) / abs(e_steps[TOOL_E_INDEX]) : ADV_NEVER)

so there can't be a division by 0. So there are only 3 ways to get a 0 result:

• The timer value T is 0. Can't be, it's limited to reasonable values.
• The steps/loop count L is 0. Can't be, it can only be 1, 2 or 4.
• e_steps value is very high, so the rounded result becomes 0. This should not happen, but you are using a much higher K value than I ever tested (400), so it might be possible at high speeds.

Let's modify the debug code to see what values we have when we get a 0. Copy the following code in a new line after every

eISR_Rate = ADV_RATE(

in the stepper.cpp:

    //Start Debugging Code
    if (!eISR_Rate) {
      SERIAL_ECHO_START;
      SERIAL_ECHOPAIR("Timer:", timer);
      SERIAL_ECHOPAIR("step_loops:", step_loops);
      SERIAL_ECHOPAIR("esteps:", e_steps[TOOL_E_INDEX]);
      SERIAL_ECHOPAIR("tool_index:", TOOL_E_INDEX);
      eISR_Rate= 65535; //Disable the extruder ISR to prevent a frozen system.
    }
    //End Debugging Code

When you get a handfull of lines, paste it as you have done before and we will see..

The printer just froze again, but this time with a Thermal Runaway on the LCD. No laptop connected, and LIN_ADVANCE enabled

This time also with the debug code or without?