bigtreetech / Eddy

95 stars 14 forks source link

mcu shutdown when calibrating #24

Closed tonywright closed 2 months ago

tonywright commented 3 months ago

Hi, I had eddy (USB) installed a week ago but then removed it when I replaced my pi3b for a pi4. When I try to re-install it I am now getting as far as "PROBE_EDDY_CURRENT_CALIBRATE CHIP=btt_eddy" after adjusting Z when I click ACCEPT klipper shuts down immediately with an "MCU 'mcu' Shutdown Rescheduled timer in the past" error and the console shows "Failed calibration - incomplete sensor data"

Below is an extract from klippy.log I have update the MCU for the control board and for Eddy

Any idea where I am going wrong? (Also why does the filament need ejecting while doing the calibration?)

Thanks Tony

version info: mcu(atmega2560) Version: v0.12.0-166-g95747542 Load: 0.09, Awake: 0.00, Freq: 16 MHz,

mcu eddy(rp2040) Version: v0.12.0-166-g95747542 Load: 0.00, Awake: 0.00, Freq: 12 MHz, Temp: 34°C

Host(aarch64, 32bit) Version: v0.12.0-166-g95747542-dirty OS: Raspbian GNU/Linux 11 (bullseye) Distro: MainsailOS 1.3.2 (bullseye) Load: 0, Mem: 191.3 MB / 909.5 MB, Temp: 48°C eth0 (10.75.75.210) : Bandwidth: 4.1 kB/s , Received: 11.6 MB , Transmitted: 713.9 MB wlan0 (10.75.75.213) : Bandwidth: 0.4 kB/s , Received: 744.1 kB , Transmitted: 3.9 kB

Requested toolhead position at shutdown time 67.193094: (153.99999999999997, 150.0, -0.048282612574520725) Stats 1273.0: gcodein=0 mcu: mcu_awake=0.029 mcu_task_avg=0.000173 mcu_task_stddev=0.000174 bytes_write=29042 bytes_read=29987 bytes_retransmit=173 bytes_invalid=7 send_seq=1197 receive_seq=1197 retransmit_seq=1117 srtt=0.002 rttvar=0.000 rto=0.025 ready_bytes=44 upcoming_bytes=2403 freq=15999718 eddy: mcu_awake=0.001 mcu_task_avg=0.000005 mcu_task_stddev=0.000004 bytes_write=1437 bytes_read=12324 bytes_retransmit=9 bytes_invalid=0 send_seq=197 receive_seq=197 retransmit_seq=2 srtt=0.000 rttvar=0.000 rto=0.025 ready_bytes=0 upcoming_bytes=0 freq=12000256 adj=12000467 btt_eddy_mcu: temp=35.5 temperature_probe btt_eddy: temp=33.4 Raspberry: temp=47.7 heater_bed: target=0 temp=19.1 pwm=0.000 sysload=0.45 cputime=17.752 memavail=734604 print_time=80.014 buffer_time=12.425 print_stall=1 extruder: target=200 temp=199.8 pwm=0.214 extruder1: target=0 temp=22.4 pwm=0.000 BatchBulkHelper batch callback error Traceback (most recent call last): File "/home/pi/klipper/klippy/mcu.py", line 71, in _do_send return xh.get_response(cmds, self._cmd_queue, minclock, reqclock) File "/home/pi/klipper/klippy/serialhdl.py", line 326, in get_response raise error("Unable to obtain '%s' response" % (self.name,)) serialhdl.error: Unable to obtain 'sensor_bulk_status' response

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/pi/klipper/klippy/extras/bulk_sensor.py", line 72, in _proc_batch msg = self.batch_cb(eventtime) File "/home/pi/klipper/klippy/extras/ldc1612.py", line 212, in _process_batch self.clock_updater.update_clock() File "/home/pi/klipper/klippy/extras/bulk_sensor.py", line 233, in update_clock params = self.query_status_cmd.send([self.oid]) File "/home/pi/klipper/klippy/mcu.py", line 75, in send return self._do_send([self._cmd.encode(data)], minclock, reqclock) File "/home/pi/klipper/klippy/mcu.py", line 73, in _do_send raise self._error(str(e)) gcode.CommandError: Unable to obtain 'sensor_bulk_status' response LDC1612 finished 'btt_eddy' measurements Failed calibration - incomplete sensor data

krautech commented 3 months ago

Make sure you've removed everything from your printer.cfg that eddy would've added and re-rune the entire calibration process.

tonywright commented 3 months ago

I had done that when I removed it the first time, this is essentially a new installation. The only difference was the Pi4 instead of Pi3b and a usb camera which I removed and tried again with the same result. I tried the Eddy in both a usb2 and usb3 sockets with no difference. I can't think of any other changes except the upgrade of klipper and mcu firmware.

Tony

krautech commented 3 months ago

Which version of klipper is it using?

tonywright commented 3 months ago

Latest one I think: (Not sure what the "Dirty" is for unless it is the klipper bits)

Host(aarch64, 32bit) Version: v0.12.0-166-g95747542-dirty

tonywright commented 3 months ago

wrong version. klipper version is v0.12.0-206-g3078912f

looxonline commented 3 months ago

wrong version. klipper version is v0.12.0-206-g3078912f

Make sure that you've programmed your MCU using firmware built on the BTT branch of klipper and then make sure that you are using the BTT branch of klipper on your SBC.

tonywright commented 3 months ago

I have done that, I updated klipper first, then the mcu's. I'll try again just in case.

Is there a way of switching klipper back to the original (Latest) version then try reinstalling eddy version?

krautech commented 3 months ago

Installing the main version of klipper will screw everything up as it's Infront of BTTs version.

Start fresh using only BTTs klipper.

Use kiauh to install and then straight away switch to BTT.

Then make firmware for any of your mcus from that version and go from there

looxonline commented 3 months ago

I have done that, I updated klipper first, then the mcu's. I'll try again just in case.

Is there a way of switching klipper back to the original (Latest) version then try reinstalling eddy version?

Yea, if you are using KIAUH you just need to change the repo in the klipper_repos.txt file back to https://github.com/Klipper3d/klipper,master

If you didn't use this file to change to the BTT repo then I highly suggest that you do as it is a far better method. I think I'll get the manual updated to reflect that.

To do it, just rename the klipper_repos.txt.example file in the klipper directory to klipper_repos.txt

Then add: https://github.com/bigtreetech/klipper,eddy to the end of the file.

After that you should be able to switch the repo between the main and eddy within KIAUH using option 6.

looxonline commented 3 months ago

Installing the main version of klipper will screw everything up as it's Infront of BTTs version.

Start fresh using only BTTs klipper.

Use kiauh to install and then straight away switch to BTT.

Then make firmware for any of your mcus from that version and go from there

No need to even install mainline. Just change the repo in the klipper_repos.txt file and install the BTT repo straight.

krautech commented 3 months ago

I have done that, I updated klipper first, then the mcu's. I'll try again just in case.

Is there a way of switching klipper back to the original (Latest) version then try reinstalling eddy version?

Yea, if you are using KIAUH you just need to change the repo in the klipper_repos.txt file back to https://github.com/Klipper3d/klipper,master

If you didn't use this file to change to the BTT repo then I highly suggest that you do as it is a far better method. I think I'll get the manual updated to reflect that.

To do it, just rename the klipper_repos.txt.example file in the klipper directory to klipper_repos.txt

Then add: https://github.com/bigtreetech/klipper,eddy to the end of the file.

After that you should be able to switch the repo between the main and eddy within KIAUH using option 6.

Forward me any changes you'd like to see and I'll add them 👍 I can see this been a benefit to some users.

looxonline commented 3 months ago

I have done that, I updated klipper first, then the mcu's. I'll try again just in case. Is there a way of switching klipper back to the original (Latest) version then try reinstalling eddy version?

Yea, if you are using KIAUH you just need to change the repo in the klipper_repos.txt file back to https://github.com/Klipper3d/klipper,master If you didn't use this file to change to the BTT repo then I highly suggest that you do as it is a far better method. I think I'll get the manual updated to reflect that. To do it, just rename the klipper_repos.txt.example file in the klipper directory to klipper_repos.txt Then add: https://github.com/bigtreetech/klipper,eddy to the end of the file. After that you should be able to switch the repo between the main and eddy within KIAUH using option 6.

Forward me any changes you'd like to see and I'll add them 👍 I can see this been a benefit to some users.

Sounds good. I'm throwing together a long list of things that should be added to make it easier to use. I am basically spending this whole weekend trying to see what I can throw at the Eddy to make things not work and then trying to get them addressed in the docs. Managed to find an interesting one last night; calibrated LDC then wife wanted me to help carry something so shut the printer down. Came back to calibrate the sensor values vs. height later and klipper added spurious text into the calibration section that messed the whole thing up. Had to remove the whole calibration section and try again. Will need to chat to Kev and get that fixed but doc updates should help in the meanwhile.

krautech commented 3 months ago

Ouch, that sounds interesting 🤔

I actually haven't bothered to dive into the code. Had too much else going on ATM.

I did want to have a look, hopefully I'll get around to it soon.

I wonder if any progress is being made on the firmware front so hopefully we can get back to mainline, it does seem to be impacting alot of people the last few days.

tonywright commented 3 months ago

Hmm, after uninstalling and reinstalling btt klipper with KIAUH and recompiling and flashing the mcu's the versions have rolled back to 11:

mcu(atmega2560) Version: v0.11.0-479-g95747542 Load: 0.32, Awake: 0.01, Freq: 16 MHz, mcu eddy(rp2040) Version: v0.11.0-479-g95747542 Load: 0.00, Awake: 0.00, Freq: 12 MHz, Temp: 34°C

Host(aarch64, 32bit) Version: v0.11.0-479-g95747542-dirty

Klipper is still version: v0.12.0-206-g3078912f

Tried calibrating again and get the same mcu shutdown error.

looxonline commented 3 months ago

Klipper is still version: v0.12.0-206-g3078912f

I'm guessing you are speaking about klipper on your main MCU that is still V12? That should be OK since the comms are between the SBC and the Eddy and bypassing the main MCU.

Another possibility is that your position_min: setting in the z stepper section is not set low enough. It should be around -2. What is it currently set to?

FYI, here are my vers which work. Note that the klipper repo is not labelled as dirty when the repo is changed using KIAUH.

image

tonywright commented 3 months ago

This is mine, installed by kiauh:

image

Using custom repository:

image

Did you generate the firmware using Klipper master or after installing BTT version? Or doesn't it matter?

ifreislich commented 3 months ago

For the purposes of "getting it working" you need to run the BTT klipper version. You need to compile the MCU firmware from this version too.

tonywright commented 3 months ago

That's what I did but I think the first time I did it I updated the firmware then changed klipper to the btt version. I might just do a new image of mainsailos (Which I did the first time) then run through the whole process again to see what I get.

Thanks

ifreislich commented 3 months ago

But the version you just posted (v0.11.0-479) is not the what you should be running. You should be on v0.12.0-166-g957475421. The version you reported was obtained by a departure from the instructions.

Essentially kiauh did this:

  1. git clone https://github.com/bigtreetech/klipper.git
  2. git checkout eddy

What you need to do is this:

  1. git checkout https://github.com/Klipper3d/klipper.git
  2. git remote add eddy https://github.com/bigtreetech/klipper
  3. git checkout eddy/eddy
tonywright commented 3 months ago

So after switching back to klipper master then adding and checking out eddy/eddy I am now getting a warning when compiling the firmware for the control board:

Compiling out/src/sensor_ldc1612.o src/sensor_ldc1612.c: In function ‘ldc1612_query’: src/sensor_ldc1612.c:143:35: warning: left shift count >= width of type [-Wshift-count-overflow] uint32_t data = (d[0] << 24L) | (d[1] << 16L) | (d[2] << 8) | d[3]; ^ src/sensor_ldc1612.c:143:51: warning: left shift count >= width of type [-Wshift-count-overflow] uint32_t data = (d[0] << 24L) | (d[1] << 16L) | (d[2] << 8) | d[3]; ^ Compiling out/src/sensor_bulk.o

the ldc1612 is the eddy sensor isn't it?

These are only warnings but seems a bit coincidental

After flashing the control board mcu is now version: v0.12.0-166-g957475421 and so is the eddy mcu.

But this has made no difference as it still shuts down when trying to calibrate the eddy.

Thanks for the help.

Tony

ifreislich commented 3 months ago

What OS and host SBC are you using to compile that's generating those warnings? Your toolchain must be different to what others are using. I'm not seeing that when I compile the firmware, not even when I cross-compile it on my AMD64 linux workstation.

tonywright commented 3 months ago

It's on a raspberry pi 4 1GB running mainsailos img.

uname = Linux tenlog 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux

the board I am compiling for is an atmega2560

ifreislich commented 3 months ago

Right, I can reproduce that warning at least. The complaint is that the type d is 8 bits and it's being shifted 24 bits to the left. But the LDC1612 is not connected to this MCU, it's on an PR2040 USB connected direct to your SBC right? you're using an Eddy, not and Eddy Coil?

My suspicion is that there's some comms issue during the Z offset calibration. Can you see if there have been some USB resets, either in dmesg or /var/log/syslog?

tonywright commented 3 months ago

I didn't quite understand why it showed up when compiling for the control board mcu because as you say it is connected via usb to the raspberry. I can only see one error in dmesg: [ 1.380607] bcm2708_fb soc:fb: Unable to determine number of FBs. Disabling driver.

I can't see any usb issues in syslog, grepping for usb just shows new device found etc

I have just tried compiling without the eddy plugged in but still get the same warning.

ifreislich commented 3 months ago

The number in brackets at the start of the line is uptime in seconds. A new device much later on is an issue if it doesn't coincide with the firmware update.

Out of interest, what was the value for the drive current calibration?

tonywright commented 3 months ago

The drive current was calibrated at 16 @ 20mm above the bed.

tonywright commented 3 months ago

After having another look at the error and a bit of googling, it appears that a time in the past error can be caused by a stepper trying to go faster than it is capable of, could this be the extruder retraction? I have lowered the max speed of the extruder to no avail and no amount of looking can I find anywhere in the code that calls for a retraction (Not that I can follow it that well)
Is the retraction a side effect of something else the calibration code is doing? Or is it specifically called and if so what speed does it call the retraction at? (I am hoping one of you know a bit about the code :) )

Thanks

Tony

BTTJUSTIN commented 3 months ago

Thanks for reaching out. Our support team does not operate using the GitHub platform but we would be happy to help you via email based ticketing system. Please visit https://biqu.equipment/pages/technical-support and submit a ticket and we will be in touch with you soon.