synthetos / g2

g2core - The Next Generation
Other
623 stars 295 forks source link

Probe (Zmin) incorrectly handled by ArduinoDue+gShield. #126

Open Tegeon opened 8 years ago

Tegeon commented 8 years ago

First of all I do not think this is a gShield problem itself, as the same gShield is working perfectly fine with Arduino Uno and GRBL. I would guess that the problem comes from Arduino Due, which is currently handling the probing operation improperly.

The probe pin is using Zmin (D18 pin on the Arduino Due) and according to the appropriate settings.h it is defined as: **// Z min

define DI5_MODE INPUT_ACTIVE_LOW // Z probe

define DI5_ACTION INPUT_ACTION_NONE

define DI5_FUNCTION INPUT_FUNCTION_NONE**

INPUT_ACTIVE_LOW is a NORMALLY_OPEN equivelent, which I would expect is correct for a probe. The probe pin (D18) should be held HIGH state (VCC = 3.3V for Arduino Due) until it touch the copper = gets connected to GND and changing state to LOW. The problem is that PROBE pin gets triggered by connecting it to GND (this I would expect OK), as well as it gets triggered when connecting it to VCC (this is not correct). Using pull-up or pull-down resistor 'forces' the probe pin to a HIGH or LOW logical state and probing gets triggered immediately at Z-probe-max height.

I performed all tests with firmware version 89.02 cloned from github and compiled with (customized settings.h and gShield platform) on February 19th.

There is more detailed description of the problem here: https://plus.google.com/+SebastianSzafran/posts/BDLhBgwnpTo

aldenhart commented 8 years ago

I pulled in the G+ conversation so this can all be captured in this issue. From https://plus.google.com/+SebastianSzafran/posts/BDLhBgwnpTo

Tegeon (Sebastian) wrote: I have played with the TinyG2 on Due again and I think I found a reason for the probing issue, which is closely related to limit switches.

By default TinyG2 requires:

NC switches should be wired using pull-down resistor connected to GND, and not pull-up. This way during normal operation circuit is closed and the limit pins get +3.3V from Due Vref or VCC pin = logic HIGH. Once limit switches gets hit, it disconnects the circuit causing immediate change to a logic LOW. The pull-down resistor limits current, so we don't fry input pins. This is correct way to wire NC switches, as if a wire gets accidentally disconnected (broken) the machine will report LIMIT ACTION and won't work.

The way it works with pull-down resistors means that the LIMIT ACTION should be activated (triggered) when changing from HIGH to LOW = on TRAILING EDGE. Currently it gets triggered on LEADING EDGE, so the LIMIT ACTION gets activated when the limit switch is released = circuit gets completed (closed) = LOW -> HIGH. With the above (pull-down) connections of limit switches the PROBE pin works OK, only the limit switches gets triggered while released.

Wiring NC limit switches with pull-up resistors holds input pins in logic LOW during normal operation, when circuit is closed. After switch gets hit its logic is immediately pulled-up to VCC = logic HIGH. The processor reports LIMIT ACTION, as the change is LOW -> HIGH = LEADING EDGE. But this conflicts with the PROBE pin logical state (see my previous post for details) and on my opinion is not correct.

I tested both setups using a breadboard and motors running in the air. It works just as I explained above.

+Riley Porter​​​​​ and +Alden Hart​​​​​ How can I change the LEADING EDGE to TRAILING EDGE for limit actions, so I could compile and test?

aldenhart commented 8 years ago

Thanks for the analysis. Can you tell me what firmware build you are using? Presumably 087.xx from edge? There are some differences in the answer depending.

aldenhart commented 8 years ago

Tegeon (Sebastian) wrote:

The hardware is ArduinoDue+gShield ver. 5b, here are the build details: [fb] firmware build 89.02 [fbs] firmware build " build" [fv] firmware version 0.98 [cv] configuration version 7.00 [hp] hardware platform 3.00 [hv] hardware version 0.00 [id] TinyG ID 0213-0215-d423

I copied settings_default.h to settings_test.h and modified the following part: // Xmin on v9 board

define_* DI1_MODE NORMALLY_CLOSED__

define__ DI1_ACTION INPUT_ACTION_STOP__

define__ DI1_FUNCTION INPUT_FUNCTIONLIMIT*

// Xmax

define_* DI2_MODE INPUT_MODE_DISABLED__

define__ DI2_ACTION INPUT_ACTION_NONE__

define__ DI2_FUNCTION INPUT_FUNCTIONNONE*

// Ymin

define_* DI3_MODE NORMALLY_CLOSED__

define__ DI3_ACTION INPUT_ACTION_STOP__

define__ DI3_FUNCTION INPUT_FUNCTIONLIMIT*

// Ymax

define_* DI4_MODE INPUT_MODE_DISABLED__

define__ DI4_ACTION INPUT_ACTION_NONE__

define__ DI4_FUNCTION INPUT_FUNCTIONNONE*

// Zmin

define_* DI5_MODE INPUT_ACTIVE_LOW // Z probe__

define__ DI5_ACTION INPUT_ACTION_NONE__

define__ DI5_FUNCTION INPUT_FUNCTIONNONE*

// Zmax

define_* DI6_MODE NORMALLY_CLOSED__

define__ DI6_ACTION INPUT_ACTION_STOP__

define__ DI6_FUNCTION INPUT_FUNCTIONLIMIT*

then compiled make PLATFORM=gShield SETTINGS_FILE=settings_test.h and loaded g2/TinyG2/bin/gShield/gShield.bin firmware onto ArduinoDue using Chilipeppr Program Board's Firmware feature and a local web server to download the file from.

Tegeon commented 8 years ago

It looks like both NC switches wired to limit pins with pull-down resistors and NO (probe) wired to Zmin with pull-up resistor should react to HIGH->LOW transition (trailing edge) and not LOW->HIGH (leading edge) as it is now. During normal operation all pins will be kept HIGH logical state and the trigger action will connect it to GND = we will face a HIGH->LOW transition.

The only change needed in the code at this moment is to change from LOW->HIGH (leading edge) trigger action to HIGH->LOW (trailing edge) trigger action and test.

With current configuration (NC limit switches connected with pull-up resistors) it works in a way, that during normal operation circuit is closed connected to GND and when triggered (switch opened) the logic goes to HIGH thanks to pull-up resistor = LEADING EDGE. But this false triggers also the PROBE pin on ArduinoDue. We do not observe this on ArduinoUno and grbl, as Uno is using pin A5 for probing, which is not related to pin D18 (Zmin).

aldenhart commented 8 years ago

Let me see if I understand correctly. You have a normally closed (NC) limit switch wired on one side to +3.3v, and on the other side to the input pin which is pulled to ground with a pulldown resistor. This sets up a circuit that is HI in normal operation, and goes LOW if the switch is hit or the switch wire is disconnected (or +3.3v is dropped). This would be an ACTIVE_LOW circuit as the input will go LOW when the switch activates.

I believe it is equally valid to wire the NC switch to ground, and the other side to the input pin and a pullup resistor to +3.3v. This circuit would hold the input LOW during normal operation, and go HI if the switch was hit or disconnected. This would be an ACTIVE_HIGH circuit.

In each case the trigger should be on the leading edge of the activation - i.e. from HI to LOW for the active low circuit, and from LOW to HI on the active high circuit.

I believe this is all currently supported in the way gpio.cpp works. The input is taken raw, then corrected for active hi / active lo by variable 'pin_value_corrected' which takes that inversion into account. INPUT_EDGE_LEADING is the onset of switch activation, and INPUT_EDGE_TRAILING is the offset - regardless of the type of circuit. Probing always activates on INPUT_EDGE_LEADING, regardless of he type of input. This should be validated, but I believe this is true and working correctly in gpio.cpp.

You also appear to have DIgital Input 5 (probe) configured correctly as ACTIVE_LOW. So unless something is not working in the gpio code I'm not sure what's going on here. But this gives me something to test.

Tegeon commented 8 years ago

This is my wiring for NO and NC switches: NC-NO.pdf I think this is a correct way how the switches should be wired.

When I connected the NC switches with pull-up resistors the limit actions (from all 3 NC limit switches) were triggered properly, but the probing was not working correctly.

The only scenario when all was working OK (with the exception that trigger action was on NC limit switch release) was when I connected: NO Probe as shown on the left circuit on the schematics attached NC limit switches as shown on the right circuit on the schematics attached

That is why I think changing the trigger edge to HIGH->LOW transition should solve this problem.

Tegeon commented 8 years ago

Lines 35-40 in the gpio.cpp states: The normally open switch modes (NO) trigger an interrupt on the falling edge and lockout subsequent interrupts for the defined lockout period. This approach beats doing debouncing as an integration as switches fire immediately.

The normally closed switch modes (NC) trigger an interrupt on the rising edge and lockout subsequent interrupts for the defined lockout period. Ditto on the method.

This is correct way of thinking when wiring NC switches with pull-up resistors, as in that case during normal operation the INPUT PIN state = logic LOW, and when the pin gets triggered the state goes to logic HIGH. We indeed have LOW to HIGH transition here. But this (NC with pull-up resistors) wiring method does affect the way PROBE pin operates and makes false triggers of the PROBE. In order to correct this, NC switches should be wired with pull-down resistors, but then we need to change in software the trigger to get activated by the HIGH to LOW transition on the leading edge.

In my situation (and I think this should be the way we wire the switches/probe: NO switch: For NO switch (the PROBE) the interrupt is triggered on HIGH to LOW (called: 'falling edge') - I presume this is CORRECT for NO switch wired with pull-up resistor.

NC switch: For NC switch (Xmin, Ymin, Zmin limit switches) the interrupt is triggered on LOW to HIGH (called 'rising edge') - I presume this is NOT CORRECT for NC switch with pull-down resistors wiring, as we need HIGH to LOW.

aldenhart commented 8 years ago

Thanks for your drawing on the switches. There's some confusion here I'd like to clear up. I'd like to stop calling things NO and NC, as NO and NC switches can be wired in different ways that have different behaviors - as your drawings clearly illustrate. Not everyone will wire things the way you do, and other devices (like optos) don't really have the concept of closed and open, just what levels are reported. Instead I'd like to use the terms:

By these definitions the circuits you provided are both active low. Probing should fire on the leading edge, which in this case is the transition from hi to low. The comments in the beginning of gpio.cpp should be revised to the above because I think they are the source of some confusion.

So I am not seeing where the problem is. Your circuits transition to LOW when activated, and you have the input configured correctly as active LOW. The way gpio works is:

I will confirm this function with a debug session to be sure. It should not require any changes to the software once the configuration is set to active low. Let's do this and see if there's still a problem.

Tegeon commented 8 years ago

I changed the settings.h file to the following (only limit switches and probe part is presented, changes marked in bold): // Xmin on v9 board

define DI1_MODE INPUT_ACTIVE_LOW //NC switch wired with pull-down resistor

define DI1_ACTION INPUT_ACTION_STOP // STOP

define DI1_FUNCTION INPUT_FUNCTION_LIMIT // LIMIT

// Xmax

define DI2_MODE INPUT_MODE_DISABLED

define DI2_ACTION INPUT_ACTION_NONE

define DI2_FUNCTION INPUT_FUNCTION_NONE

// Ymin

define DI3_MODE INPUT_ACTIVE_LOW //NC switch wired with pull-down resistor

define DI3_ACTION INPUT_ACTION_STOP

define DI3_FUNCTION INPUT_FUNCTION_LIMIT

// Ymax

define DI4_MODE INPUT_MODE_DISABLED

define DI4_ACTION INPUT_ACTION_NONE

define DI4_FUNCTION INPUT_FUNCTION_NONE

// Zmin

define DI5_MODE INPUT_ACTIVE_LOW //Z probe (NO) wired with pull-up resistors

define DI5_ACTION INPUT_ACTION_NONE

define DI5_FUNCTION INPUT_FUNCTION_NONE

// Zmax

define DI6_MODE INPUT_ACTIVE_LOW //NC switch wired with pull-down resistor

define DI6_ACTION INPUT_ACTION_STOP

define DI6_FUNCTION INPUT_FUNCTION_LIMIT

For NO/NC wiring details please see the wiring diagram in a post above or here: https://github.com/synthetos/g2/files/151195/NC-NO.pdf

Then I made quick tests: TEST1: NC limit switches -> event triggered when switch is pressed = OK TEST2: TouchPlate using PROBE (probe connected to Zmin) -> event triggered when Probe hits GND = OK TEST3: Autolevel using PROBE -> event triggered on each probe location, move to next location and probe and so on up to the end of probing cycle = OK

I will do more testing while using the machine next days, but it seems that no change in the code is needed.

The problem was caused by the misleading explanation suggesting INPUT_ACTIVE_HIGH as NORMALLY_CLOSED switch equivalent. This is applicable when the switch is connected to pull-up resistor. Based on my experiance this configuration interferes with PROBE pin behavior and do not work properly, or better to say it did not work for me. When the NC (normally closed) switch is connected with pull-down resistor the appropriate D_PIN_MODE has to be changed to INPUT_ACTIVE_LOW.

I agree with Alden Hart that it is better to use ACTIVE_HIGH and ACTIVE_LOW naming convention instead of NC/NO to avoid misunderstanding.

cmcgrath5035 commented 8 years ago

This comment is about the schematic pdfs: Your "NO" schematic has 10K in series with 100nf. Why is 10K in series? - that reduces the effectiveness of the capacitor substantially. Ah, I looked again, you wanted an RC filter. in that case, you would want the 10K between the switch and the port pin, with the 100nF across the port pin.

100nF might not be enough. See below for what Synthetos uses on G2 based hardware.

And I see no capacitor in the "NC" circuit

Any switch (mechanical or 'electronic') is going to effectively bounce a bit or be noisey, at the open close transition.

I note that tinyGV9 (prototype boards) use 2.7K pull ups to VCC (3.3V) and 0.47uF to ground, all on the pcb, very close to the MCU IC. That is 470nF. My 2 cents - high probability of transition 'open' to 'closed' noise and multiple 'edges', some likely higher frequency than fw can follow.

That said, I have not tried Sebastian's experiment on a V9.

Alden - does the Due have any port conditioning hardware on board that can be turned on/off in microcode?

Tegeon commented 8 years ago

I tried PCB probing with Chilipeppr yesterday and it still have some problems - see pictures. Some probes are OK, and then it continuously triggers at Z=0.5 (Z-probe start height) for some time and again correct probes and so on - see pictures. probe1 probe2

Tegeon commented 8 years ago

Today I did more tests with noise eliminated. I used my 'running in the air' Nema 17 kit, wires isolated, low motors current, RC filters.

The probing works wrong based on my experience. It begins correct and suddenly start the bad-probes at Z-probe-max height with no respect to the probe pin logic level. This is sometimes 0V and sometimes 3.3V all in the same probing cycle. No really a rule for logic HIGH or LOW causing the issue.

I tested also with firmware 87.01 available via Chilipeppr Programmer Widget and for that firmware version to work I change the probe wiring to pull-down resistor circuit. When probe switch is opened it is 0V, and when it closes it goes 3.3V. That looks like version 87.01 is compiled with probe set ACTIVE_HIGH by default. The same situation happened: after few correct probes I got some bad, good again and bad and good... with probe pin logic level HIGH or LOW.

I also found that rarely after machine and the TinyG2 first power-hungry Touch Plate 'reacts' two times on the first usage. It goes double high (2mm once, stops for a short time and go high by another 2mm again).

What do I need to start debugging, Atmel ICE Debugger? 

benjamind commented 8 years ago

@giseburt am I right in thinking that if this issue is related to the debounce code then we're looking at the _handle_pin_changed function in gpio.cpp?

From taking a quick look I think you might be right that this is the issue. From my reading of that function if the contact time of the probe acting is very short then the lockout timer will prevent detection of the change to the pin state. Order of effect is as follows (correct me if I'm wrong)...

  1. pin changes from 1 to 0 (NO switch has closed, pullup is overriden by path to ground through switch)
  2. _handle_pin_changed detects the state change to input high ($in5 = 1), and sets edge = INPUT_EDGE_LEADING.
  3. pin changes from 0 to 1 (NO switch is now open, pullup takes us back to 1)
  4. _handle_pin_changed exits immediately because lockout timer is in effect
  5. since the pin has already changed to 1 and the switch is open, there's no further changes to pin value, so we never get out of the input high state.

Does this sound about right?

benjamind commented 8 years ago

Upon further inspection I have this pin state issue on all axes.

If I do a homing to X or Y, the machine moves towards the switch at the appropriate search speed, triggers the switch, backs off, then slowly moves back to the switch at latch speed. Then jumps away from the switch by the offset amount I have configured.

However, if i issue a $in command after the sequence is complete the switch that was triggered is still showing as 1 even when its clearly no longer tripped.

Has anybody confirmed working homing and probing in a reproducible way on G2 on a v9 board? (I assume this code would also be the same on the Arduino +GShield combination, but perhaps the interrupt sequence is different there?).

ewidance commented 8 years ago

I got the same issue, even if i'm not on the same hardware (arduino Due + passive shield ( https://github.com/labsud/GTwo ) + TB6560 red driver board. It appears to be more visible on probe than on endstops, because endstop signals are conditionned by TB6560 before being routed to the shlield (opto isolated). On the probe, if contact time is very short, software totally hangs, as if some states where missing in the probing stage machine . It is necessary to reestablich contact to allow finishing of probing cycle.

I'll try to add external hardware debounce filter ( https://hackadaycom.files.wordpress.com/2015/11/proper_debouncing-sch.png ) to see if the fix works.

ewidance commented 8 years ago

@benjamind you're right about debounce.I looked on the code to improve software debounce, but i was not able to fix it by software. So I 've added an hardware debouce on Z min (10K / 1uF / 74HC14) like on http://www.next.gr/other-circuits/switch-circuits/switch-debouncer-l13728.html and it works fine. There's no more stops in probing. I'm currently extending that hardware debouce to all inputs (Xmin Ymin, ZMax, Probe, Estop, Hardware interlock) because when i was playing with homing i had he same unpredictables issues.

Now the question is : how to fix the software....

benjamind commented 8 years ago

Ice work @ewidance! I too have had issues on the homing inputs as well as probing. Perhaps it will be worth working up a little pcb to provide hardware debounce for 6 or 8 inputs.

You are right though this should be solvable in software but might need some rearrangement of the input code.

ewidance commented 8 years ago

The fine manual reflects the fact it needs an hardware debouce : https://github.com/synthetos/g2/wiki/Homing-g2#switch-inputs ..

benjamind commented 8 years ago

So it does!

Though I am actually using a v9 which that page indicates should already be debouncing in hardware, though its evidently not...

melearp commented 8 years ago

New to Github and TinyG2.

I am trying to replace a dead Xpro with a TinyG2 setup. It is still on my desk, with no driver shield as yet. I am using Chilipeppr with v9=true. I have a Due clone and have built G2 with settings changed to reflect my machine. I have made sure that DI5_MODE is set to INPUT_ACTIVE_LOW as per previous post in this thread.

Successes so far are that:

  1. I can talk to the Due via Chilipeppr and even run Gcode files, samples and my own. All looks great.
  2. I suffer from the "Due doesn't start application on power up" problem common on R3 boards, but I can live with that by wiring in a manual reset switch. That is proven working now too.

My problem is the Z-Probe, which I used all the time with the Xpro. At first it did not work at all on TinyG2, so I have breadboarded a circuit with a pull up resistor (10K) and a manual switch to simulate the plate, with partial success. I don't have a capacitor to hand to do the full debounce as per the book. The symptoms are now as follows:

a) If the touch plate feed rate is greater than about 70 mm/min, then it works and works reliably b) If the touch plate feed rate is less than about 70 mm/min, then I have to press the touch switch twice to get it to stop. This is also reliable. The interval between presses can be almost nothing to many seconds. It doesn't seem to matter. c) around the 70 mm/min boundary (+- 2 mm/min), there can be some variation as to whether it works on one touch or two, but it always works on one or the other.

Greater than 70mm/min is too fast for comfort for this function, so I need to solve it before implementing on the machine itself.

This sounds like a timing problem. Does anyone have any ideas about debugging and fixing this. Any help gratefully received.

Mel.

PS From $sys [fb] firmware build 89.03 [fbs] firmware build " build" [fv] firmware version 0.98 [cv] configuration version 7.00 [hp] hardware platform 3.00 [hv] hardware version 0.00 [id] TinyG ID 0213-0215-d473

melearp commented 8 years ago

So I thought that I would look at the code . Clearly, it was not obvious to me where my problem might lie, but it seemed that changing the DI5 action to stop might at least change the behaviour. Well, it did and it didn't. The boundary between requiring one touch and two dropped to about 45 mm/min. An odd behaviour change. So I put the DI5 action to none, reflashed, and the boundary remained at 45.

So I am still in the position where the touch plate is not satisfactory. And the boundary between 1 and 2 touches varies depending on what, I do not know.

An ideas please??

Happy to provide my settings.h if that helps.

masha256 commented 8 years ago

I finished reworking my X-Carve to use a Due+gShield so I could switch to G2. I am having the exact same probing issue mentioned in this thread:

A single "touch plate" action seems to work ok, but when trying to do the probing, the first few touches sometimes work ok, but then it subsequent probes fail as G2 seems to think the probe input is always active. I monitored quite aggressively using $in and my multimeter on the Zmin pin, and electrically, everything seems fine. All my switches are wired to pull-ups and have an RC filter. Limits are ACTIVE_HIGH, wired as NC switches. Probe is ACTIVE_LOW.

Schematic is available here: https://github.com/makerhqsac/tinyg_g2_breakout

What I noticed using $in is that the probe pin seems to latch into an active (1) state, even when no longer touching the copper plate. What is strange is I can remove the clip connected to the bit and manually touch it to the copper a few times, and it seems to "clear" the input, and a couple more probes then complete successfully (after reclipping the probe wire onto the bit). Then for a while, the input was actually reversed. $in would show 1 while the probe was disconnected, and 0 while connected. After a few more touch cycles, it seemed to reset back to regular operation where it was 0 when not touching, and 1 when touching. It definitely seems like a software thing - like the pin input state is somehow affected by a probe operation.

I am currently using the latest 'edge' branch pulled from github and compiled locally.

I am happy to test out any modifications to my setup if it helps track down the issue.

cmcgrath5035 commented 8 years ago

machadolab: Comments from ewidance above suggest that perhaps 10K and 0.1uf is not adequate port pin filtering for some layouts. Try an experiment with 1uF? It might help, but might not 'solve'. An active Schmitt trigger gate might help as well.

masha256 commented 8 years ago

Some new findings for everyone, that perhaps helps someone track down and fix the root cause. I changed INPUT_LOCKOUT_MS in gpio.h down to 10, which didn't help. I then decided "hey, I have added hardware debounce circuitry to all inputs, and used shielded cables everywhere - what if I totally disable the software debounce". Sure enough, after I set INPUT_LOCKOUT_MS to 0, probing is working great! I probed several hundred points in a row without a single false trigger. All limit switches still home perfectly as well.

check it out: https://lh3.googleusercontent.com/k7xq_ULCTLIqwxfAFI-lmbUsasgLreV1Po5dWtnKCUYUabpP2bR9YdMuX2nxMJLwNKt0ya6sLmh4dA=w1280-h960-no

I wouldn't suggest doing this unless you are confident in your electrical setup. I am still baffled by the root cause. I suspect it has something to do with state being handled only by interrupts vs checking the pin value via a port register when actually needed.

Another thing that I stumbled upon while doing this testing is that TinyG seems to use machine coordinates when a G38.2 is issued. I ended up breaking some bits because I had my machine Z zero lower than my work piece because I didn't think it mattered, and when the probe started, the first point would probe good, but then the spindle would quickly jog down and crash the bit into the work peice. After several WTF's spoken aloud, I figured out it was using the machine coordinates. So after setting both machine zero and G54 zero before starting the probe, I got perfect results. I would advocate for G38.2 using the 'current' work coordinate system, but perhaps I am missing some logic as to why always using machine coordinates is better.

giseburt commented 7 years ago

This should have been fixed in the current edge. I'll move this to In Progress. Can someone on this issue please confirm it's fixed in Edge?

antbarney commented 7 years ago

EDIT: I think this mostly was a hardware/bounce issue. I have added some more aggressive filtering and now have probing working quite at the moment.

@giseburt, I have just started to work out using the probing function but this still seems to be a issue in the current edge branch. I have tried a bunch of settings of the INPUT_LOCKOUT_MS from 0 to 20ms and still have the input state get stuck in the wrong state many times causing Auto leveling to fail. I am using a simple RC filter on the Z-Max input (since thats hard coded to Probe).

{"r":{"sys":{"fb":100.18,"fbs":"100.18-1-gf809-dirty","fbc":"settings_cbeam.h","fv":0.99,"hp":3,"hv":0,"id":"0084-d639-18c6-9ce","jt":1.25,"ct":0.01,"sl":0,"lim":1,"saf":0,"m48e":1,"mfoe":0,"mfo":1,"mtoe":0,"mto":1,"mt":10,"spep":1,"spdp":0,"spph":1,"spdw":1,"ssoe":0,"sso":1,"cofp":1,"comp":1,"coph":1,"tv":1,"ej":1,"jv":2,"qv":0,"sv":1,"si":250,"gpl":0,"gun":1,"gco":1,"gpa":2,"gdi":0}},"f":[1,0,5]}

{"r":{"6":true},"f":[1,0,5]} (its stuck in true state at this point even tho the pin is high)

benjamind commented 7 years ago

I can confirm that this is still an issue. Replicated the behavior I had a year ago with today's master edge version.

benjamind commented 7 years ago

I did a bunch more investigation on this tonight. Its definitely the lockout that is causing the problem.

https://github.com/synthetos/g2/blob/edge/g2core/gpio.cpp#L120

If at any point you have a probe pin change that is faster than the lockout period then if that change caused the probe to latch to true, then it does not unlatch. I recorded a whole bunch of samples of this on my scope tonight, any time i had a probe that went from high to low and back up to high again in less than 10ms I would end up with the pin state stuck in high.

I understand that the lockout is an attempt to avoid debouncing, but if you're going to do that you also need to add a timeout that will re-sample the pin state after the lockout ends in order to baseline the pin to its actual state again.

benjamind commented 7 years ago

I've also just discovered that I might have been wiring this up incorrectly.

I had wired this as a NO switch, the touchplate or my workpiece were wired to ground, and the probe was clipped to my bit. However, since I properly grounded and shielded my spindle and I have metal bearings when I connect the probe clip to the bit it gets pulled to ground immediately. Its not quite a perfect ground since there's a bunch of noise through that ground path, but its a lot closer to ground than the capacitor / resistor network was able to pull up for that pin.

Any idea how I should rewire this given that when my probe clip is attached to the bit it will be effectively grounded?

benjamind commented 7 years ago

Ok that was mind numbingly obvious. Just reversed my wires and problem solved. I now have the DI5 GPIO connected to the plate or my workpiece, and the GND pin connected to my bit. Problem solved!

benjamind commented 7 years ago

Spoke too soon. I'm giving up for the night, but it appears the lock timeout is still an issue. After switching my wires around it still managed to latch the pin state incorrectly. Commenting out the set lock line in gpio.cpp seems to have fixed this. However, when running a probing session I got about 8 probes in before everything froze and the board went into PANIC.

 G38.2 Z-1.5 F25
{"r":{},"f":[1,0,16]}
{"sr":{"posz":0.952,"vel":25,"momo":1,"stat":7}}
{"sr":{"posz":0.848}}
{"sr":{"posz":0.744}}
{"sr":{"posz":0.64}}
{"sr":{"posz":0.536}}
{"sr":{"posz":0.432}}
{"sr":{"posz":0.328}}
{"sr":{"posz":0.225}}
{"sr":{"posz":0.12}}
{"sr":{"posz":0.017}}
{"sr":{"posz":-0.024}}
{"prb":{"e":1, "z":-0.025}}
{"sr":{"posz":-0.025,"vel":0,"momo":4,"stat":3}}
 G0 Z1.5
 G0 X13.349960923194885 Y58.349960923194885
 G0 Z1
{"r":{},"f":[1,0,8]}
{"r":{},"f":[1,0,43]}
{"r":{},"f":[1,0,6]}
{"sr":{"posz":0.502,"vel":325.5,"momo":0,"stat":5}}
{"sr":{"posz":1.516,"vel":39.58}}
{"sr":{"posy":62.606,"posz":1.534,"vel":5382.11}}
{"sr":{"posy":58.35,"posz":1.411,"vel":138.61}}
{"sr":{"posz":1,"vel":0,"stat":3}}
 G38.2 Z-1.5 F25
{"r":{},"f":[1,0,16]}
{"sr":{"posz":0.957,"vel":25,"momo":1,"stat":7}}
{"sr":{"posz":0.854}}
{"sr":{"posz":0.749}}
{"sr":{"posz":0.645}}
{"sr":{"posz":0.54}}
{"sr":{"posz":0.437}}
{"sr":{"posz":0.332}}
{"sr":{"posz":0.229}}
{"sr":{"posz":0.125}}
{"sr":{"posz":0.021}}
{"prb":{"e":1, "z":0.000}}
{"sr":{"posz":0,"vel":0,"momo":4,"stat":3}}
{"sr":{"stat":3}}
 G0 Z1.5
 G0 X13.349960923194885 Y43.349960923194885
 G0 Z1
{"r":{},"f":[1,0,8]}
{"r":{},"f":[1,0,43]}
{"r":{},"f":[1,0,6]}
{"sr":{"posz":0.726,"vel":37.39,"momo":0,"stat":6}}
{"sr":{"posz":0.737,"vel":0}}
 G38.2 Z-1.5 F25
{"r":{},"f":[1,0,16]}
 $in
 {"ej":""}
{"r":{"ej":1},"f":[1,0,11]}

This may have just been noise on the other limit switches causing the panic but I'm too tired to investigate further tonight.

benjamind commented 7 years ago

Back to it today.

After removing the lock timeout I've been able to do a couple of successful probing runs. However I'm still getting random failures, heres one:

{"sr":{"posz":0.556,"vel":61.68}}
{"sr":{"posz":0.5,"vel":0,"stat":3}}
 G38.2 Z-1.5 F25
{"r":{},"f":[1,0,16]}
{"sr":{"posz":0.455,"vel":25,"momo":1,"stat":7}}
{"sr":{"posz":0.352}}
{"sr":{"posz":0.247}}
{"sr":{"posz":0.143}}
{"sr":{"posz":0.053,"vel":0.03}}
{"er":{"fb":100.26,"st":250,"msg":"Probe cycle failed - probing failed"}}
{"prb":{"e":0, "z":-1.500}}
{"sr":{"posz":-1.5,"vel":0,"momo":4,"stat":2}}

In this case probing had been working fine, it starts another probe command, and we can see the z moving down to near zero, I then get the probing failed error, and the probe reports that we're at z-1.5 and then afterwards we get the updated position. This is clearly not whats actually happened here since there's no way we jumped from z0.053 to z-1.5 without a) the bit digging into the piece which it definitely has not, and b) us getting at least one status report in between, especially at 25 feedrate.

In other cases I've just had random Panics during the middle of probing, no real reason why as far as I can tell. These seem to be reduced by disabling all the other inputs (though none of them are configured to emit Panic anyway).

giseburt commented 7 years ago

Hi Ben,

I'm catching up with this thread.

It's not likely the lockout. This tripped me up a few times as well, but the lockout isn't your typical Arduino-like denounce where you wait to see what a switch lands on then act. Instead, what happens is, when there's an edge it immediately takes action, then sets the lockout timer and ignores any more edges during that time.

The assumption that must hold true is that there's not a spike/pulse on the input that is shorter than the timeout, or it'll lose state.

The error you're seeing - 250 - I believe indicates that it stopped before then probe length completed without an edge on the probe input.

So, if you use a G38.2 Z10 F200, and it moves down 10mm (or inches, depending) and it doesn't see the switch change, it's an error. (It shouldn't be a panic, just an alarm.)

If you use a G28.3 Z10 F200 and it doesn't see the switch change, it'll spit out a msg but it's not an error (alarm, panic, or otherwise).

Note also that probing uses the high-speed jerk. If it's set too high you may not be seeing the motion you expect, and with it being the Z axis moving down it may actually just be gravity dragging down a stalled axis.

Hopefully somewhere in there is somehing that helps! :-)

-Rob On Sun, Jun 4, 2017 at 2:40 PM Ben Delarre notifications@github.com wrote:

Back to it today.

After removing the lock timeout I've been able to do a couple of successful probing runs. However I'm still getting random failures, heres one:

{"sr":{"posz":0.556,"vel":61.68}} {"sr":{"posz":0.5,"vel":0,"stat":3}} G38.2 Z-1.5 F25 {"r":{},"f":[1,0,16]} {"sr":{"posz":0.455,"vel":25,"momo":1,"stat":7}} {"sr":{"posz":0.352}} {"sr":{"posz":0.247}} {"sr":{"posz":0.143}} {"sr":{"posz":0.053,"vel":0.03}} {"er":{"fb":100.26,"st":250,"msg":"Probe cycle failed - probing failed"}} {"prb":{"e":0, "z":-1.500}} {"sr":{"posz":-1.5,"vel":0,"momo":4,"stat":2}}

In this case probing had been working fine, it starts another probe command, and we can see the z moving down to near zero, I then get the probing failed error, and the probe reports that we're at z-1.5 and then afterwards we get the updated position. This is clearly not whats actually happened here since there's no way we jumped from z0.053 to z-1.5 without a) the bit digging into the piece which it definitely has not, and b) us getting at least one status report in between, especially at 25 feedrate.

In other cases I've just had random Panics during the middle of probing, no real reason why as far as I can tell. These seem to be reduced by disabling all the other inputs (though none of them are configured to emit Panic anyway).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/synthetos/g2/issues/126#issuecomment-306062178, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXj0QqVG-mOpIVuJ0wR9_-NlUesZNjcks5sAwgjgaJpZM4HiIGQ .

giseburt commented 7 years ago

Oh, bad examples from me! I'll correct them in a minute.

Should be G38.2 Z-10 F200 to error (negative Z), and G38.3 Z-10 F200 for the non-error (negative Z and 38 not 28). On Sun, Jun 4, 2017 at 3:34 PM Rob Giseburt giseburt@gmail.com wrote:

Hi Ben,

I'm catching up with this thread.

It's not likely the lockout. This tripped me up a few times as well, but the lockout isn't your typical Arduino-like denounce where you wait to see what a switch lands on then act. Instead, what happens is, when there's an edge it immediately takes action, then sets the lockout timer and ignores any more edges during that time.

The assumption that must hold true is that there's not a spike/pulse on the input that is shorter than the timeout, or it'll lose state.

The error you're seeing - 250 - I believe indicates that it stopped before then probe length completed without an edge on the probe input.

So, if you use a G38.2 Z10 F200, and it moves down 10mm (or inches, depending) and it doesn't see the switch change, it's an error. (It shouldn't be a panic, just an alarm.)

If you use a G28.3 Z10 F200 and it doesn't see the switch change, it'll spit out a msg but it's not an error (alarm, panic, or otherwise).

Note also that probing uses the high-speed jerk. If it's set too high you may not be seeing the motion you expect, and with it being the Z axis moving down it may actually just be gravity dragging down a stalled axis.

Hopefully somewhere in there is somehing that helps! :-)

-Rob On Sun, Jun 4, 2017 at 2:40 PM Ben Delarre notifications@github.com wrote:

Back to it today.

After removing the lock timeout I've been able to do a couple of successful probing runs. However I'm still getting random failures, heres one:

{"sr":{"posz":0.556,"vel":61.68}} {"sr":{"posz":0.5,"vel":0,"stat":3}} G38.2 Z-1.5 F25 {"r":{},"f":[1,0,16]} {"sr":{"posz":0.455,"vel":25,"momo":1,"stat":7}} {"sr":{"posz":0.352}} {"sr":{"posz":0.247}} {"sr":{"posz":0.143}} {"sr":{"posz":0.053,"vel":0.03}} {"er":{"fb":100.26,"st":250,"msg":"Probe cycle failed - probing failed"}} {"prb":{"e":0, "z":-1.500}} {"sr":{"posz":-1.5,"vel":0,"momo":4,"stat":2}}

In this case probing had been working fine, it starts another probe command, and we can see the z moving down to near zero, I then get the probing failed error, and the probe reports that we're at z-1.5 and then afterwards we get the updated position. This is clearly not whats actually happened here since there's no way we jumped from z0.053 to z-1.5 without a) the bit digging into the piece which it definitely has not, and b) us getting at least one status report in between, especially at 25 feedrate.

In other cases I've just had random Panics during the middle of probing, no real reason why as far as I can tell. These seem to be reduced by disabling all the other inputs (though none of them are configured to emit Panic anyway).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/synthetos/g2/issues/126#issuecomment-306062178, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXj0QqVG-mOpIVuJ0wR9_-NlUesZNjcks5sAwgjgaJpZM4HiIGQ .

benjamind commented 7 years ago

So here's the scenario.

I am at Z2, my high-speed and normal jerk settings are the same so no worries there on the movement, and the z axis is not stalling. I am probing with G28.3 Z-1.5 F25.

The axis moves down from Z2 towards Z0, it touches at just above Z0, something like Z0.053. The probing immediately fails and it does not backoff. We then see a status report indicating we're now at Z-1.5 or sometimes it says we're at Z-10. But this only occurs after the probing error has been emitted. We're clearly at neither of these heights since the bit is just barely touching the stock.

So I don't think this is what you're describing above.

benjamind commented 7 years ago

Ok working through the cycle_probing.cpp and the gpio.cpp implementations its clear that removing the timeout is causing the issue above. However what can we do about the timeout causing the pin state to remain high? When do you resample the pin if the pin returns to non active state during the timeout?

benjamind commented 7 years ago

I do really think that you need to register a function to be called after the timeout expires, such that you can then re-sample the pin to ensure that you catch the case of the pin going low then returning high during the timeout window.

giseburt commented 7 years ago

If there is a pulse of the signal then there is an electrical error. I do agree that we need a periodic check to ensure rhat, after the timeout, the signal state is what it was before the edge. It wouldn't help you much in this case since the action to take if it's found to have changed is to throw an error.

Just for the sake of example, and let's say the switches are active low. 1) They are normally high 2) An edge from high to low is detected, action is taken 2.1) the lockout is set 3) any edges during the lockout are ignored 4) the lockout ends, the line needs to be low, else the edge was an erroneous pulse

If the switch erroneously pulsed then for some other reason the signal went back to high even though the probe is in position to be in contact with the sensor. On Sun, Jun 4, 2017 at 4:00 PM Ben Delarre notifications@github.com wrote:

I do really think that you need to register a function to be called after the timeout expires, such that you can then re-sample the pin to ensure that you catch the case of the pin going low then returning high during the timeout window.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/synthetos/g2/issues/126#issuecomment-306066645, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXj0Y-bg_KYMmHsBv07lLun2RvRcFLsks5sAxrbgaJpZM4HiIGQ .

benjamind commented 7 years ago

The logic of your scenario is fine in a perfect environment. However, there's one edge case that you're not considering and I think is what I'm seeing here.

  1. Z is lowered until contact is made closing the switch and moving the pin from high to low
  2. edge transition is detected, action is taken and lockout is set
  3. the bit moves away from the surface of the plate because its a spindle and can move freely, contact can be very very momentary especially with single flute spiral bits which have a very sharp leading edge point
  4. switch is 'open' pin moves from low to high
  5. timeout expires after transition from low to high but because the pin is already high when Z is raised there is no interrupt to reset the recorded state.

A periodic check would totally fix this.

benjamind commented 7 years ago

Ok, so I reverted to the edge branch no changes and ran the following test with my scope attached to the DI5 input:

  1. fresh state (just powered on and connected)
  2. reset Z0 just to be sure (G28.3 Z0)
  3. start probing with G38.2 Z-20 F21
  4. struck the probe against the touchplate
  5. Captured the following trace of the touch event on DI5 http://goo.gl/mZBzfU
  6. Observed following output in console:
    G28.3Z0
    {"r":{},"f":[1,0,8]}
    {"sr":{"posz":0,"stat":3}}
    G38.2 Z-20 F21
    {"r":{},"f":[1,0,15]}
    {"sr":{"posz":-0.065,"stat":7}}
    {"sr":{"posz":-0.153}}
    {"er":{"fb":100.26,"st":250,"msg":"Probe cycle failed - probing failed"}}
    {"prb":{"e":0, "z":-20.000}}
    {"sr":{"posz":-20,"stat":2}}

Probing clearly failed here, and I can't see why. The trace looks reasonable, the cap + resistor network I have on the pin (0.1uf cap to gnd, 220k ohm resistor to vcc) seem to be doing their job. Total length of the 'active' time on the trace here is about 28ms so should not have fallen foul of the lockout and after clearing the alarm and issuing $in I can see that the state is correct (i.e. $in5 = false) so its not the lockout at play here.

Any ideas?

Additional info:

$di:

{"r":{"di1":{"mo":0,"ac":0,"fn":0}},"f":[1,0,4]}
{"r":{"di2":{"mo":0,"ac":0,"fn":0}},"f":[1,0,1]}
{"r":{"di3":{"mo":0,"ac":0,"fn":0}},"f":[1,0,1]}
{"r":{"di4":{"mo":0,"ac":0,"fn":0}},"f":[1,0,1]}
{"r":{"di5":{"mo":0,"ac":0,"fn":4}},"f":[1,0,1]}
{"r":{"di6":{"mo":2,"ac":0,"fn":0}},"f":[1,0,1]}
{"r":{"di7":{"mo":2,"ac":0,"fn":0}},"f":[1,0,1]}
{"r":{"di8":{"mo":2,"ac":0,"fn":0}},"f":[1,0,1]}
{"r":{"di9":{"mo":2,"ac":0,"fn":0}},"f":[1,0,1]}

I also reproduced this a bunch of times, with much longer traces to ensure there were no other errant signals on the pin and the trace is perfectly clean as you'd expect. I've confirmed that probing is working with long deliberate touches, but they have to be very stable, which just isn't the case in reality.

benjamind commented 7 years ago

Any further thoughts on this? Would love to get probing working properly.

aldenhart commented 7 years ago

Ben, Still looking into this. Thanks for the ping.

benjamind commented 7 years ago

Let me know if there is anything I can do to help / test.

giseburt commented 7 years ago

Could you re-post that trace? In the github web interface you can drag images to the text entry box to upload them.

On Jun 4, 2017, at 7:04 PM, Ben Delarre notifications@github.com wrote:

Ok, so I reverted to the edge branch no changes and ran the following test with my scope attached to the DI5 input:

fresh state (just powered on and connected) reset Z0 just to be sure (G28.3 Z0) start probing with G38.2 Z-20 F21 struck the probe against the touchplate Captured of the touch event ![trace of di5 pin] (https://lh3.googleusercontent.com/-ZPodzBEd8dhis69MNmzCcba2jls5y9yga88NI2zb-2MiCt8QV-X63yzWZ2piF8U48lo5x6eCY-ghtur_ix6HG2JG9fAwzflQx6SgcbVELvW0Ws3d4KAQ5gsFW7X15QTFQHECDdprY-VExppFWd4x-7ltFpWcy7hRlHycOn-if2S-JCRVkPxdERM4ZXgdqhrwc9bsSy6loDFPBrerkUhjGwoRHtTF_JLux2PkvXi_cm9KJ8MMWMXSh3XedL1Yf3OEyZApn7EePkOmgGd4TNK7rRY2vWMc1E8SXvfC2U8H1xQ1RbBi71pYw2xjCkYaGQxq46DBWUC99dHKRhGzXvpr6dBIvt_eUCS0dS9IVRFolWW5mwpZyG2XIbxuCYpT0xPeaQupphmhs3_LiAMYPuCJQuESKN39MnCXTYXd7VtaumCCnb-yXhfUtEqwTyBhV8aLe1Qs5rq976P981GtMT-7m-X0VPSpEv2hqCXXmyWe2rFR0cBaTEukgI-uE9Hp-5-XAisXROTdJflk3e47ec6ZEXmM6DhEnqLT2N3huUdMj7Pu5QQeR7NahOx4MhDeBBjXZL7_6WULsOnx65sjs212wMb_U05s-XQkbtaqzZfz8QOaac0QI1m=w1134-h1510-no https://lh3.googleusercontent.com/-ZPodzBEd8dhis69MNmzCcba2jls5y9yga88NI2zb-2MiCt8QV-X63yzWZ2piF8U48lo5x6eCY-ghtur_ix6HG2JG9fAwzflQx6SgcbVELvW0Ws3d4KAQ5gsFW7X15QTFQHECDdprY-VExppFWd4x-7ltFpWcy7hRlHycOn-if2S-JCRVkPxdERM4ZXgdqhrwc9bsSy6loDFPBrerkUhjGwoRHtTF_JLux2PkvXi_cm9KJ8MMWMXSh3XedL1Yf3OEyZApn7EePkOmgGd4TNK7rRY2vWMc1E8SXvfC2U8H1xQ1RbBi71pYw2xjCkYaGQxq46DBWUC99dHKRhGzXvpr6dBIvt_eUCS0dS9IVRFolWW5mwpZyG2XIbxuCYpT0xPeaQupphmhs3_LiAMYPuCJQuESKN39MnCXTYXd7VtaumCCnb-yXhfUtEqwTyBhV8aLe1Qs5rq976P981GtMT-7m-X0VPSpEv2hqCXXmyWe2rFR0cBaTEukgI-uE9Hp-5-XAisXROTdJflk3e47ec6ZEXmM6DhEnqLT2N3huUdMj7Pu5QQeR7NahOx4MhDeBBjXZL7_6WULsOnx65sjs212wMb_U05s-XQkbtaqzZfz8QOaac0QI1m=w1134-h1510-no) Observed following output in console: G28.3Z0 {"r":{},"f":[1,0,8]} {"sr":{"posz":0,"stat":3}} G38.2 Z-20 F21 {"r":{},"f":[1,0,15]} {"sr":{"posz":-0.065,"stat":7}} {"sr":{"posz":-0.153}} {"er":{"fb":100.26,"st":250,"msg":"Probe cycle failed - probing failed"}} {"prb":{"e":0, "z":-20.000}} {"sr":{"posz":-20,"stat":2}} Probing clearly failed here, and I can't see why. The trace looks reasonable, the cap + resistor network I have on the pin (0.1uf cap to gnd, 220k ohm resistor to vcc) seem to be doing their job. Total length of the 'active' time on the trace here is about 28ms so should not have fallen foul of the lockout.

Any ideas?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/synthetos/g2/issues/126#issuecomment-306076527, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXj0XvohndO6D8Zzer8CqzsJjQ6ZWhbks5sA0YVgaJpZM4HiIGQ.

benjamind commented 7 years ago

Sure thing:

trace

giseburt commented 7 years ago

So it looks like the pulse is ~3ms long? If you set the lockout to 1ms does it work?

As long as you don't have any additional bounces after that 1ms it should work. Baically you want the lockout to be more than any bounces and less than when it'll legitimately go high again. On Fri, Jun 16, 2017 at 3:18 PM Ben Delarre notifications@github.com wrote:

Sure thing:

[image: trace] https://user-images.githubusercontent.com/332843/27243367-56d08536-5296-11e7-8c2d-658787b23570.jpg

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/synthetos/g2/issues/126#issuecomment-309124437, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXj0dp6CWkgHGJ0KsaARM94TnRDPX4eks5sEuMugaJpZM4HiIGQ .

benjamind commented 7 years ago

No the pulse (the period when the signal is 'low') is about 25ms long. Its less than the lockout period. Each grid square on that trace is 5ms

Tegeon commented 7 years ago

It takes ~33msec for the whole touch-release cycle (signal edge-to-edge) based on the oscilloscope output, while the transition from H to L is very short, surely far less than a msec. Once I started using optocouplers on my TinyG over a year ago, I had no single noise related issue since that time. Possibly applying the same approach would help debugging, assuming electrical noise is not an issue anymore. The edge transition time based on the optocoupler output characteristics is in range of 5-100 microseconds, depending on the load resistance.

What tool(s) is needed to allow full in-circuit debugging here, Atmel ICE?

aldenhart commented 7 years ago

Atmel ICE will do it. It's integrated with Atmel Studio and works with native GDB if you go that way.

Can you post details of your opto circuit? Devices and coupling network?

benjamind commented 7 years ago

I think from looking at the code as long as the signal stays low for longer than the lockout time it should work. And I can confirm that if the signal goes back up to high before the lockout expires then it stays stuck high - that's a bug that definitely needs a fix but is probably separate from this issue.