sim- / tgy

tgy -- Open Source Firmware for ATmega-based Brushless ESCs
http://0x.ca/tgy/
688 stars 387 forks source link

Blueseries 30A with mulitstar 690kV fails during operation #25

Open Sp1k3 opened 11 years ago

Sp1k3 commented 11 years ago

Actually truth said, I have been using this ESC for quit a while now almost a year. And this weekend all of a sudden the ESC stops working in midair. Clearly upgraded with Simonk firmware. However it has been giving me signs, I could see the gopro video twitching, and small oscillations on the quad when flying. I was convinced it had to be one of the motors. But I found out the hard way it was an ESC failing. After examining the schematic, I couldn't help to notice that the AVR is directly driving the Low side Power MOSFETs. Now that the frequency is significantly higher ~18kHz, I tend to think this could be the cause of the failure... I believe the driver is not capable of delivering so much ac current at such high frequency, but that it must be beyond its limits. Truth said there is some resistance before the gate, but once grounded the first spike of current is significantly higher than the DC specified in the spec of the 8A (40mA). However I still have to confirm this theory. Im going to replace the AVR on the failed device and reflash it, then see if this is what failed. In any case, I do not believe anything else could have failed. My avg current was way lower 30A, ~6A. After looking at the power FETs they all look ok, but I will give you more ifno as soon as the new AVRs arrive.

sim- commented 10 years ago

Hello! Most ports of the ATmega8 are capable of sourcing and sinking quite a bit continuously, like 200mA or so. But this is a DC rating. 18kHz is not a very high frequency for logic, and the capacitance at the gate for these size of FETs is not that significant.

I have yet to see an ATmega8 fail because of this, and the only <= 30A ESCs I've seen that don't do this are the TowerPro (clones? at HobbyKing), which I think even they changed recently. Even the proper drivers like the IR2101S don't have much better specifications -- check the data sheet.

Anyway, your failure case sounds strange. Are you using OpenPilot firmware, by chance? (It has been showing weird drop-outs on all motors for me since I upgraded the firmware the other month.)

If you'd like to confirm the hardware through software, you could try enabling CHECK_HARDWARE in tgy.asm. Otherwise, I typically hot-air off all of the FETs and test them all before rebuilding.

Simon-

Sp1k3 commented 10 years ago

Hi Simon, First of all, let me tell you how much I appreciate your work and for the great job you are doing with code writing!

Well this was first my theory as to what has failed. However to confirm this theory of mine, I replaced only the AVR reflashed it with simonk, and set the proper fuses, then it worked again perfectly. So the fault was definitely in the AVR. I checked the inputs and all are protected by resistors. So by exclusion there isn't much else that can fail in an IC. Let me give you a little history on this device. I have flashed it about a year ago with your firmware and since then it was on a platform working perfectly for about a year. Then lately I started observing small disturbances in the video, then I actually saw the quad having small disturbances in flight indoor, thus without other possible disturbances. I thought it might have been one of the motor bearings, but in reality it was the ESC and if failed in mid air. So I wanted to understand why this happened. So I looked at the schematic and saw the components, and figured the most likely cause was the AVR. After replacing it the ESC worked again thus proving its failure. Unfortunately I cannot have it opened at work to analyze what failed. But as described above I believe it is the IOs that failed. Dont get me wrong. I'm not blaming your firmware. I think its a bad design to begin with. The Atmega spec gives continuous current sourcing at 40mA. Now the 30A ESC is the one with most danger, because it has actually 3 DMOSs with 100Ohm in series to each gate capacitance. When the IO goes high, the DMOS gate is at zero, the the IO sees both the full voltage (5V) and the current spike of (5/100/3) which is ~150mA, this generates a power dissipation in the IC on the IO driver of .15*5=0.75W. One would need to integrate is over time to find out exactly how much energy=heat the driver is dissipating per cycle. I think the ESC is simply badly designed to begin with, and that the driver is degenerating over time, and your code just speeds the degradation process. However so far it is only one device that failed this way. I have others now on this platform from the same date as the other. So I will continue flying with them to see if any other fails... for now I really dont see much you can do. If this really is an issue, we should simply recommend people not to flash these 30A ESCs as they aren't reliable. The 40A ESCs have a buffer in between which reduces the strain on the uC, and takes it up himself, but these IR2101s are designed for that. the 20A ESCs have only 2 MOSs to drive with 100 ohm, so obviously this is also better in terms of reliability.

Sp1k3

owenduffy commented 10 years ago

There are several fundamental flaws in this analysis. The gate circuit has a time constant of something less than 1µs, and the very short gate charge pulses at 16kHz are not likely to cause much dissipation. This is evidenced by the fact that the MCU is usually quite cool.

This just whips up FUD (fear, uncertainty and doubt) unnecessarily. Empiracal evidence is that these things are not failing at a significant rate with this symptom.

Owen

sim- commented 10 years ago

Do you still have the old ATmega8? It would be interesting to see if any of the pin driving is broken. I have never actually seen this happen -- only shorting of the protection diodes between GND and the pin an the pin and Vcc.

As for glitchy failures, it should work until it doesn't. I'd be much happier with the cause being related to the oscillator, as moisture and/or physical stress on it can cause all sorts of problems. Touching it with a finger is enough to change the frequency and/or stop it, sometimes very briefly or in strange ways, that can make it appear glitchy. The watchdog is enabled to try to prevent damage in this case, but the minimum response time setting is 16.3ms, which is kind of long if FETs are enabled. This wouldn't stress the I/O pins any more, though, unless the gate actually fails on one or more of the FETs.

Sp1k3 commented 10 years ago

After thinking about it a little more, I realized that most of the power dissipated during switching is in the external resistors and not the driver itself. So or the driver failed due to elettromigration, which would be due to too elevated RMS current, or as you said something else failed. However I have a hard time believing that the RMS is above 10-20mA. I can exclude issues with external DMOSs , as I measured the oxide resistance to be healthy MOhm, 2 fell in the >300kOhm. I can also exclude the oscillator as now there are no glitches at least in the preliminary tests, I would need to run further tests on a platform, as the frequency of occurrence was not so high, once every15 sec at the end, beginning was around once every 5min or so... Unfortunately I threw away the device... That wasn't too smart... In any case I think it might have been a faulty device. Which was bound to fail. the glitches were becoming more and more frequent till failure. They were just brief moments. I think that the 16ms would be sufficient to cause a glitch in the behavior of the platform to be visible to the naked eye and video...

Den100 commented 10 years ago

i have some idea about 22 poles motor. i confirm issue with HK BS 12a esc. the rotation magnetic fields opposite rotating rotor, i mean - if rotor have RHrotation, magnetic fields rotate LH. if motor have 26 poles opposite 22 the rotation will be the same RH-RH or LH-LH. So timing in one case must be "+" on other case "-". in case 22 poles 24 stator motor ESC must have DELAY (~2,5-3,5 Degree), i mean then esc triggering from "sensor" coil it must waiting 3 degree and switch next coil, if motor will be have 26 poles esc must predicted i mean it must switch coil 3 degree before sensor coil will be triggered. pls try to contact with SK and ask rebuild firm ware - i think it very easy can be done. also what timing used it SK firm ware (constant, variable, what degree). i will be glad to see 0 degree timing firm ware (HK BS 12a) if DELAY timing FW demand some time.

sim- commented 10 years ago

Hi Den100. Are you asking for a separate timing advance based on rising versus falling zero-crossing detection? I'm not quite sure what you are asking.

sim- commented 10 years ago

What picture?

Den100 commented 10 years ago

12-16 or 24-22 first top left, then CW if controller can trigger Maximum Minimum Sin wave then according my suggestion, it must trigger Max/Min (actual moment them Voltage change direction in coil), it heppen then magnets in position TOP RIGHT. count 3 degree delay (bottom right), turn-on next phase.

sim- commented 10 years ago

Hello! I am sorry, I still do not understand what you are trying to explain. Are you trying to describe a problem specific to higher pole count motors that would not happen in the case of a 2 pole, 3 tooth motor, for example?

Note that the software really has no idea how many poles there are, so it drives the motor the same way regardless. The term "electrical degrees" is based on the minimal case, which is why a commutation step is referred to as 60 degrees (there being 6 possible steps for a full revolution). So, what do you mean by ~3 degrees? Are you referring to that many degrees assuming a 22 pole motor?

owenduffy commented 10 years ago

The first diagram with 24 slots and 24 poles will during one step of a 6 step commutation cycle, have current flow into one wire and out of another, and the third wire is used for sense.

The diagram shows four adjacent coils labelled "sens", that is not the way the armature is wired. At any step in that cycle, every third coil is a sense coil, 8 in total in this case.

Any theories based on incorrect wiring of the motor will be flawed!

Owen

Den100 commented 10 years ago

pls try answer for my question can ESC trigger Sin voltage Max or Min i mean moment then voltage stop raise/fall and start fall/raise with out hardware rebuilding? hk-mystery20a_sch schematic in pic.

Hello! I am sorry, I still do not understand what you are trying to explain. Are you trying to describe a problem specific to higher pole count motors that would not happen in the case of a 2 pole, 3 tooth motor, for example? - actually no, i try explane that motor can be 2P 3T or 4P 3T, and driving this motor must be different. In case 4p3T or most use 14P 12T esc for timing using zero-cross detection, but if Poles<Tooth esc must trigger Min Max sin wave. 28-24

owenduffy commented 10 years ago

Yet another wiring diagam for the motor, but it is not how they are wired!

Owen

Den100 commented 10 years ago

In zero cross pic 28p24t motor - analog 14p12t. Yes - it is... they wired like in pic. if not - post right wiring PLS let me get A for my Question

Den100 commented 10 years ago

So, what do you mean by ~3 degrees? Are you referring to that many degrees assuming a 22 pole motor? physical degree according 22 pole pic. in electrical it ~37

Den100 commented 10 years ago

Checked - problem in motor. I take same motor, 4114 size of stator. 185KV Y 22h24T, and run under 31V - no issue with low and medium timing HW 40A pentium ESC. i think HK motor not symmetrical enough, because magnets have covered with nickel and shape not sharp as in 180 KV motor. so problem not in electric - problem in mechanic of HK 22 poles motor.