jflight-public / jesc

48 stars 20 forks source link

Motor wires de-soldering on Nameless AIO 412(T) #6

Closed stylesuxx closed 4 years ago

stylesuxx commented 5 years ago

I came across a very strange issue - unfortunately not personally, but I have now heard from two people that the solder joints of the motors get so hot after flashing JESC that the motor wires de-solder. Re-flashing the original version of BLHELI_S does not seem to solve the issue.

One of those cases (and attempt of debugging) is documented in the comments here => https://brushlesswhoop.com/blheli_s-rpm-filter/ NLRC 412T, 1206 motors on 3S, BF4.1 RC1.

Another case I came across on Facebook just now - NLRC 412T, 1304 motors on 4S.

Interestingly enough I have no problems on my NLRC AIO 412 with 1103 motors on 2S, BF4,1 RC1.

I would like to help debugging, but I am not sure which information I can help out with.

Ericarias84 commented 5 years ago

I am the guy with the 1304 motors. Very odd issue. I am running this firmware on a bunch of other builds without issues (some AIO fcs). After I flashed the firmware, the motors initially armed but than I got some smoke from the board and it appears some of the escs burned out.

etracer65 commented 5 years ago

@stylesuxx

Re-flashing the original version of BLHELI_S does not seem to solve the issue.

That would seem to rule out the ESC firmware don't you think?

@Ericarias84

Sometimes motors or ESCs just fail.

Make sure you've set the motor_poles parameter in Betaflight correctly. Little tiny motors like that may not be the default 14 poles. If you don't have this set properly the RPM Filter won't work properly and it's possible that the motors will get hot. But it wouldn't cause a motor to smoke instantly.

stylesuxx commented 5 years ago

@etracer65 the first case was working before flashing JESC and would still overheat even after being reverted to it's original state (BF and BLHELI_S). The motor_poles were set correctly to 12.

Also the motors were not getting hot at all, it seems it was the FET's that got hot and thus de-soldered the motor wires from the flight-controller.

etracer65 commented 5 years ago

and would still overheat even after being reverted to it's original state (BF and BLHELI_S)

That's my point. This would rule out a firmware problem and point to a hardware problem.

Edit: Or possibly the wrong firmware file used in both cases. The BLHeli_s firmware defines important operating parameters (the numbers in the file name). If the wrong values are used it can damage the hardware. Also wouldn't be the first time a manufacture shipped an ESC with the wrong settings.

stylesuxx commented 5 years ago

OK, so you are saying it was just a coincidence that the problems started after flashing JESC?

Wrong firmware version was also my first guess but we double checked that afterwards.

Unfortunately I can only provide second hand information - just wanted to post here to make people aware of a potential problem with this constellation. Since it happened two times already, chances are that it might happen again.

Thanks for your input.

stylesuxx commented 5 years ago

So I just spoke to NamelessRC. One of their engineers looked into it and said that you should use ESC version P-H-90. Both boards come flashed with P-H-15.

So @etracer65 was right with his assumption that the manufacturer shipped the board with the wrong settings. Although I do not understand why this only became an issue once flashed with JESC and was not an issue from the beginning?

Find attached a screenshot of my conversation with NamelessRC on facebook.

nlrc_jesc

Ericarias84 commented 5 years ago

That is interesting. How did you and a few other people get it to work on your Nameless board? Which version do you have flashed?

stylesuxx commented 5 years ago

From what I understand, only people with 2S got it working without issues. I flashed P-H-15 of JESC, it came with P-H-15 of BLHELI_S. As it seems I was just lucky with my 2S setup not to run into any problems.

I have heard from another guy, also running 2S but DSHOT600 that he was getting temp warnings in beta. This might be an indicator too, that it is running too hot even on 2S.

EDIT: I just flashed mine to P-H-90 and I can not feel a difference in flight with my 85mm whoop. BUT I now have longer flight times with my 300mAh 2S batteries.

etracer65 commented 5 years ago

@stylesuxx The page that's apparently related to you at: https://brushlesswhoop.com/namelessrc-nlrc-aio-412t-jesc/ is very misleading.

The only news here is that the manufacturer shipped the ESCs with the wrong firmware settings. The settings they supply were incorrect enough to cause physical damage to the ESCs. The deadtime is derived based on the electrical characteristics for the FETs based on the manufacturer design and datasheet. It is not something that would be set differently for different firmwares or something you would "tune". The manufacturer just screwed up. If I owned one of these ESCs I would be contacting the manufacturer for replacements even if they haven't failed yet because they're already likely damaged to some extent.

Is it possible that people flying with RPM filtering enabled using JESC might be working the ESC harder due to better flight characteristics? Sure. Could this put more stress on the already marginal ESC? Sure. But that's not the underlying cause. The incorrect deadtime is.

joelucid commented 5 years ago

Hey guys - great you got this issue sorted out. Not sure why the issue started when installing Jesc. Jesc uses the very same code as blhelis for managing pwm in 24khz pwm mode. The experimental 48khz mode has some differences but none that could cause this.

Beyond coincidence the only explanation I can come up with is that the manufacturer installed a firmware that was tagged as P-H-15 but was really a P-H-90. This would cause exactly the issues described here: it would work as purchased but Jesc would install the P-H-15 version and even after flashing back to blhelis you would now have a version that doesn’t work. Any update in blhelis configurator would have led to the same outcome in that case

Etracer is also right: it’s entirely possible that the esc went bad over time and just started to show symptoms by coincidence when Jesc was installed.

stylesuxx commented 5 years ago

@joelucid I was also suspecting that it might simply be tagged wrongly, but in the meantime NamelessRC replied and stated that no - they in fact do flash P-H-15 in the factory: nl_statement

I now asked him why this would be necessary in their opinion. Is there any way I can (easily) dump the current firmware from the firmware to compare the hex files.

@etracer65 thank you for your feedback regarding my article. I now added a disclaimer that people should not flash different versions in attempts of "tuning" their quadcopters.

As you can see from the above screenshot the manufacturer states that P-H-15 is indeed the correct firmware that is flashed in the factory and that we should use P-H-90 if we want to use JESC. I also did ask him about his reasoning for this and that people here are of the opinion that a different deadtime for JESC makes no sense.

I also invited their engineer to join the discussion here.

joelucid commented 5 years ago

@Ericarias84, did you use the experimental 48khz mode or 24khz?

joelucid commented 5 years ago

I found an issue that the experimental 48khz pwm mode scaled the dead time down in half. This needs to be done for any power related variables since the pwm runs twice as fast. But not for the dead time.

@Ericarias84, if you did in fact use the experimental 48khz pwm version that could explain the issue. This would also explain why some people don't have a problem (the ones on the stable 24khz pwm version) and others do (users of the 48khz pwm beta).

joelucid commented 5 years ago

I pulled the 48khz beta for now until I have the time to address the issue.

joelucid commented 5 years ago

Just flew 10 packs through the fixed 48khz pwm Version and had no issues. Would anybody like to test this out in the nameless aio? I can provide a hex.

stylesuxx commented 5 years ago

@joelucid the first guy that contacted me, flashed JESC according to my screens - which show 24kHz (which is also the default selection). Screenshot from 2019-09-19 13-19-27

I'd try it, but the Problem does not show on my 2S build anyways, so I am not of much help.

joelucid commented 5 years ago

@stylesuxx - ok. Well it still seems he's unsure so who knows. Many people are in the bigger is better dept - so maybe it was an unconscious choice. I've flown the 24khz version so much that I'm sure there's nothing similar in there.

Ericarias84 commented 5 years ago

I also used 24 when I flashed.

joelucid commented 5 years ago

Well - that brings us back to the hypothesis that P-H-15 might not be a suitable fw for the esc. 24khz pwm mode has been flown on hundreds of quads without this issue. It does seem to be hw specific then.

stylesuxx commented 5 years ago

@joelucid OK - so which options do we have to confirm this hypothesis?

Any other - possibly non destructive options? Maybe I should try and measure the temperature of the FET's on the bench with 2S @ 20% throttle: let the motors spin for 20s, measure the temp of the FET's with JESC, flash BLHELI_S and re-do the measurement. Not sure how accurate the measurement would be - what do you think?

joelucid commented 5 years ago

Maybe I should try and measure the temperature of the FET's on the bench with 2S @ 20% throttle: let the motors spin for 20s, measure the temp of the FET's with JESC, flash BLHELI_S and re-do the measurement. Not sure how accurate the measurement would be - what do you think?

Yeah that's a good idea. You don't even need to connect motors. The dead time prevents the upper and lower side FETs to short the power supply while switching - so it's independent of any external load. This makes it very easy to spot: no load, just a bit of throttle in the motors tab. If the dead time setting is ok the fets should not get warm. If the fets get warm you should have enough time to pull the motor back to zero to prevent any damage.

If it kills the ESC on 3s it should be enough for the ESCs to get warm on 2s. You can then also check if there's any difference if you use P-H-90.

stylesuxx commented 5 years ago

All measurements at 1150 throttle.

Variant Temp in Celsius
BF 4.1, BLEHLI_S - STOCK 42
BF 4.1, BLHELI_S - flashed P-H-15 42
BF 4.1, BLHELI_S - flashed P-H-90 40
BF 4.1 JESC - P-H-15 (without telemetry flashed) 42
BF 4.1 JESC - P-H-90 (without telemetry flashed) 40
BF 4.1 JESC - P-H-15 (with telemetry flashed) 40
BF 4.1 JESC - P-H-90 (with telemetry flashed) 40
BF 4.1 JESC - P-H-15 (with telemetry flashed & RPM filtering enabled) 65 almost instantly - did not reach 1150 throttle
BF 4.1 JESC - P-H-90 (with telemetry flashed & RPM filtering enabled) 42

I tripple checked all the measurements.

RPM filtering enabled are those settings:

# set looptime to 4k to have enough computational time
set gyro_sync_denom = 2
set pid_process_denom = 1

# Start with dshot 300 which is most robust
set motor_pwm_protocol=DSHOT300

# basic settings required to enable rpm filtering:
set dshot_bidir=on

# enable debugging for noise assessment
set debug_mode=gyro_scaled

# reconfigure dynamic notch filter to focus on narrow frame resonances
# since motor noise will already be eliminated by the rpm filter.
set dyn_notch_range = medium
set dyn_notch_width_percent = 0
set dyn_notch_q = 250
joelucid commented 5 years ago

@stylesuxx sounds like a plan! There's probably not much of a need to test BF 4.0 though.

stylesuxx commented 5 years ago

@joelucid Yes, I thought that too, that's why I only tested with BF 4.1 in different constellations - I edited my initial post already btw.

joelucid commented 5 years ago

Sorry didn’t see you already had results. Well this certainly is very fascinating. I’ll try to reproduce. Thanks for testing!

joelucid commented 5 years ago

So far I can't reproduce this with my debug escs. Anybody know how to get a hold of a Nameless AIO 412(T) in Germany?

stylesuxx commented 5 years ago

@joelucid I'll send you mine - I am from AT. Message me: stylesuxx@gmail.com But I would like to have it back if at all possible.

joelucid commented 5 years ago

I have prepared one test we could still try: could you flash the Telemetry hex called Test version (internal use only) using jesc configurator? I've disabled actually sending telemetry but this still works with the inverted dshot protocol. That way we can eliminate that as factor.

stylesuxx commented 5 years ago

@joelucid stays at 40C with this telemetry hex.

joelucid commented 5 years ago

One more idea: go back to the normal Telemetry hex but set your loop time to 1k.

stylesuxx commented 5 years ago

Gets slightly less hot than with 4k.

Ericarias84 commented 5 years ago

Question for you guys not related to this problem. I have a 6s build running 32bit escs. I am running RPM filter firmware on the escs, is it safe to use 48 PWM frequency or should I stick with 24?

RadianceNL commented 4 years ago

Is there any news on this? I just received my board and would like to flash JESC with telemetry but obvious don't want to fry my board.

joelucid commented 4 years ago

I’ll be back from vacation tomorrow. Hopefully Chris board will have arrived so I can run some tests.

RadianceNL commented 4 years ago

Thanks for the quick reply. I'll be patient. Weather here is crap anyways. :-)

joelucid commented 4 years ago

Alright, the board arrived here and I did some first tests. I flashed different FW's and held my thumb on the escs while increasing throttle to about 1150 in the motors tab. I used a 2s battery. With P-H-15 the FETs became hot very quickly in all cases (BLHeli_S, JESC + Telemetry + Bidir on and JESC + Telemetry + Bidir off). With P-H-90 the FETs didn't get hot in any case at all.

Personally I couldn't observe a huge difference between the different tests that did get hot. I'm inclined to think that the FETs just need a higher dead time than 15. Unfortunately the FETs don't seem to be marked so I can't look them up in a datasheet.

I'll try to contact nameless to get the FET specs.

joelucid commented 4 years ago

BTW even P-H-30 was already much, much better. Probably not necessary to go to P-H-90.

stylesuxx commented 4 years ago

Interesting - that contradicts my tests. And brings us technically back to square one: Why were people able to fly it without issues before flashing but not after flashing JESC? So all of the people that smoked their FET's after flashing JESC had just bed luck, or do the FET's have to work harder after flashing JESC and having RPM filtering enabled?

Thank you for looking into this and thank you for contacting nameless. I always hit them up on facebook and they tend to respond rather quickly.

joelucid commented 4 years ago

Yeah @stylesuxx it is strange. One thing that's different in jesc is that I only apply the new rc packet to pwm once the telemetry packet is sent. That could influence timing a bit.

The good thing about what I found is that it is consistent with the other data - we haven't seen this issue on any other esc. Hopefully nameless will provide the FET specs so we can confirm what timings should be consistent with the switching speed of the FETs.

stylesuxx commented 4 years ago

I hope so too - but since it would probably mean, that they have to admit a mistake I am wondering if it will happen.

Did you test if there is any difference in temp with P-H-30 when using 3 or 4S?

joelucid commented 4 years ago

So far I only tested 2s. No response from nameless on FB thus far.

joelucid commented 4 years ago

Ok - new insights: tried blhelis 16.7 P-H-15 on 3s and the fets get too hot to touch immediately when applying power.

I also asked around and @conuthead provided this input:

@Joe Lucid Is that a passive design? ( no gate driver) . If it is the circuit I think it is .. it uses p channel upper fets and is super slow to turn off. I use the _90 dead time.

Looking at the board it does indeed look like there are no gate drivers:

Image from iOS (2)

Finally nameless answered this morning and said they wouldn't share the FET specs. They also gave this advice: image

Which kind of sounds like not any P-H-15 fw will do - it has to be the one they preinstalled. I probed further but they went silent.

Unless I hear anything else from nameless I think anybody updating their FW on this esc should use P-H-90 as they recommend for jesc. And that includes updates to BLHeli_S FW.

joelucid commented 4 years ago

Here's the top of the board. It has 3 transistors per esc. That makes it likely this design (https://www.rcgroups.com/forums/showthread.php?2898397-Another-strange-BlHeli_S-1-3s-DIY-esc) which is the one that requires a dead-time of 90.

Maybe there is an incentive to make ESCs appear to switch faster than they actually can for marketing purposes.

Image from iOS (4)

stylesuxx commented 4 years ago

Great detective work - thank you. Interestingly enough they do not post ESC timings anywhere on their product pages. But them telling us not to upgrade BLHEL_S at all really sounds strange.

I guess our only way to find out the truth would be dumping the ESC FW from a new board and compare that to a known P-H-15. I guess that's not worth anyone's time, better get the info out to only use the P-H-90 FW no matter with BLHELI_S or JESC.

joelucid commented 4 years ago

Yeah - if this sort of thing happens more often I would need to implement some kind of checksumming of the installed fw to avoid upgrading mistagged versions. But so far it’s just this one. Getting the word out would be good. I’ll add it to the install instructions. Maybe you could update your readership as well. I’ll keep the board for a couple of days in case anything comes back from nameless if that’s ok with you.

stylesuxx commented 4 years ago

Will do - I will update this article with your new findings. Sure, keep it another week or two.

RadianceNL commented 4 years ago

I have a fresh board where I didn't do anything with the ESC yet so could make a dump if someone can explain how to.

joelucid commented 4 years ago

@RadianceNL which platform do you run jesc-configurator on? I could make a version that extracts the firmware.

RadianceNL commented 4 years ago

Windows 10

Just realized that I did change the motor direction of one of the ESC but on the firmware that came from the factory.

joelucid commented 4 years ago

Ok, please download https://jflight.net/downloads/JESC-Configurator_win64_1.2.4.zip.

Start the jesc configurator, connect to esc and say "Read Settings". It'll take a lot longer than usual. When done save debug log and post here, please.