Hardware compatibility - Githubissues

romaintaillandier1978 commented 3 years ago

Hello

I should present my goals. We are trying to embed a rapberry pi 3B, with a 4G hat, and the pijuice, to monitor an ip camera connected to the raspberry with a classic ethernet cable. That device will be powered by an external stable supply. Camera will be powered by another separated supply. The 4G hat is SixFab LTE hat, with a Telit module LT910C1 https://sixfab.com/product/raspberry-pi-base-hat-3g-4g-lte-minipcie-cards/ https://sixfab.com/product/telit-mini-pcle-4g-lte-module/

We have try 2 stack combinaison :

Raspi (at bottom), pijuice, 4G hat (top)
Raspi (at bottom), 4G hat, pijuice (top).

Pijuice is powered by its micro USB. (We also have try to power throw GPIO, with power on raspi, globally similar results)

Raspi is connected to 4G hat by the pin bus, and using a usb cable (like in this tutorial https://docs.sixfab.com/docs/getting-started-with-base-hat-and-telit-le910c1-module), with pi juice on the top, or between both. Rasberry is embed with a raspbian lite no desktop. System has a few tools, like open VPN, nagios nrpe plugin, iptables t nat configuration. nothing exotic.

Using the pijuice to do nothing, (no system task, no events connected) the whole system works perfectly. We want to use pijuice to clean power off the raspberry when (if) there is a power cut. And restart when power come back. Actually we are not able to perform this. Each time the pijuice power off the raspberry, the raspberry is not able to perform a clean boot. Raspberry is waked up, but the boot simply fail at various step. (I would add details if necessary)

We was able to setup the pijuice configuration that works as awaited with a simple raspberry (no 4G modem hat). and were able to create awaited configuration with 2 simple threshold to power off at 10% battery, and power on at 15% battery. We done this with a clean raspbian with desktop (factory configuration) for simplifing pijuice configurations. That 'partial device' was stable and functionnal. As awaited, remove power fired clean powered off (SYS_FUNC_HALT_POW_OFF) at 10%, and good restart at 15% few minutes after power comes back. Repeatable.

Now as soon as we add the 4G hat, we get many troubles.

Pijuice settings are not available anymore (no tray icon, click on menu/preference/pijuice do nothing)
network do not work anymore.

After a few days of testing various combinaison of pijuice parameters, that final test, by adding little thing until something goes wrong, make us feel that something is wrong about power, and pijuice is not working properly, may be because 4G hat pump too much power on GPIO ?

I would like to have your opinion, if possible. and may be few advices to make our solution workable, and reliable

Thank you for any help. Romain.

mmilann commented 3 years ago

Which battery you are using? It may be overloaded, modem can draw 1.5A-2A.

romaintaillandier1978 commented 3 years ago

I use the default battery BP7X 1820.

for example, when the i talked about pijuice that do not provide GUI anymore, this is with power supply, not under battery. I have also tried to provide 2 power : One for the pijuice, and one for the raspberry, with same results.

In my objective configuration, without trying to get the raspberry powered on, just a clean power off. Pijuice was able to get the system powered 15 minutes (pijuice on top) or near one hour (pi juice in middle). fully functionnal. And at power on, there is the power supply.

mmilann commented 3 years ago

BP7X can provide maximum of around 1.1A at 5v GPIO, definitely not enough for stable powering modem plus Rpi.

Do you mean that just PiJuice software is not working on desktop Linux, or also other desktop services shows errors?

Also another thing to check is current capacity of power supply/adaptor.

romaintaillandier1978 commented 3 years ago

Your concern about power is legal. My power is 2A My supply looks weak, i will try a 4A one tomorrow. But that definitely not explain most of what i observe.

1) Same configuration, raspberry, modem, pijuice. pijuice : no treshold for power on / power off, no task, no events. PiJuice is still there, but do not drive nothing. I manually halt the raspi when i want to halt. I click on power button of pijuice, to start the whole device. If i remove power, pijucie provide power with battery, until i halt manually. Everything is very stable and very reliable. everything is working perfectly.

2) When i add a battery treshold to halt and start, in pijuice. when I remove external power. Pijuice continue to power (with that battery) everything well. Everything works perfect. (even with only 1A). the whole device is fully funtionnal, over battery, and works perfectly. Pijuice success in halting raspberry, when battery treshold is reached. I put back the power supply. Pijuice success to electrically restart raspiberry, when battery treshold is reach. And i can see boot sequence on screen. But boot sequence can not be terminated.

But I know pijuice can do what i want, because of previous tests 1. What is different between

'sudo halt' then click pijuice SW1 to start (1)
stop on treshold, and start on treshold (2) ?

What do you think ?

About the desktop linux, Pijuice softwares looks to works fine when modem not plugged. When i add the modem :

pijuice tray software (showing battery in status bar) simply not appears.
ijuice menu (in preference menu) exists, but when click on it, to get settings GUI, nothing happens, simply not appears.

romaintaillandier1978 commented 3 years ago

Another question. I would like to try to use pijuice only to clean shut down. And get raspberry to start himself when power supply comes bak (as it would do with no pijuice.) From the moment pijuice is stacked, i can not get raspberry to boot himself on power, even if i plug power to its micro usb. How can i manage to perform this ?

romaintaillandier1978 commented 3 years ago

Today I have tried to disable pijuice.service. (removed from startup) now the boot sequence seems to run perfectly.

Is seems the service is conflicting with my own boot sequence, somehow.

mmilann commented 3 years ago

Do you see something like "communication error" displayed when you launch PiJuice gui when it gets stucked?

If you just want to power off as soon as power source is interrupted, than you need just to set "no power" event to SYS_FUNC_HALT_POW_OFF in System Events Tab. Also for powering on as soon as power is restored, enable "wakeup on charge" with threshold set to 0 in System Task Tab. Can you send screenshot of your settings, System Task and System Events Tabs?

Also what is firmware version?

romaintaillandier1978 commented 3 years ago

For the moment it is not possible to get a screen shot. Because for the first time since a week that i manage with pi juice, i get a stable reliable repeatable predictible behavior : Pi juice prevent rapsberry to start, and shut power at middle of boot sequence. for all configuration i can invent. It seems now not even possible to access pijuice configuration, for the moment.

I don't know how i will get out of that situation.

Please find config file and dump, of my best configuration. Working fine with linux desktop version, no modem. Harware version is 2.2.3 (printed on pijuice board). Firmware version is 1.4, PiJuice-V1.4_2020_01_17.elf.binary

{ "system_task": { "enabled": true, "ext_halt_power_off": { "enabled": false, "period": "" }, "wakeup_on_charge": { "enabled": true, "trigger_level": "15" }, "watchdog": { "period": "" }, "min_charge": { "threshold": "10", "enabled": true }, "min_bat_voltage": { "threshold": "" } }, "system_events": { "low_charge": { "enabled": true, "function": "SYS_FUNC_HALT_POW_OFF" } }, "user_functions": { "USER_FUNC1": "", "USER_FUNC2": "", "USER_FUNC3": "", "USER_FUNC4": "", "USER_FUNC5": "", "USER_FUNC6": "", "USER_FUNC7": "", "USER_FUNC8": "", "USER_FUNC9": "", "USER_FUNC10": "", "USER_FUNC11": "", "USER_FUNC12": "", "USER_FUNC13": "", "USER_FUNC14": "", "USER_FUNC15": "" } }

AND

{ "general": { "run_pin": 0, "i2c_addr": "14", "i2c_addr_rtc": "68", "eeprom_addr": 0, "eeprom_write_unprotected": false, "precedence": 1, "gpio_in_enabled": true, "usb_micro_current_limit": 1, "usb_micro_dpm": 0, "no_battery_turn_on": true, "power_reg_mode": 0, "charging_enabled": true }, "led": [ { "function": "CHARGE_STATUS", "color": [ 60, 60, 100 ] }, { "function": "USER_LED", "color": [ 0, 0, 0 ] } ], "button": { "SW1": { "PRESS": { "function": "NO_FUNC", "parameter": 0 }, "RELEASE": { "function": "NO_FUNC", "parameter": 0 }, "SINGLE_PRESS": { "function": "HARD_FUNC_POWER_ON", "parameter": 800 }, "DOUBLE_PRESS": { "function": "NO_FUNC", "parameter": 0 }, "LONG_PRESS1": { "function": "SYS_FUNC_HALT", "parameter": 10000 }, "LONG_PRESS2": { "function": "HARD_FUNC_POWER_OFF", "parameter": 20000 } }, "SW2": { "PRESS": { "function": "NO_FUNC", "parameter": 0 }, "RELEASE": { "function": "NO_FUNC", "parameter": 0 }, "SINGLE_PRESS": { "function": "USER_FUNC1", "parameter": 400 }, "DOUBLE_PRESS": { "function": "USER_FUNC2", "parameter": 600 }, "LONG_PRESS1": { "function": "NO_FUNC", "parameter": 0 }, "LONG_PRESS2": { "function": "NO_FUNC", "parameter": 0 } }, "SW3": { "PRESS": { "function": "USER_FUNC3", "parameter": 0 }, "RELEASE": { "function": "USER_FUNC4", "parameter": 0 }, "SINGLE_PRESS": { "function": "NO_FUNC", "parameter": 0 }, "DOUBLE_PRESS": { "function": "NO_FUNC", "parameter": 0 }, "LONG_PRESS1": { "function": "NO_FUNC", "parameter": 0 }, "LONG_PRESS2": { "function": "NO_FUNC", "parameter": 0 } } } }

As i said, using the pijuice service (base on pijuice_sys.py), prevent my device to boot up. So i disabled it. so i am quite sure that /var/lib/pijuice/pijuice_config.JSON is not used. I manually set SetWakeUpOnCharge(0) and run 'sudo halt'. That worked fine a dozen of times, successfully boot up when power come back. Repeatable, reliable ... until it definitely fails. I was forced to physically unplug pijuice. I have restore factory parameters everywhere.

This run me to the situation I described at the top of this answer.

I have written a script to replace pijuice_sys.py. Script is quite simple : if power is present, do nothing. if power is not present and battery > 10% do nothing SetWakeUpOnCharge(0) sudo halt.

I added the script to crontab every 3 minutes.

it have worked a few times. But it was necessary to manage with COMMUNICATION_ERROR, because it was fired sometimes. I would say 1/10 or 1/15.

And finally all was crashed and not able to restart until i unplug the pijuice, and run the raspberry stand alone, disable pijuice.service, re-plugged pijuice, restaore factory settings.

Thank you for your help.

mmilann commented 3 years ago

Strange thing in configuration is that "wakeup_on_charge" config is very close to "min_charge" threshold, difference only 5%. So in this situation when battery is overloaded by modem, it may work, however when start booting charge level cannot be so precise and counts down fast so it quickly gets below 10% during boot and than system task can trigger shutdown, especially if power is weak. Try setting "wakeup_on_charge" higher for example 25%.

Another try is that you disable "low_charge" and instead enable "no power event" to halt as soon as power interrupts.

romaintaillandier1978 commented 3 years ago

Thank you, remarks about precision of battery charge level, is quite interesting. But since i do not use pijuice service anymore, it do not explain what i get. Unforntunately, i could not test that approach.

this afternnon i get another weird behavior. Now battery is indicated to be 0% not present. (i have check and recheck that it is well inserted.)

mmilann commented 3 years ago

If you use custom process instead of system task than also use higher threshold, what I see that you are setting to 0: SetWakeUpOnCharge(0)

instead try setting significantly over 10%: SetWakeUpOnCharge(25)

romaintaillandier1978 commented 3 years ago

Hello

First, I have replaced the 5V 2A power by a 5V 4A one. I have to perform more tests, but all look more reliable. Thank you for this.

Second. To answer your question, I often get COMMUNICATION_ERROR. Is there a way to manage this ? something 80% of times, sometime 20%, generally looks more frequent when battery is low. Not predictable

Third I really don't know what could be the problem. But when the pijuice make my RPI boot after powered up (SetWakeUpOnCharge(25)), i get almost every time trouble at boot, that manifest with dhcpcd service time out. I never get that, when i use the hardware switch SW clic to boot. So i have the feeling that there is differences.

Anyway. After one week of work, i can't get something reliable with that modem AND the pijuice. So i must resign to remove pijuice from final hardware list.

Anyway your reactivity was apreciated.

Finally, as a remark. I am not 100% sure, so please take care with following. While trying to integrat pijuice, i have search with network, and dhcpcd services. The boot conflicts I encountered could be due to your service not compliant to my system (or may be not). I have no network until modem is fully started, and configured. Your service need network to start : "After=network.target" (https://github.com/PiSupply/PiJuice/blob/master/Software/Source/debian-base/pijuice.service#L3) I think this line make boot sequence to wait for DHCPCD service to achieve something, that could never happen, because the modem setup service is not started.

So according to this article https://www.raspberrypi.org/forums/viewtopic.php?t=221507 I have added the /etc/udev/rules.d/99-i2c.rules and updated your service with

Requires=dev-i2c\x2d1.device
After=dev-i2c\x2d1.device

May be i am wrong, but I think your service do not need network, so may be it's not the best way to wait on that condition. That's why i think that it could be an idea to dig, about waiting for i2c instead of network.

Hope that help.

mmilann commented 3 years ago

@romaintaillandier1978 thanks for the feedback. For now, regarding "COMMUNICATION_ERROR" I can just recommend new updated software/firmware(still in test phase), with improvements, that you can install from this package: https://github.com/PiSupply/PiJuice/blob/v1.7/Software/Install/pijuice-base_1.7_all.deb but before remove old version.

"After=network.target" really could be unnecessary, making pro-longed pijuice service start. @tvoverbeek do you think removing this and replacing with After=dev-i2c\x2d1.device may produce other issues?

tvoverbeek commented 3 years ago

Network.target does not mean the network is completely up (ip address assigned). Only that the network management has started. It does not wait for dhcpcd. Anyway it makes more sense to wait for i2c to be up. However this needs an additional file in /etc/udev/rules.d. See e.g. https://www.raspberrypi.org/forums/viewtopic.php?t=221507#p1565523 Will test this scenario and if OK add the change to the 1.7 branch.

mmilann commented 3 years ago

yes, certainly better to wait for i2c than just terminate service if possible.

tvoverbeek commented 3 years ago

Waiting for i2c-dev works with one problem.

When Pi is off and PiJuice is in low power state then when applying power to the Pi, the Pi boots but it takes a while before the PiJuice goes to normal run mode. When using the RTC (with EEPROM address on 0x50) the PiJuice often is too late to respond to the i2c-rtc probe on address 0x68 -> sudo hwclock -r does not work.

When applying power to the PiJuice in this situation it works fine since the PiJuice is in run mode before it wakes the Pi.

mmilann commented 3 years ago

@tvoverbeek yes pijuice detects that Rpi is booting when GPIO current goes over 50mA, but this detection is done every 4 seconds so there is mcu delay to exit standby nd become ready for I2C command. Also mcu exits standby if there is high to low change on SDA line, however because it needs time for I2C on mcu to initialise after standby exit it may miss very first command over I2C. For access over python it is not problem because it does retry, but rtc it declares non-response.

So one solution if possible to run service immediately after I2C module loaded but before rtc module and do status read so mcu exits standby.

Another solution to do GPIO interrupt on SDA line, high to low, before rtc module load. Not sure how this could be done, maybe possible to insert some GPIO code somewhere out of pijuice service early in boot process?

tvoverbeek commented 3 years ago

@mmilann Thanks for the explanation about how the Pijuice wakes up. Problem is the rtc module is loaded automatically by the device-tree fragment in the HAT eeprom (when eeprom address is 0x50) and then probes i2c address 0x68. If the probe fails, no rtc support in the OS. I think about how we can handle this. Anyway when using the 'wait for i2c-dev' way the rtc is probed about 0.2 seconds earlier than when using the current 'wait for network' at least on my Pi4B.

tvoverbeek commented 3 years ago

@mmilann I think I have a solution for the "rtc probe too early because PiJuice is not in run mode yet" problem. This situation can be recognized when the rtc_ds1307 module is loaded but the path /dev/rtc does not exist. In that case unload the rtc_ds1307 module and reload it with modprobe. Then /dev/rtc exists and sudo hwclock -r works. Tested this and it works. Now to integrate this in the pijuce service startup.

Note: this also covers the case where the user has not specified a rtc module (by changing the eeprom address to 0x52 and not specifying an rtc overlay). Then rtc_ds1307 will not be loaded and the initial test fails.

PiSupply / PiJuice

Hardware compatibility #643