Questions

[DrRob]

Hi,

This is the only way I could find to contact you - hope that's ok?

Two questions:

1. Are you using batteries to power your ESPLogger? I've also built a temperature/humidity/lux sensor node, using an ESP-12E, AM2302 and TSL2561, for monitoring our greenhouse (it's the closest I get to gardening...), and wondering how best to power it. The combined power supply range is quite narrow - 3.3v min for the AM2302 and 3.6v max for the ESP and the TSL2561.

Linear regulators waste too much power while the ESP is asleep, but the low quiescent power regulators I've found can't handle the burst of power required when transmitting on wifi.

So curious what you're using.

1. What's different in your customised ESP firmware?

Thanks, Rob.

[Aeprox]

Hey Rob

This way other people can join in the conversation/share the knowledge. Win-win if you ask me! :+1:

1. I'm currently struggling with the same issues. At the moment I'm simply using a single Li-Ion cell with a charger/protection board. This keeps the supply voltage between 4.2V and 3.4V which is fine for the DHT22. (Plus it gives me a nice micro-USB port to charge the sensor) I've put a 1N4004 diode between the TSL2561 and Vcc to drop its voltage down enough to be within the operating range (2.7-3.6V). The ESP module itself however is running straight from this voltage for the same reasons you mentioned.

With the cost of a module being so low, and many people on the internet reporting that they're using them without problems way beyond their specifications, I don't see it as a big priority at the moment to tweak this. I simply haven't got the time right now to thinker with it, so I'm just glad that it's working and telling me how the greenhouse is doing while I'm at work.

I am however planning on upgrading the battery to a 2S Li-ion configuration in the future, which would require a regulator. What kind of regulators have you tried? Would a high capacity capacitor help with the spikes?

You've mentioned putting the ESP to sleep. Did you use dsleep()? Did it work without the occasional crashes/zombie mode? Which nodeMCU build did you use?

1. I've forked their 20150406 release (the latest stable at that moment) and disabled certain modules I didn't need. (see Aeprox/nodemcu-firmware@3f1ada0bc69a22688f6be786c832057af0f9410f) This gave about 2kB of extra free heap-space at startup. If your setup works fine with the default build then there's really no need to use my build.

I'll be sure to add this information to the readme at some point in the future

greetings

Aeprox

[DrRob]

Hi Aeprox,

I destroyed my first ESP-12E by mistakenly connecting it to 10V, which resulted in blue smoke and a smell that lingered for several days. Consequently I'm a bit nervous about over-volting the ESP now...

I'm wondering about using a regulator with an enable pin connected to GPIO16-RST (assuming the polarity of that signal is right for the enable pin of the regulator), and then separately using some number of diodes to drop the battery voltage to somewhere lower than 3.3V, but enough to enable the ESP to deep sleep and then wake. E.g. 3V dribbles through the diode to keep the ESP alive while asleep, then as soon as it wakes the regulator is enabled and the supply jumps to 3.3V.

I tried running it from 3 rechargeable AAs (~3.8V according to my cheap/inaccurate voltmeter) through an IN4001, which results in ~3.3V while asleep, dropping to just below 3V when awake. It works, but the humidity readings seem to jump around a lot. Not sure how reliably it would wake up though (I haven't tested a lot yet). Less than 3V is well below the minimum for the temp/humidity sensor, and the datasheet recommends 5V for best operation.

I have had a few cases of it not waking, even when running at 3.3V on a hungry (5ma minimum) regulator, though I haven't tested much yet.

I tried an SPX3819M5-L-3-3 in an 5 pin SOT23 package, but this was waaay smaller than I'd realised, and I guess my shakey soldering killed it because I could only get 0.5V from it, and then the output pin snapped off while I was checking the connections. :-/

So currently I'm using an LD1117V33, which consumes 5ma when nothing is drawing current from the output.

I'm using nodemcu_integer_0.9.6-dev_20150406.

I notice there's a watchdog timer (tmr.wdclr()), but no clues in the docs about how to set it up to reset the ESP if it times out? But I suppose that's only useful while the ESP is awake?

Rob.

[DrRob]

My logger is now deployed to the greenhouse: https://thingspeak.com/channels/40838#publicview , attached to a 7000mAH SLA battery. I need to switch to a regulator with low quiescent current - for now I'm just testing with the one I have, and attach a solar panel to keep the battery topped up. I should be able to switch to a smaller battery then.

The two lux readings seem to have hit a max value of 4740. I haven't pulled in your updated version yet, or looked at the code to see if it adjusts the gain/integration time automatically.

One thing I forgot to mention before is that the first thing my code does on waking is set a timer for one minute to put the ESP back to sleep, in case something goes wrong and it doesn't complete the steps that normally lead to going back to sleep ASAP.

[Aeprox]

Nice to see you've got your setup going! Did you use the ESP's ADC for the voltage reading? I've been thinking about using solar as well. The Li-Ion charger/protection board I'm using (based on tp4056 IC) could easily be hooked up to a 5V solar panel.

Just a quick answer to your latest question: For use outside (with loads of (in)direct sunlight), I've found it best to use no gain at all. The values max out at around 4740 when it's using 13ms integration time, iirc. To get the full 16-bit range you'll need to use the 402ms integration time. Makes sense come to think of it: the sensor's ADC hasn't got the time to read all bits when using shorter integration time.

I've recently made some commits to the master branch that allow you to easily configure these settings, after I had strugled with them myselve.

I'll be back at the end of the week with a more meaty reply!

greets

Aeprox

[DrRob]

Yes - I used a simple voltage divider (820K and 56K) from 12V battery positive to ground, with the mid-point connected to ADC. Took longer to solder it than to code it. It's a very productive platform to develop with. Just need to get the power consumption down and ensure it's reliable.

[DrRob]

One other thing I've noticed - I switched the wi-fi connection from a guest network on a Virgin Media Superhub to a TP-Link router running OpenWRT (which is nearer the greenhouse), and now when the ESP wakes up it is still connected to wi-fi even after being asleep for 5 minutes (i.e. wifi.sta.getip() doesn't return nil). Not sure why that is, but it's good because reconnecting to the other one was taking 5-10 seconds.

[DrRob]

I wonder if it's bad to reset the wifi AP config often (i.e. does it rewrite the EEPROM every time, even if the details haven't changed, and is there a limited number of write cycles)? There doesn't seem to be any way to read the AP config...

[DrRob]

I updated to your version of the TSL2561 code to get the longer integration time, but now the light levels are hitting a new ceiling of 65535. Looks like I need to test for that and switch off the x16 gain if so. (I note that it's an analog gain, so I wonder how accurate the x16 factor is...)

[DrRob]

I switched to x1 gain, but it's still hitting the 65535 ceiling some of the time. (See http://www.noblesque.org.uk/greenhouse.html) The integration time needs to be reduced (from 402ms) when this happens, which should cause the resulting values to be 0.252 (for 101ms) or 0.034 (for 13.7ms) times smaller.

I'm puzzled about why I was seeing a max value of 4740 before - the datasheet says that with an integration time of 13.7ms, the max possible value is 5047 (and with 101ms it is 37177).

I could really do with having a way to update the scripts remotely. I'm wondering if the ESP could poll a web server for updated scripts (use an ETag and HTTP If-None-Match header), and if any are newer, save them to flash, compile them, and then reset. To be safe, the update needs to be atomic, so it'll need to keep two versions (I have an ESP-12E so it has plenty of flash).

I connected a solar panel to the 12v battery and it seems to be doing something because I can see the voltage rise a little bit during the day (I guess that means there must be at least a little charging current going in).

[DrRob]

Please see http://ideasandbox.blogspot.co.uk/2015/06/tsl2561-light-sensor-ranges.html - I'd be keen to get your feedback on it.

[Aeprox]

Hey Rob

Sorry for the delay! Since I last replied I've updated to the latest release of nodeMCU (20150627) and I haven't had any weird hangs/crashes when using deepsleep. I've also implemented a way of reading the supply voltage as you did, but I opted for the (fairly new) SDK function readVdd, which removes the need for a voltage divider altogether.

The module has been running with deepsleep enabled for over 2 weeks now and the voltage has only just now dropped below the ceiling of readVdd (which is around 4V). I'm using 2 18650 Li-ion cells in parallel with a protection board (this kind from a cheap chinese seller) that charges them to 4.2V and cuts off at 3.2, so I'm hoping to get about a month of runtime out of these old cells that came from an old broken laptop battery. An incredible increase when compared to the 2 to 3 days of runtime I had when the module didn't go into deepsleep in-between readings! I bet that if you'd plug a solar panel into the battery's protection board, this set-up could run almost year-round without the need for a top-up (though I'm afraid it won't survive harsh winters).

Your blog was an interesting read! One question though; how did you graph out the range of the different integration times? If I read the graph correctly, the longest integration time gives a higher resolution, but a smaller absolute range? I don't have a good environment or light source to test this myself at the moment.

Concerning your question about updating code over the air: I've seen implementations of telnet on the ESP. So in theory it would certainly be possible to access the lua interpreter, or even write new files, using telnet. I don't think anyone has written a tool that handles it from your side though (sending the commands over telnet instead of serial port), so that would be up to you. Would be interesting to see it work though, I'm sure plenty of people would like support for something like this in esplorer or luatool.

Thanks for the input!

Aeprox

[DrRob]

Hi,

I tried the readVdd function but found that it crashed the ESP - maybe that's fixed in a newer verison? I'm sticking with the voltage divider because I'm reading the battery voltage on the other side of the regulator. I have a solar panel attached to the battery now - you can see the effect when the sun is on it in the charts - here's the last 5 days:

http://api.thingspeak.com/channels/40838/charts/5?width=1000&height=1000&days=5&dynamic=true&title=Battery%20level%20(840%20%E2%89%85%2012V)

I guess the battery voltage also varies with temperature. It'd be better to monitor charging current as well. I wonder if I can get a voltage plus current sensor with i2c output?

I created the chart in a spreadsheet (see attached), from a table that I worked out by studying the datasheet.

I've reworked the TSL2561 reading code to try the four different combinations of gain (on or off) and integration time (13.7ms or 101ms) until it finds one that isn't out of range, to get the best possible resolution but stay within range. In bright sunshine it is still hitting the maximum value. I'll probably get around that by pointing the sensor at the greenhouse floor, so that it's less affected by the direction of the sun.

I showed it to the fishi-wifi people ( https://github.com/hamishcunningham/fishy-wifi/issues/6) and they said they'd accept a pull request - I have to figure out how to do that (I'm not so familiar with git...).

Regarding the OTA update - I did something like this with an mbed. Here's what I have in mind for the ESP:

1. On start up, do an HTTP request to read a software version from a URL, and compare it to a record of the installed version, stored in a local file.
2. If the current version is higher than the installed version:
   1. Delete the folder of previous version files, if any (not the currently installed version - the one before that)
   2. Read a list of the new version files (HTTP URLs) and checksums from another URL
   3. Read all of the files into a new directory, and check the checksums
   4. If all load successfully, update record of the current version and update init.lua to point to the new folder and reset
   5. If not, delete the downloaded files and carry on as normal (will try to download again next time)

It means there needs to be enough flash memory for two copies, but we can't overwrite the current version or any failure will kill it.

On 5 July 2015 at 14:38, Michael Lucas notifications@github.com wrote:

Hey Rob

Sorry for the delay! Since I last replied I've updated to the latest release of nodeMCU (20150627) and I haven't had any weird hangs/crashes when using deepsleep. I've also implemented a way of reading the supply voltage as you did, but I opted for the (fairly new) SDK function readVdd, which removes the need for a voltage divider altogether.

The module has been running with deepsleep enabled for over 2 weeks now and the voltage has only just now dropped below the ceiling of readVdd (which is around 4V). I'm using 2 18650 Li-ion cells in parallel with a protection board (this kind http://www.aliexpress.com/item/1Pcs-High-Quality-5V-18650-Lithium-Battery-Charging-Board-Micro-USB-1A-Charger-Module-Drop-Shipping/32290456683.html?spm=2114.32010308.4.200.lhdjXQ from a cheap chinese seller) that charges them to 4.2V and cuts off at 3.2, so I'm hoping to get about a month of runtime out of these old cells that came from an old broken laptop battery. An incredible increase when compared to the 2 to 3 days of runtime I had when the module didn't go into deepsleep in-between readings! I bet that if you'd plug a solar panel into the battery's protection board, this set-up could run almost year-round without the need for a top-up (though I'm afraid it won't survive harsh winters).

Your blog was an interesting read! One question though; how did you graph out the range of the different integration times? If I read the graph correctly, the longest integration time gives a higher resolution, but a smaller absolute range? I don't have a good environment or light source to test this myself at the moment.

Concerning your question about updating code over the air: I've seen implementations of telnet on the ESP. So in theory it would certainly be possible to access the lua interpreter, or even write new files, using telnet. I don't think anyone has written a tool that handles it from your side though (sending the commands over telnet instead of serial port), so that would be up to you. Would be interesting to see it work though, I'm sure plenty of people would like support for something like this in esplorer or luatool.

Thanks for the input!

Aeprox

— Reply to this email directly or view it on GitHub https://github.com/Aeprox/ESPLogger/issues/3#issuecomment-118621353.

[DrRob]

ps. How much current is draw from the battery by the Li-ion protection board when the ESP is in deep-sleep?

On 5 July 2015 at 14:38, Michael Lucas notifications@github.com wrote:

Hey Rob

Sorry for the delay! Since I last replied I've updated to the latest release of nodeMCU (20150627) and I haven't had any weird hangs/crashes when using deepsleep. I've also implemented a way of reading the supply voltage as you did, but I opted for the (fairly new) SDK function readVdd, which removes the need for a voltage divider altogether.

The module has been running with deepsleep enabled for over 2 weeks now and the voltage has only just now dropped below the ceiling of readVdd (which is around 4V). I'm using 2 18650 Li-ion cells in parallel with a protection board (this kind http://www.aliexpress.com/item/1Pcs-High-Quality-5V-18650-Lithium-Battery-Charging-Board-Micro-USB-1A-Charger-Module-Drop-Shipping/32290456683.html?spm=2114.32010308.4.200.lhdjXQ from a cheap chinese seller) that charges them to 4.2V and cuts off at 3.2, so I'm hoping to get about a month of runtime out of these old cells that came from an old broken laptop battery. An incredible increase when compared to the 2 to 3 days of runtime I had when the module didn't go into deepsleep in-between readings! I bet that if you'd plug a solar panel into the battery's protection board, this set-up could run almost year-round without the need for a top-up (though I'm afraid it won't survive harsh winters).

Your blog was an interesting read! One question though; how did you graph out the range of the different integration times? If I read the graph correctly, the longest integration time gives a higher resolution, but a smaller absolute range? I don't have a good environment or light source to test this myself at the moment.

Concerning your question about updating code over the air: I've seen implementations of telnet on the ESP. So in theory it would certainly be possible to access the lua interpreter, or even write new files, using telnet. I don't think anyone has written a tool that handles it from your side though (sending the commands over telnet instead of serial port), so that would be up to you. Would be interesting to see it work though, I'm sure plenty of people would like support for something like this in esplorer or luatool.

Thanks for the input!

Aeprox

— Reply to this email directly or view it on GitHub https://github.com/Aeprox/ESPLogger/issues/3#issuecomment-118621353.

[Aeprox]

Rob,

You're correct. In past version of nodeMCU, the module would crash when calling readVdd() if it was connected to an AP in station mode. They recently tweaked this slightly, and the function now tells the user that it can't be called with wifi enabled, instead of simply crashing. I worked around this limitation by disconnecting the wifi before going into deepsleep and only connecting after calling readVdd. I opted not to use a voltage divider because it will always leak a small amount of current, no matter how sophisticated. But yeah, in your situation it's a necessity I guess.

I have noticed that the voltage readings are a lot more stable during nighttime when temperatures are lower. However, when averaged out, the day-time readings still seem pretty consistent in my case. If you still have some spare pins on the module then I'd certainly go for that I2C current/voltage sensor. The more data, the more fun, right? :)

I've got my TSL2561 in a spot that's always shady as well (also prevents the temperature sensor being heated by the sun) The sensor is very sensitive to the direction it's pointing at according to the datasheet so indirect sunlight is the way to go.

Creating a pull request requires you to fork their repo, make changes to your fork, commit changes to github and make a pull request. Easier said then done ofcourse. Git is amazing but has a steep learning curve which can be very discouraging imho.

Interesting idea! Good example of a continuous integration system on a small scale/platform. I don't think flash size would be a big limiting factor. Considering how little RAM we've got to work with, it's unlikely your codebase would be big enough not to fit into flash twice. You could even do this on a file-per-file basis when you detect flash is getting low. A lot more riskier of course, but you already knew that!

I haven't measured the current draw during sleep yet. Haven't felt the need because the battery seems to be lasting longer than I anticipated. The TP4056's datasheet states it's 50µA average, 100µA max. The biggest current draw during sleep is the power LED (which you could just unsolder it if the need arrives) at about 8mA, according to the mighty google.

[DrRob]

I've seen mine go into zombie mode a couple of times in the last few days. It seems to run the battery down quicker when it does that. I'll update to the latest nodemcu and see if that helps (I'm still on 0.9.6-dev_20150406 currently).

Are these the steps required to keep the wifi off until needed?:

Call wifi.sta.autoconnect(0) to prevent wifi from connecting automatically on startup. I wonder if it's bad to call this every cycle, in case it rewrites a block of flash memory every time, even if the value hasn't changed? There doesn't appear to be a way to read the setting first, and set it only if necessary. But maybe the code that implements this method does that.

Before going into deep sleep, call wifi.sta.disconnect() and then wifi.sleeptype(wifi.MODEM_SLEEP) to power down the wifi (does that persist through deep sleep?).

After waking and doing whatever sensor-reading and calculations are required, call wifi.sleeptype(wifi. NONE_SLEEP) and then wifi.sta.connect(), wait until it's connected, send the data, then repeat the above before going back to deep sleep.

[DrRob]

Ooo, just found this: https://github.com/nodemcu/nodemcu-firmware/wiki/nodemcu_api_en#wifisetphymode

[DrRob]

https://github.com/nodemcu/nodemcu-firmware/wiki/nodemcu_api_en#nodedsleep

Seems like I also need to select option 4 to prevent the wifi from being powered up after deep sleep. Some testing required...

[Aeprox]

Hey rob

You're correct, the build I'm using (0.9.6-dev_20150627) seems to squash the zombie bug that's still in 20150406. My module has been up and running for about 3000 dsleep cycles without turning into an undead, brain-eating creature. The high power consumption is a known symptom of "zombie mode", probably because the module fails to boot to completion and is stuck in a power hungry mode. They've also added some (very useful) API functions with this latest build, although their documentation is still lagging behind a bit.

BTW, wifi.sleeptype(wifi.MODEM_SLEEP) doesn't actualy power down the wifi, it just puts wifi in an energy savings mode. This is the default mode of the ESP8266, so it does persist in a way, no need to set it every time. The other sleep mode, light sleep, consistently crashes my module and reboots.

I haven't played around with the dsleep options since their documentation is very lacking (even in the official ESP8266 SDK docs), so I simply do

• readVdd()
• wifi.sta.connect()
• read sensors & send to server
• wifi.sta.disconnect()
• node.dsleep(delay)

They do mention here https://github.com/nodemcu/nodemcu-firmware/wiki/nodemcu_api_en#parameters-1 that when not passing an option it uses the current configuration. What that means? I have no idea, but maybe that's why wifi.sleeptype persists through reboot? Some testing required indeed!

Also, you can pass the autoconnect option along with wifi.sta.config() function. I did it like this

     -- always make sure we're in station mode
    if(wifi.getmode() ~= wifi.STATION) then wifi.setmode(wifi.STATION) end

    -- check if current config != config in usersettings
    conf_ssid, conf_password = wifi.sta.getconfig()
    if((conf_ssid~=SSID) or (conf_password~=password)) then
        wifi.sta.config(SSID,password,0)
    end

which prevents any unwanted writes to flash

[Aeprox]

Hey rob

I'd just like to mention that I've ported a TSL2561 library to the nodeMCU firmware as a C module recently and made a pull request to the developers. So with some luck, we'll have no more need for a lua library. Pull request is available here if you'd like to give it a go: nodemcu/nodemcu-firmware/pull/611

[DrRob]

Interesting, thanks. I've got a growing backlog of updates to do for my Greenhouse sensor (too many distractions currently...)

On 29 August 2015 at 12:59, Michael Lucas notifications@github.com wrote:

Hey rob

I'd just like to mention that I've ported a TSL2561 library to the nodeMCU firmware as a C module recently and made a pull request to the developers. So with some luck, we'll have no more need for a lua library. Pull request is available here if you'd like to give it a go: nodemcu/nodemcu-firmware#611 https://github.com/nodemcu/nodemcu-firmware/pull/611

— Reply to this email directly or view it on GitHub https://github.com/Aeprox/ESPLogger/issues/3#issuecomment-135975338.

[DrRob]

On 18 July 2015 at 09:41, Michael Lucas notifications@github.com wrote:

You're correct, the build I'm using (0.9.6-dev_20150627) seems to squash the zombie bug that's still in 20150406.

I finally got around to updating mine (to nodemcu_integer_0.9.6-dev_20150704) and disappointingly it has got stuck in zombie mode again after a few days... :-/

Is yours totally cured of that now?

Rob.

[DrRob]

I forgot to mention that the dht_lib that I'd been using before wouldn't work after updating the firmware, so I switched to using the dht API that's built into the firmware.

Which seemed to work ok, but now I've noticed that the temperature keeps jumping up and down by 8.9 degrees. :-/

http://api.thingspeak.com/channels/40838/charts/1?width=640&height=480&average=0&days=1&dynamic=true&type=spline&title=Temperature%20(celsius)

[DrRob]

I've also just noticed that the temperature values are generally incrementing in steps of 1 whole degree, except when it jumps up or down by 8.9 degrees. It should have a resolution of 0.1 degrees. Hmmm. I think I may have made an error somewhere...

[Aeprox]

Strange behavior indeed..

Mine hasn't had the zombie bug since the update. Been running without problems for nearly a month now after a recharge of the batteries. Do all the connected pins have proper pull up/down resistors in place? I've used 2.2k, but anything between 1k and 10k should do the trick. NodeMCU devboard or olimex esp8266 boards have good schematics available

Did you solve your problem in the end?

[DrRob]

I don't have many resistors connected, maybe that's the problem. I'll compare with the NodeMCU dev board schematics - thanks.

On 5 October 2015 at 01:06, Michael Lucas notifications@github.com wrote:

Strange behavior indeed..

Mine hasn't had the zombie bug since the update. Been running without problems for nearly a month now after a recharge of the batteries. Do all the connected pins have proper pull up/down resistors in place? I've used 2.2k, but anything between 1k and 10k should do the trick. NodeMCU devboard or olimex esp8266 boards have good schematics available

Did you solve your problem in the end?

— Reply to this email directly or view it on GitHub https://github.com/Aeprox/ESPLogger/issues/3#issuecomment-145403864.

[DrRob]

Ok, a couple of extra pull-up resistors added, and a few values of existing resistors adjusted. Fingers crossed it can avoid zombie-mode now.

Some pads remain unconnected - I wonder if that's ok? (GPIO14, GPIO12, TXD, RXD, and the six extra flash chip pads of the ESP-12E board).