ESP8266 Client program

[TilBlechschmidt]

Hi, I've written an almost feature-complete program that you can upload to an ESP8266 to track it.

Just to take any doubts about the ESP actually being capable of tracking even though it has such a small antenna I can confirm that it works very well. I can track across the whole house in which room I'm currently in with almost no errors (might have to setup some more APs to get rid of the last bit of error) and around here are about 8-10 networks. Note that you'd have to re-learn every location for the ESP and cannot reuse the data from other devices like your phone (unfortunately) due to the difference in antenna size, geometry and gain.

You can configure every setting via a web UI from your smartphone or PC. It features tracking and learning as well as integration with my own smart-home project (basically it broadcasts it's position to a UDP multicast group). Have a look yourself: If anybody is interested in the tracking code that's available in the TrackingData.cpp file. It sends new tracking data at a frequency of about 0.5-0.25 Hz and enters a deep sleep mode for 60 seconds if the preconfigured WiFi network is not available and tracking is enabled.

Instructions on how to install and use it are available over at the FINDTracker repository.

It might happen that I've missed something in the install procedure or just missed a nasty bug so please feel free to open issues or PRs.

@schollz maybe include it into the docs section for the ESP8266 if it meets your standards :smile:

[Abulrock]

this is amazing i cant wait to try it. However; i'm very new to ESP and have no idea where to start. i'm just wondering if you can help me get started on how to install these files on the chip. for example do i use platformIO to upload it all

and just wondering which version of ESP8266 you are using?

thanks and my apology for the stupid question

[TilBlechschmidt]

I am using the ESP8266-12E on a NodeMCUv2 Development board. But it shouldn't matter what kind of ESP chip you have as long as it has ~100 KB of space on the Flash chip for the SPIFFS. Regarding the question, I'm not sure if you can flash the binary blobs from the release page onto an ESP8266 other than the 12E due to differing flash chip sizes (unsure on that though since I only have that one for testing). So the first question would be what kind of Chip you are using and what OS you are on. It would be the easiest if you follow the "Building on your own" section of the README.md inside the repo, which requires you to install PlatformIO, npm and gzip. I am still working on a way to simplify that so that you only have to download PlatformIO but I couldn't wrap my head around that just yet :smile:

It should be possible to reduce the installation procedure down to only installing PlatformIO though so I'll look into that the following days.

[Abulrock]

thanks for the feedback, i also have the NodeMCUv3, but i order a couple of other ESP8266 01 and 12E but still didnt arrive. i would assume you are running it off battery, just wondering how long its lasting with you? i managed to upload your files onto the NodeMCU, just using PlatformIO. i think it should be running fine, i do see the device on my WIFI and now i'm connected to it, but i cant seem to access http://findtracker/. if i disconnect from the ESP try to access it via its IP i get a message saying File Not Found, i'm i missing something?

[TilBlechschmidt]

I am planning on running it on a battery and measured the battery drain to be about 80mA whilst scanning on the NodeMCUv2 (@5V). That results in a solid 24h of tracking time at an interval of ~2s between position updates (estimation that does not take the battery degradation into account though). The standby drain (not in range of the configured WiFi) is approximately 4mA so that should last just over a week on the same battery and since you will not be home all day long you can expect about 2 days. That's all estimations though I still have to test it.

It might be that the hostname is not correctly resolved when you join the WiFi the ESP8266 created, I'll investigate that in a few hours. The "File Not Found" message might be due to the SPIFFS not being correctly recognized. As far as I know the flash layout gets set when you flash the binary blob onto the ESP and it might be that that did not happen when you flashed them or it is misaligned and therefore not recognized. I've got a possible fix for that though. As the website consists of only a single file (compiled from the actual sources) I plan on integrating it as a constant into the C++ code. The result would be that you only have to flash a single binary blob and the whole SPIFFS part would be obsolete. It reduces the headroom for possible errors to creep in unnoticed. I'll do that in the next couple hours so stay tuned.

[TilBlechschmidt]

Okay so I just investigated and since I cannot convince the linker to just flash my binary blob into the flash chip along with the sketch (because it attempts to load the file into the DRAM which obviously overflows at >100KB). That basically means that I have to use the SPIFFS to store the webpage.

As you can see in this image it attempts to put the data into the Flash.bin sector, which is too small. It would be great to insert it into the User Data block without the need for the SPIFFS since that's overkill for a single binary file.

If anybody knows how to flash directly to the EEPROM let me know

@Abulrock Now it means that you have to flash the SPIFFS on your own (the SPIFFS.bin file) but as you said it didn't work. I'd love if you could try to build the SPIFFS by yourself. To do so you should download the repository from this link and install PlatformIO from this link. Then open up a terminal inside the FINDTracker directory and run the following whilst your ESP is connected via USB:

SKIP_WEBPACK=true platformio run --target uploadfs

Note that on windows that would be

set SKIP_WEBPACK=true
platformio run --target uploadfs

[Abulrock]

thank you very much for all your help. i think we are getting there, after some trail and error i managed to get the webserver to work. (nothing with your files, i'm just new to this) but still if try access through http://findtracker/ it doesnt work but if i try via the ESP IP then i get a webpage. so i think the SPIFFS installed right. the issue i'm having now is that the Site is asking me to login with a username and pass, and there's a message in the background saying "please wait While....." did i miss a step where i was supposed to define the username and pass?

[TilBlechschmidt]

Nope, you didn't I just noticed that a few minutes ago and added the default password and username to the README. It's admin and supersecret and the message in the background notifies you that it is loading the configuration which requires the authentication.

[TilBlechschmidt]

There is an auth variable in the configuration which is supposed to define the AP and config password but there is no way to change that just yet.

[Abulrock]

Thanks :) webserver is running, the last thing is that i'm getting an error when trying to Learn. (Unable to connect to the tracking server), the one in the setting is "ml.internalpositioning.com" and i can reach it fine from my broswer.

[Abulrock]

i think its solved, right now i'm learning rooms. the server should be "ml2.internalpositioning.com" and that seemed to solve the problem. thanks for your support and i'll report back if i come across something, and feel free to let me know if you need testing

[TilBlechschmidt]

Ahh okay, that makes sense there. Since ml.internalpositioning.com is forcing HTTPS (which is good :smile: ) and the ESP attempts to make an HTTP requests it gets rejected. I'll change the defaults accordingly. By the way, I just noticed a bug that makes the ESP enter the sleep mode when it is not active and not connected (unintended) so I am curious on how you actually managed to load the page there :laughing:

Actually, how did you connect to the ESP? Did you change the default config in the defaults.cpp? Currently, I cannot manage to reach it when connected to the FINDTracker238109321 WiFi network...that's what's making me a bit perplex on how you managed to do it :laughing:

EDIT:
Okay, so it seems like my second ESP8266 or rather the flash chip is simply broken. It delivers round about 50 kilobytes of the page and then aborts every single time. Anyway I changed it so that the ESP now responds a lot quicker in configuration mode and is actually reachable via 10.10.10.1 :smile:. I'll finish up the readme and push that soon.

[Abulrock]

i think so, because when i was adding a port at the end it seemed to load then fail, until i changed the URL. so far it seems be running stable with me but then again i got it to work about an hour ago :). i was able to Learn two rooms and when i switch to tracking it stays in tracking (so far powered from USB) even after power off and back on. using findtracker/ is still not working with me so i edited default.hpp and added my WIFI SSID and Pass, then i got the IP address its on. I used that to access it from my phone or laptop :) i will continue to learn more rooms and switch to battery power later, i'll keep you posted how its running

[TilBlechschmidt]

Yeah, the configuration is persistent. If you enable tracking it does all kinds of things to preserve power like switching to a longer and more low power sleep period (ESP's MODEM_SLEEP) and after a reset it disables the AccessPoint. Additionally it goes into deep sleep mode when it cannot reach your preconfigured network so keep that in mind when carrying it out of the range of your home WiFi. In order for DeepSleep to work properly, you have to connect GPIO16 to the reset pin. If you don't it'll enter an everlasting sleep until you press the reset button :smile:

Okay. I now changed it so that you can actually access it using the AP it creates via a static IP. That is supposed to work now I'm unsure about my semi-bricked ESP though....since I do not have a third one for reference I cannot confirm it. You could try it out by disabling tracking, resetting your ESP and then connecting to the WiFi it created (in the realms of FINDTracker123456789).

In order to fix the findtracker/ URL not being available when using the hotspot I'd need to add a DNS server onto the ESP8266 but for some reason that renders it unusable as soon as I call dnsServer.start. No idea why but I guess for the initial configuration I'll just stick with the static URL for now.

[Abulrock]

i uploaded the new code and tried again, but i keep having issues trying to connect to the esp network, it asks for a password. (it could be my laptop) then some times it might let you log in without one. Findtracker/ still not working but you can access it via 10.10.10.1. once logged in the Learn doesnt work you cant reach the server for some reason. but disconnect from the ESP network and access it via the IP the WIFI router assign and Learn works fine. Another small bug i ran into is changing the Group name causes the browser to freeze but a refresh fixes it.

on the other hand Mr. Aliexpress delivered today some ESP8622 link so i thought i'll give it a try and upload your code into it and after some trail and error i got it working :) just ran a quick test, i was able to learn and track, it also picked up on the rooms that were learned with nodeMCU. However; i ran into some issues, because i didnt reassign the pins, i guess it went to deep sleep and was not able to track anymore. and the other issue is the battery consumption was very high, higher than the nodeMCU but i'm sure its because the code wasn't built for it and it could be fixed. (also have two E12 on the way, might be easier with these)

[TilBlechschmidt]

It is normal that learning doesn't work when you are connected to the ESP network since you cannot reach your configured tracking server (ml2.internalpositioning.com). Regarding the login sometimes asking for password only infrequently: I suppose you are referring to the configuration website login right? Your browser caches the login information and dumps it after a specific (unknown) period of time or when changing hosts (e.g. 10.10.10.1 is a different device than your own WiFi IP).

Regarding the config page being highly inaccessible when using the 10.10.10.1 (the bug I had with my other ESP) it is only appearing when the ESP runs in AP_STA mode (aka it runs as AccessPoint and WiFi client at the same time) which is totally weird. Might be due to my power supply delivering too low of a current (Laptops USB port).

Unfortunately, the ESP01 you linked does not provide you with access to GPIO16 directly. That means that you have to manually hook up the two terminals on the PCB if you want to utilize it. Have a look at this article, in the picture, there is a green arrow which indicates the pins you'd have to connect. I'll include an option in the defaults.hpp to disable deepSleep though if you don't want to tinker with your board ;)

What is your power consumption at? Mine hovers at round-about 80mA @ 4.8V whilst scanning. That roughly corresponds to 24h on a 2000mAh battery.

[TilBlechschmidt]

I added in a script to set the WiFi SSID and passphrase of your personal network to the repository and will eventually replace it with a very minimal website later on. This script is intended to be used when you are connected to the ESP8266 Hotspot and automatically reboots the chip after it set the values.

[Abulrock]

i uploaded the latest files and everything was running ok for a little while, then all of the sudden the wifi drops and the blue light blinks around 1sec. i saw you mentioned that its because its looking for the network but nothing changed while i was sitting it just cut off. anyways i tried to login into the ESP network but the page just loads but nothing happens. i tired to flash the files again but it didnt fix it, so i had to revert back to and older version and its working again. similar thing happened yesterday but i thought i did something wrong. not sure why this happens, maybe a defect in my nodeMCU.

thanks for the tip on the ESP 01, i tried to hack it like it showed in the pictures but i dont think it worked. it's pulling around 180mA, which is way to high the nodeMCU pulls same as you around 75 -80 mA.

[TilBlechschmidt]

Regarding the LED is it blinking in an interval of 1s aka 500ms off and 500ms on?

I experienced issues with the page loads via the ESP network as well and haven't found a solution for that yet. It seems to be related to some instability of the network since the connection between the two just drops when I start transmitting large amounts of data (the webpage). Eventually, I will just replace the somewhat complex and feature complete config page with a simple "configure your WiFi" page with the sole purpose of getting the ESP connected to your network. The learning has to be done after that step anyways so I guess that's the best solution.

Did you make sure the ESP8266 is in range of your WiFi network (sometimes it just freaks out and the connection drops frequently a few rooms away, other days it works fine for me....bit weird)? Also, can you hook your ESP up to your computer and run

pio -f -c atom device monitor --port /dev/ttyUSB0 --baud 115200

you might need to change /dev/ttyUSB0 to your appropriate port

[TilBlechschmidt]

btw I just pushed some commits that should stabilize the connection to the configuration AP and replaced the configuration page with a basic config page for the WiFi settings only (when those are set and the ESP is connected to your WiFi you get the full blown config).

And I had some problems with my NodeMCU not connecting to my home network as well. Seems weird. No idea why though. Might be because of the small antenna but since other people are getting insane ranges with it I doubt that's the root cause.

Regarding your high power usage, have a look at this post. It describes nominal power draw of just the ESP8266 chip. It somewhat corresponds with my results (and yours from the NodeMCU) if we add some power for the UART chip on top.

When measuring the ESP01 did you take the different voltage into account (or did you measure in-front of the voltage regulator?)? Since 180mA @ 3.3V correspond to 0.6W which would be around 120mA at 5V. Still, it should be way lower since the NodeMCU contains a UART and voltage regulator drawing additional power which the ESP01 board you linked does not have.
Did you reset the ESP once after activating tracking? If you don't it'll keep the AP open which drains a bunch more.

[Abulrock]

i figured out the issue with power consumption, first i was using a step up to 3.3v with two AA batteries and since ESP uses a lot of current the step up needed more power. when measuring from the battery directly the mA was very high but when measuring from the 3.3 step up OUT the measure was around 75 to 80mA, which i think is normal.

i built a very small tracker with ESP12F (which i believe is suppose to have a better antenna than 12E), and 3.3v voltage regulator and two 250mah Lithium, the result is very small device with what i think is better tracking than nodeMCU, and its drawing 70 to 75mA . i also flashed it with the latest files, and it seems to work better, thanks for that. just one issue is that for time to time the ESP network (Findtacker 1234567) is asking for a password (not the browser but the actual wifi), not sure why, even when i try from my phone. then it just works with any pass or the security drops off and you can login to enter the router wifi pass.

i have a small question, i think its silly but just wondering how do you track a room?, meaning are you leaving the device on a table for 10 mins (for example) or is it in your pocket while you walk around the room for a bit? i didnt see anything mention on best way to track.

[TilBlechschmidt]

Good to hear you found the power issue. The ESP Network is "supposed" to ask for the same password the config requires. I'll look into it and try to figure out what could cause that. I updated the code so that it uses a default password if the config is empty. Check if that fixes it (can't test it since I'm unable to reproduce the issue). You have to recompile though, I didn't build the binaries.

Nice to hear about your little tracker. How much battery life are you getting out of those two 250mAh cells?

I am tracking by just tossing it onto a table and letting it sit there for a few minutes (I took like 20-30 "fingerprints" as reported by the UI which actually equals ~4x the amount in actual WiFi scans). Generally, that is sufficient but especially with rooms in close proximity (like two rooms that are below two other learned rooms that are side by side) it requires some alternations in the learning positions. I might add a "how to learn" button.

[Abulrock]

thanks for the tip on how to track best. i deleted my previous tracking and learned them again by just keeping the tacker on the table or in my pocket, i think i'm getting better results now.

i timed it twice and the little tracker is lasting around 6 to 6.5 hours, but i still have the blue led, and its blinking every 2 seconds so i think if i just break it, i will get maybe another hour. its fully charged (which is around 420mah) in one hour so its not bad for my use for now.

another question and maybe a request, i'm just wondering how you are pulling location? for example just a script on a loop? if its not too much work for you, would it be possible to add HTTP GET on the website, so when the location is change the url is triggered? like assigning a url to trigger to each location on the tracking page.

[TilBlechschmidt]

At the moment the ESP8266 is sending a UDP multicast message to the group 224.0.0.1 at port 1337. The message is a simple JSON string taking this form:

{
    "action": "state",
    "payload": {
        "user": "yourUsername",
        "location": "currentLocation"
    }
}

In case you do not have access to UDP multicast networks it shouldn't be too difficult to add an HTTP callback. Just let me know whether or not the above suffices.

I am in the process of building a small tracker based on a 2000mAh, the barebones 12-E, an LDO and an induction charging coil and will be testing with that. I'm hoping for a device that you can simply throw into your pocket and forget about. Unfortunately, batteries are always a big problem (literally) since the battery will be bigger than all PCB's added together (charging circuit, induction coil, voltage regulator and ESP) but I suppose it would be nice having ~24h "home"-time with a single charge.

[Abulrock]

Sorry for the late response it's been a busy couple weeks.

I don't have access to a UDP multicast networks, so it's not too much work for you, it would be great to add a http request call, I think it will help to reduce the pull rate for tracking location and easier I guess to implement in home automation. (I'll be able to send http request to domoticz to turn on the lights for example)

How's your setup going? I have a couple wireless chargers around and I was thinking of the same thing actually.

[TilBlechschmidt]

I'll work on it the coming days. I got my 3000 mAh battery ready to go (was searching for a 2 Ah one but couldn't get one for a reasonable price and chose the marginally bigger 3 Ah one) including a charging board, voltage regulator and ESP. It all fits into a ~23mm tube that's about 9cm long. I might have bricked my ESP though since I cannot flash my sketch nor get any output from the bootloader :cry: . I checked the wiring against one of my development boards and there it seems to be working. I'm still hoping it has to do with the reset method not working as intended but I might be off ordering a bunch from china to experiment with.

Just for reference, here are the parts I used:

• Samsung ICR30F 3 Ah battery cell
• MCP1700-3302E voltage regulator
• TP4056 LiPo charging board

The cylindrical battery imposes the need for a circular charging coil though.

[Abulrock]

Thanks for the feedback, looking forward to the new version. I think (MCP1700-3302E voltage regulator) might not be delivering enough current when voltage drops a bit. I had a similar issue the ESP would work for a bit then after some time it starts loosing connection​ and restarting. So I changed the regulator I was using to this one (DD0503MA Ultra mini DC 3.7V 4.5V 5V to 3.3V DC DC Step Down) which outputs 500ma. This combined with the esp12 is using around 60ma which is the best I've seen. Hope this helps

[scottdalton]

I'm new to trying out FIND - just set it up this afternoon. However, I have a lot of ESP8266 based devices around the house and I'm wondering if anyone has approached using esp8266 based units for find-lf servers (for the track-all-the-things without an app approach) instead of using raspberry pi's?

I have a ton of already-in-use sonoff and esp8266 devices doing random stuff that if they could also perform dual duty here would be great. I don't actually have any raspberry pi's around, and I haven't dug into this at all yet. Just trying to get a feel for if any work here has already been done.

[hulkco]

Aewsome job @TheMegaTB . I am pleasantly surprised, as the support for ESP8266 has evolved, since I made the request on June 19, 2016.