vortigont / espem

Energy monitor with dashboard/metrics collector based on ESP32 controller and PeaceFair PZEM-004T/PZEM-004Tv30 Power Meter
GNU General Public License v3.0
61 stars 16 forks source link

How to Download the Code? #11

Closed Alaa-Aim closed 6 months ago

Alaa-Aim commented 2 years ago

Hi Sir I'm still new on this field, may I know how to download the code to the ESP32? which tool to use for download? is it working on PZEM V3.0?

vortigont commented 2 years ago

Hi @Alaa-Aim, the project could be build with PlatformIO, you can find more detailed description at the BUILD page. You also need to upload an FS image to the ESP, there is a shell script for this if you run linux, for windows you might need to install bash for windows, check some online tutorials for this. Yes, it works pretty fine with PZEM v3.0

Alaa-Aim commented 2 years ago

Hi @vortigont , thanks for your prompt response, I'm using Windows, I'll install VS code and platformIO as Add-one, for the FS image, do you mean the SPIFF files ?

vortigont commented 2 years ago

yeah, only this project uses newer LittleFS instead of SPIFFS. Current Arduino ESP32 framework can't create such an images by default, so you need to download an mklittlefs.exe tool from this page and drop it to the project's folder where the platformio.ini is and you'll be able to do pio run -t uploadfs then. Do not forget to populate ./data/ dir first by running resources/respack.sh script.

Alaa-Aim commented 2 years ago

@vortigont thank you for your response, I tried ton build this using VSCode with no success, I always get error in the included libraries, I've tried to install the required libraries, libraries installed successfully but have the red error line under the libraries, any advice ? thank you in advanced

vortigont commented 2 years ago

@Alaa-Aim you do not have to install any libs, platformio should do it during building. Might be something with your setup, maybe 'git' is missing in your system or 'python'? What is your OS - linux or windows? Can you show the error you have in the build terminal console? I can build you the firmware binaries, not a problem, but it would be tricky to upload it intro ESP32 without properly working Platformio environment.

Alaa-Aim commented 2 years ago

many thanks for your response, my OS is Windows, I've installed Arduino, VSCode then added PlatformIO IDE, but I still have the same errors, I installed fresh OS and install Arduino and VSCode only, still the same issue, terminal as below : image

Processing espem (platform: espressif32; board: wemos_d1_mini32; framework: arduino) --------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via-v, --verbose` option CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/wemos_d1_mini32.html PLATFORM: Espressif 32 (3.4.0) > WeMos D1 MINI ESP32 HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa) PACKAGES:

vortigont commented 2 years ago

here is your issue UserSideException: Please install Git client from https://git-scm.com/downloads:

Alaa-Aim commented 2 years ago

Dear @vortigont , already installed git , this is what I got:

image 'Processing espem (platform: espressif32; board: wemos_d1_mini32; framework: arduino) --------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via -v, --verbose option CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/wemos_d1_mini32.html PLATFORM: Espressif 32 (3.4.0) > WeMos D1 MINI ESP32 HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa) PACKAGES:

vortigont commented 2 years ago

ugh... subprocess.CalledProcessError: Command '['git', 'describe', '--abbrev=4', '--always', '--tags', '--long']' returned non-zero exit looks like that's a Windows issue... pretty ugly OS :) I'll fix that and let you know, need to find some windows hosts to test :))))

vortigont commented 2 years ago

Meanwhile you can workaround it, open 'platformio.ini' file and replace this line !python flags.py with -DGIT_REV="git_xxx" Mind the spaces indent!

Alaa-Aim commented 2 years ago

@vortigont many thanks for your replied and patient, really appreciate it, wait to hear from you, bear with me, I'm still noob in this field and still learning.

vortigont commented 2 years ago

@Alaa-Aim you are wecome! You may also face some issues with building FS image under windows. Actually never tested it, because it requires some unix tools to be available under windows, usually comes with git-bash installed. Anyway, here is an FS image file for you attached. Just drop it under .pio/build/espem spiffs.zip

Alaa-Aim commented 2 years ago

@vortigont I did replaced '!python flags.py' with '-DGIT_REV="git_2.34.1"', still have error, thanks for the SPIFFS

image

'Processing espem (platform: espressif32; board: wemos_d1_mini32; framework: arduino) --------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via -v, --verbose option CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/wemos_d1_mini32.html PLATFORM: Espressif 32 (3.4.0) > WeMos D1 MINI ESP32 HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa) PACKAGES:

Terminal will be reused by tasks, press any key to close it. '

vortigont commented 2 years ago

OK, here is the real revision string -DGIT_REV='"v2.0.1-10-gfba6"', try this one

vortigont commented 2 years ago

or maybe omit single quotes, just -DGIT_REV="v2.0.1-10-gfba6" not sure how windoze likes it :)

Alaa-Aim commented 2 years ago

or maybe omit single quotes, just -DGIT_REV="v2.0.1-10-gfba6" not sure how windose likes it :)

Windows like it with two quotes, compile is successful but when connect to it and navigate to 192.168.4.1 it shows "not found", I remembered to upload FS, when I ran platformio run -t uploadfs , this is what i got:

PS D:\ESP> platformio run -t uploadfs Can not remove temporary directory D:\ESP\.pio\build. Please remove it manually to avoid build issues Processing espem (platform: espressif32; board: wemos_d1_mini32; framework: arduino) --------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via -v, --verbose option CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/wemos_d1_mini32.html PLATFORM: Espressif 32 (3.4.0) > WeMos D1 MINI ESP32 HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa) PACKAGES:

Alaa-Aim commented 2 years ago

PS D:\ESP> platformio run -t upload Can not remove temporary directoryD:\ESP.pio\build. Please remove it manually to avoid build issues Processing espem (platform: espressif32; board: wemos_d1_mini32; framework: arduino) --------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via-v, --verbose` option CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/wemos_d1_mini32.html PLATFORM: Espressif 32 (3.4.0) > WeMos D1 MINI ESP32 HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa) PACKAGES:

Leaving... Hard resetting via RTS pin... ================================================= [SUCCESS] Took 27.99 seconds ================================================= Environment Status Duration


espem SUCCESS 00:00:27.991 ================================================= 1 succeeded in 00:00:27.991 ================================================= `

vortigont commented 2 years ago

Nice! :))) Check BUILD page, you need mklittlefs,exe binary to make FS iamges.

l but when connect to it and navigate to 192.168.4.1 it shows "not found"

you can also navigate to http://192.168.4.1/update and upload fs image I gave you from there

Alaa-Aim commented 2 years ago

Nice! :))) Check BUILD page, you need mklittlefs,exe binary to make FS iamges.

l but when connect to it and navigate to 192.168.4.1 it shows "not found"

you can also navigate to http://192.168.4.1/update and upload fs image I gave you from there

I navaigate to 192.168.4.1/update, and uploaded the SPIFFS you gave me, the user interface worked but I can't change the name of AP and I can't connect to any WIFI network, any easy way for me to compile my own SPIFFS? really appreciate you response and replies.

vortigont commented 2 years ago

No need, you can do all from WebUI. Internal AP name is hostname actually, can change under the 'device name' section. WiFI connection is set under WiFi Client section, need to fill both SSID and password, there is no scanning for available AP's around. Once connected, esp will shutdown internal AP and you can access it via http://hostname/ in your network.

Alaa-Aim commented 2 years ago

Actually I did all you mentioned, but when I enter the SSID & Password (entered manually as no scanning functionality) , ESP rest but nothing changed, still in AP mode and I can't change the host name nor connect to the WiFi , all above things I did it from webUI , but no success

Alaa-Aim commented 2 years ago

image

vortigont commented 2 years ago

Ugh... not sure. Pls, try to build your own FS image as described in DOCS. Or you can try to clear config from "System setup" page. Anyway you can try to connect your PZEM device right away and check it :))) Going offline for today, will be back tomorrow. Cheers!

Alaa-Aim commented 2 years ago

device right away and check it :)))

I managed to make my FS alaa85@DESKTOP-KSFF4NQ:/mnt/d/ESP/resources$ ./respack.sh Preparing resources for ESPEM FS image into ../data/ dir EmbUI resources requires updating Refreshing EmbUI css files/pics... ./respack.sh: 112: unzip: not found refreshing embui js files... alaa85@DESKTOP-KSFF4NQ:/mnt/d/ESP/resources$

this the files I had , image

still have the same error when I run platformio run -t uploadfs

"PS D:\ESP> platformio run -t uploadfs Can not remove temporary directory D:\ESP\.pio\build. Please remove it manually to avoid build issues Processing espem (platform: espressif32; board: wemos_d1_mini32; framework: arduino) --------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via -v, --verbose option CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/wemos_d1_mini32.html PLATFORM: Espressif 32 (3.4.0) > WeMos D1 MINI ESP32 HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa) PACKAGES:

Alaa-Aim commented 2 years ago

@vortigont I managed to upload the SPIFFS I made, the languge become Russian (I guess) and I still have the same issue, can't change the device name, can't connect any WiFi even though I've tried many WiFis with no luck

image

vortigont commented 2 years ago

@Alaa-Aim OK, I was able to run this under windoze :) I was even able to run without -DGIT_REV hack. I use windows 7, not sure if Win10 has some tricks, heard from other ppl that it's better to install python via Microsoft store or whatever it is called... nevermind. Actually you can just simply remove this line from platformio.ini, should be fine. I've made some small fixes in repo, you need to redownload it from scratch, rebuild firmware and upload it into MCU,. Hope this will fix WiFi issue, even might not need to rebuild FS image. Just checked with a brand new esp32 board - it works pretty fine :)

What I did is installed git for windows with that feature enabled image

than I was able build fw and run respack.sh script right from the VSCode terminal image

I managed to make my FS ./respack.sh: 112: unzip: not found refreshing embui js files... alaa85@DESKTOP-KSFF4NQ:/mnt/d/ESP/resources$

this the files I had ,

That is not right! You miss unzip, so you actually have nothing. the directory with data is not resources, it's just sources. The real data for LittleFS goes into ./data dir. You should have there at least index.html.gz and css/js dirs.

  • Building SPIFFS image from 'data' directory to .pio\build\espem\spiffs.bin '"./mklittlefs.exe"' is not recognized as an internal or external command,

you are still missing mklittlefs.exe. AGAIN, pls read BUILD.md and download binary mklittlefs.exe for windows from specified REPO and place it into the repo's dir.

the languge become Russian (I guess)

you can change it back to English from "settings" page - top drop down list.

Alaa-Aim commented 2 years ago

@vortigont , thanks for your effort and response, I downloaded the new respo, compiling was not success with the included platformIO.ini, i had to use the one we made yesterday, and compiling was successful, FS build was successful to after I placed the mklittlefs.exe in resources folder and I got index.htmland css/js dirs. alaa85@DESKTOP-KSFF4NQ:/mnt/d/ESP/resources$ ./respack.sh Preparing resources for ESPEM FS image into ../data/ dir EmbUI is up to date refreshing embui js files... alaa85@DESKTOP-KSFF4NQ:/mnt/d/ESP/resources$ the I used the two commands to upload the code and the FS platformio run -t upload platformio run -t uploadfs

everything was successful, but when I connect to the ESP for configuration, the image below is what I got, I used brand new MCU and I Still have this issue

image

vortigont commented 2 years ago

everything was successful, but when I connect to the ESP for configuration, the image below is what I got

this is still wrong, you are missing css/js files in the ./data/ dir, this means respack.sh script failed to download new files. And firmware source files are wrong - EmbUI version must be 2.6.1 with new sources. Try to delete project folder completely and download again from GitHub

Alaa-Aim commented 2 years ago

@vortigont I removed the whole folder and downloaded a new one, the FS is made with no errors, below screen shots for the files i had, image image image image

when i use the provided .ini file, I get error when compiling as shown below : "Processing espem (platform: espressif32; board: wemos_d1_mini32; framework: arduino) --------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via -v, --verbose option CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/wemos_d1_mini32.html PLATFORM: Espressif 32 (3.4.0) > WeMos D1 MINI ESP32 HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa) PACKAGES:

and when I use the file we modified yesterday it compiles with no errors.

and still get the same page as last comment.

please advise, thanks for your efforts and response

vortigont commented 2 years ago

below screen shots for the files i had,

somehow you miss css files, you need to check the output of respack.sh script, maybe it misses some tool. Anyway, here is the data dir content, just unpack it to yours ./data/ data.zip

and when I use the file we modified yesterday it compiles with no errors.

do not use yesterday's platformio.ini file. Use new one, just remove the line !python flags.py Your Windows is nasty :)

vortigont commented 2 years ago

Always clean .pio folder on new rebuild, it may contain previous versions of libs

Alaa-Aim commented 2 years ago

Always clean .pio folder on new rebuild, it may contain previous versions of libs

@vortigont may you teach me to clean. pio folder? many thanks

vortigont commented 2 years ago

just delete it :) it's under project's folder

Alaa-Aim commented 2 years ago

just delete it :) it's under project's folder

I did that, I deleted the whole folder and copy new respo. and I still have the same issue

vortigont commented 2 years ago

did you removed !python flags.py line from platformio.ini file? Does the build succeeded?

Alaa-Aim commented 2 years ago

@vortigont after many tries, it connected to the WiFi and I can access it from the WebUI, but I can't change the device name the web interface same not right, below screen shots

image image image

vortigont commented 2 years ago

web interface same not right

your FFS image misses .css files, same as before. Unpack the archive to your ./data/ dir and reflash FS.

Or out can access internal FS via FTP client and upload missing files directly to the MCU FTP creds are user: ftp, pass: ftp

Alaa-Aim commented 2 years ago

@vortigont Now it works fine after uploading the new ,/data folder, may I know how to fix my FS files? how to build files same as what you uploaded? below screen shots for the new webUI and for the files i had inside my resources folder, many thanks for your response and efforts

image image image image image

vortigont commented 2 years ago

may I know how to fix my FS files?

Not sure, you need to check the output of the respack.sh script, it might throw some error why it can't unpack css files. You can run it via git-bash shell terminal. Also it might be useful to run it with -f flag to force resources update. mklittlefs,exe must be placed to the project's root dir, right near the platformio.ini file Are you able to change the hostname now?

Alaa-Aim commented 2 years ago

may I know how to fix my FS files?

Not sure, you need to check the output of the respack.sh script, it might throw some error why it can't unpack css files. You can run it via git-bash shell terminal. Also it might be useful to run it with -f flag to force resources update. mklittlefs,exe must be placed to the project's root dir, right near the platformio.ini file Are you able to change the hostname now? @vortigont I placed the mklittlefs.exe in the root directory and when I run '. /respack.sh' no error appears, but when I cross checked the files, it seems less files generated.

host name, I still not able to change, I'll do what you mentioned and revert back to you.

I was using linux shell to run ./respack.sh , now I used git-bash and now all files generated correctly, upload it to the ESP32 and works fine, many thanks for your prompt responses and efforts.

P.S: I still can't change the host name, but it's not a problem now Edited : Now I'm able to change the host name

vortigont commented 2 years ago

now all files generated correctly, upload it to the ESP32 and works fine, many thanks for your prompt responses and efforts.

great! Finally :) u r welcome )

Alaa-Aim commented 2 years ago

Dear @vortigont , Merry Christmas , sorry for bothering you again, I wanted to modified you project a little, I wanted the ESP32 to trigger an action when the voltage goes below or above a threshold (i.e. turn on a LED), but couldn't find the WebUI getting the voltage value from, cloud you please point this out for me, so I can use to trigger the "action"

image

vortigont commented 2 years ago

@Alaa-Aim, Better open new issue next time, not to mix topics. UI update task is triggered in method ESPEM::wspublish() You can hack into there for testing only, but it's not the best place due to it does not run when no WebUI clients are present if (!embui.ws.count()

Or you can hijack to the call-back lambda of the sampler function, it runs every second, collecting sampled data regardless of the WebUI clients.

But much better way would be to create your own function like ESPEM::wspublish() and attach it as a separate scheduled task to run periodically https://github.com/vortigont/espem/blob/cf2daac20c2610e45a0a762c1e8ac5bfa5a87875/espem/espem.cpp#L59-L61

Aleks130699 commented 2 years ago

Good afternoon, I also had problems compiling on windows 10, but like deleting a line !python flags.py from platformio.ini, helped in compilation, there were no problems with fs compilation, connected pzem 004t v3 to esp8266, receives information, but not how I can't force to connect to wifi, while if wifi is without a password, then esp connects without problems.

vortigont commented 2 years ago

@Aleks130699 ok, first question - you are working with esp8266 or esp32? Current master branch does not support esp8266. Regarding WiFi - what is your issue? Are you able to access WebUI via internal AP and enter WiFi credentials for your WiFi-AP?

vortigont commented 2 years ago

like deleting a line !python flags.py from platformio.ini, helped in compilation

tnx for confirmation, I do not have win10 to test this. Could you show a console output or a screenshot with an error. I'm already thinking about completely disabling this script for Windows :)

Aleks130699 commented 2 years ago

I use esp8266, everything works through an internal access point, I can enter data for wifi, but esp does not connect to my wifi, and creates an access point again, but if you remove the security from my wifi, then esp works fine through my network

Aleks130699 commented 2 years ago

Все работает только не подключается к wifi.... image

vortigont commented 2 years ago

@Aleks130699 попробуйте собрать версию с дебагом, в build_flags нужно добавить

  -DEMBUI_DEBUG
  -DEMBUI_DEBUG_PORT=Serial

или -DEMBUI_DEBUG_PORT=Serial1 если первый порт занят вероятно это проблема совместимости 8266 и роутеров от асус, бага висит на гитхабе. Как вариант еще можене попробовать и/или добавить флаг -DPIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK22x_190313 это более старый сдк, но с ним 8266 нормально работает в режиме 802.11n

Aleks130699 commented 2 years ago

У меня роутер кеннетик, ну также пробовал и с телефона раздавать, тоже не захотел, а есть еще лампа с вашим проектом, она работает без проблем, ну там версия 2.4.5, совет сейчас попробую