search

dlmcpaul / EnphaseCollector

Enphase Solar Metrics Collector
Mozilla Public License 2.0
69 stars 19 forks source link

Battery data not in database #55

Open drache42 opened 1 year ago

drache42 commented 1 year ago

The codebase has code to get the battery info, for instance: https://github.com/dlmcpaul/EnphaseCollector/blob/master/src/main/java/com/hz/models/envoy/json/Production.java#L42

However, none of it shows up in the database.

Is this an incomplete feature or a bug?

dlmcpaul commented 1 year ago

Incomplete. I am not even sure if the data collected is correct as I have no battery to test with.

drache42 commented 1 year ago

I have a battery. I could test if you'd like. I have everything compiling now, so it should be fairly simple to run, test, collect data.

If you tell me what you need, I can carve out time to do it.

On Sat, Apr 22, 2023, 16:05 dlmcpaul @.***> wrote:

Incomplete. I am not even sure if the data collected is correct as I have no battery to test with.

— Reply to this email directly, view it on GitHub https://github.com/dlmcpaul/EnphaseCollector/issues/55#issuecomment-1518892970, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD2PAI5U3YSH3A5OKU3STZDXCRP3RANCNFSM6AAAAAAXH4E3BQ . You are receiving this because you authored the thread.Message ID: @.***>

dlmcpaul commented 1 year ago

Ok I committed some code to dump the battery info to logs. Please run and let me know what sort of data we are getting so we can decide what to write to the database.

Be good to see data for

drache42 commented 1 year ago

Where are the logs? Console logs aren't showing battery info.

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::               (v2.6.13)
2023-04-23 07:02:21.961  INFO 1 --- [           main] com.hz.EnphaseCollectorApplication       : Starting EnphaseCollectorApplication using Java 17.0.6 on enphase with PID 1 (/app started by root in /)
2023-04-23 07:02:21.962  INFO 1 --- [           main] com.hz.EnphaseCollectorApplication       : No active profile set, falling back to 1 default profile: "default"
2023-04-23 07:02:22.562  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2023-04-23 07:02:22.611  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 42 ms. Found 5 JPA repository interfaces.
2023-04-23 07:02:23.098  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2023-04-23 07:02:23.109  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2023-04-23 07:02:23.109  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.68]
2023-04-23 07:02:23.205  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/solar]  : Initializing Spring embedded WebApplicationContext
2023-04-23 07:02:23.205  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1208 ms
2023-04-23 07:02:23.493  INFO 1 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-04-23 07:02:23.542  INFO 1 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version 5.6.12.Final
2023-04-23 07:02:23.550  INFO 1 --- [           main] org.hibernate.cfg.Environment            : HHH000205: Loaded properties from resource hibernate.properties: {hibernate.bytecode.use_reflection_optimizer=false, hibernate.bytecode.provider=none}
2023-04-23 07:02:23.601  INFO 1 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2023-04-23 07:02:23.696  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2023-04-23 07:02:23.828  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2023-04-23 07:02:23.851  INFO 1 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2023-04-23 07:02:24.264  INFO 1 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-04-23 07:02:24.270  INFO 1 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-04-23 07:02:24.322  INFO 1 --- [           main] c.h.c.EnphaseSystemInfoConfig            : Reading system information from Envoy controller endpoint http://192.168.107.15/info.xml
2023-04-23 07:02:24.636  INFO 1 --- [           main] c.hz.components.ReleaseInfoContributor   : Application Release version is 0.26
2023-04-23 07:02:25.257  INFO 1 --- [           main] o.s.b.a.w.s.WelcomePageHandlerMapping    : Adding welcome page template: index
2023-04-23 07:02:26.120  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 6 endpoint(s) beneath base path '/actuator'
2023-04-23 07:02:26.175  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path '/solar'
2023-04-23 07:02:26.194  INFO 1 --- [           main] com.hz.EnphaseCollectorApplication       : Started EnphaseCollectorApplication in 4.493 seconds (JVM running for 4.8)
2023-04-23 07:02:26.405  INFO 1 --- [           main] com.hz.services.LocalDBService           : Summarising Event table prior to 2023-04-23T00:00
2023-04-23 07:06:04.291  INFO 1 --- [nio-8080-exec-3] o.a.c.c.C.[Tomcat].[localhost].[/solar]  : Initializing Spring DispatcherServlet 'dispatcherServlet'
2023-04-23 07:06:04.291  INFO 1 --- [nio-8080-exec-3] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2023-04-23 07:06:04.293  INFO 1 --- [nio-8080-exec-3] o.s.web.servlet.DispatcherServlet        : Completed initialization in 2 ms
dlmcpaul commented 1 year ago

You look to be running release 0.26 instead of from the current code base. How are you starting the application?

drache42 commented 1 year ago

Oh I was running the Docker-builder file, not Docker-build

After removing the bash code in the dockerfile After adding # syntax=docker/dockerfile:1.4 to Dockerfile-build (because of https://github.com/dlmcpaul/EnphaseCollector/issues/58) I get the following output, which is now running the correct version 2023-04-23T07:52:40.008-07:00 INFO 1 --- [ XNIO-1 task-3] c.hz.components.ReleaseInfoContributor : Application Release version is unreleased

Still no battery logs though.

2023-04-23T08:01:08.841-07:00  INFO 1 --- [           main] com.hz.EnphaseCollectorApplication       : Starting EnphaseCollectorApplication using Java 17.0.6 with PID 1 (/app started by root in /)
2023-04-23T08:01:08.845-07:00  INFO 1 --- [           main] com.hz.EnphaseCollectorApplication       : No active profile set, falling back to 1 default profile: "default"
2023-04-23T08:01:09.197-07:00  INFO 1 --- [           main] o.s.c.i.support.PropertySourceProcessor  : Properties location [classpath:/release.properties] not resolvable: class path resource [release.properties] cannot be opened because it does not exist
2023-04-23T08:01:09.792-07:00  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2023-04-23T08:01:09.872-07:00  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 72 ms. Found 5 JPA repository interfaces.
2023-04-23T08:01:10.685-07:00  WARN 1 --- [           main] io.undertow.websockets.jsr               : UT026010: Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be used
2023-04-23T08:01:10.707-07:00  INFO 1 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2023-04-23T08:01:10.709-07:00  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1643 ms
2023-04-23T08:01:11.002-07:00  INFO 1 --- [           main] o.f.c.internal.license.VersionPrinter    : Flyway Community Edition 9.5.1 by Redgate
2023-04-23T08:01:11.002-07:00  INFO 1 --- [           main] o.f.c.internal.license.VersionPrinter    : See what's new here: https://flywaydb.org/documentation/learnmore/releaseNotes#9.5.1
2023-04-23T08:01:11.002-07:00  INFO 1 --- [           main] o.f.c.internal.license.VersionPrinter    : 
2023-04-23T08:01:11.013-07:00  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2023-04-23T08:01:11.225-07:00  INFO 1 --- [           main] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection conn0: url=jdbc:h2:/internal_db/solar_stats_db_v2 user=SA
2023-04-23T08:01:11.227-07:00  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2023-04-23T08:01:11.240-07:00  INFO 1 --- [           main] o.f.c.i.database.base.BaseDatabaseType   : Database: jdbc:h2:/internal_db/solar_stats_db_v2 (H2 2.1)
2023-04-23T08:01:11.301-07:00  INFO 1 --- [           main] o.f.core.internal.command.DbValidate     : Successfully validated 2 migrations (execution time 00:00.012s)
2023-04-23T08:01:11.308-07:00  INFO 1 --- [           main] o.f.c.i.s.JdbcTableSchemaHistory         : Creating Schema History table "PUBLIC"."flyway_schema_history" ...
2023-04-23T08:01:11.335-07:00  INFO 1 --- [           main] o.f.core.internal.command.DbMigrate      : Current version of schema "PUBLIC": << Empty Schema >>
2023-04-23T08:01:11.345-07:00  INFO 1 --- [           main] o.f.core.internal.command.DbMigrate      : Migrating schema "PUBLIC" to version "1 - InitSchema"
2023-04-23T08:01:11.369-07:00  INFO 1 --- [           main] o.f.core.internal.command.DbMigrate      : Migrating schema "PUBLIC" to version "2 - MigrateH2DatabaseToV2"
2023-04-23T08:01:11.374-07:00  INFO 1 --- [           main] o.f.core.internal.command.DbMigrate      : Successfully applied 2 migrations to schema "PUBLIC", now at version v2 (execution time 00:00.042s)
2023-04-23T08:01:11.451-07:00  INFO 1 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-04-23T08:01:11.501-07:00  INFO 1 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version 6.1.7.Final
2023-04-23T08:01:11.503-07:00  INFO 1 --- [           main] org.hibernate.cfg.Environment            : HHH000205: Loaded properties from resource hibernate.properties: {hibernate.type.preferred_instant_jdbc_type=TIMESTAMP, hibernate.bytecode.use_reflection_optimizer=false, hibernate.id.db_structure_naming_strategy=single, hibernate.bytecode.provider=none, hibernate.type.preferred_duration_jdbc_type=BIGINT}
2023-04-23T08:01:11.673-07:00  INFO 1 --- [           main] SQL dialect                              : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2023-04-23T08:01:11.703-07:00  WARN 1 --- [           main] org.hibernate.orm.incubating             : HHH90006001: Encountered incubating setting [hibernate.type.preferred_duration_jdbc_type].  See javadoc on corresponding `org.hibernate.cfg.AvailableSettings` constant for details.
2023-04-23T08:01:11.706-07:00  WARN 1 --- [           main] org.hibernate.orm.incubating             : HHH90006001: Encountered incubating setting [hibernate.type.preferred_instant_jdbc_type].  See javadoc on corresponding `org.hibernate.cfg.AvailableSettings` constant for details.
2023-04-23T08:01:11.867-07:00  WARN 1 --- [           main] org.hibernate.orm.incubating             : HHH90006001: Encountered incubating setting [hibernate.id.db_structure_naming_strategy].  See javadoc on corresponding `org.hibernate.cfg.AvailableSettings` constant for details.
2023-04-23T08:01:11.873-07:00  WARN 1 --- [           main] org.hibernate.orm.incubating             : HHH90006001: Encountered incubating setting [hibernate.id.db_structure_naming_strategy].  See javadoc on corresponding `org.hibernate.cfg.AvailableSettings` constant for details.
2023-04-23T08:01:11.885-07:00  WARN 1 --- [           main] org.hibernate.orm.incubating             : HHH90006001: Encountered incubating setting [hibernate.type.preferred_duration_jdbc_type].  See javadoc on corresponding `org.hibernate.cfg.AvailableSettings` constant for details.
2023-04-23T08:01:11.885-07:00  WARN 1 --- [           main] org.hibernate.orm.incubating             : HHH90006001: Encountered incubating setting [hibernate.type.preferred_instant_jdbc_type].  See javadoc on corresponding `org.hibernate.cfg.AvailableSettings` constant for details.
2023-04-23T08:01:12.303-07:00  INFO 1 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2023-04-23T08:01:12.309-07:00  INFO 1 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-04-23T08:01:12.878-07:00  INFO 1 --- [           main] c.h.c.EnphaseSystemInfoConfig            : Reading system information from Envoy controller endpoint http://192.168.107.15/info.xml
2023-04-23T08:01:14.447-07:00  INFO 1 --- [           main] com.hz.utils.MetricCalculatorStandard    : Standard Consumption Metric Calculator Activated
2023-04-23T08:01:14.513-07:00  INFO 1 --- [           main] io.undertow                              : starting server: Undertow - 2.3.4.Final
2023-04-23T08:01:14.517-07:00  INFO 1 --- [           main] org.xnio                                 : XNIO version 3.8.8.Final
2023-04-23T08:01:14.522-07:00  INFO 1 --- [           main] org.xnio.nio                             : XNIO NIO Implementation Version 3.8.8.Final
2023-04-23T08:01:14.567-07:00  INFO 1 --- [           main] org.jboss.threads                        : JBoss Threads version 3.5.0.Final
2023-04-23T08:01:14.596-07:00  INFO 1 --- [           main] o.s.b.w.e.undertow.UndertowWebServer     : Undertow started on port(s) 8080 (http) with context path '/solar'
2023-04-23T08:01:14.632-07:00  INFO 1 --- [           main] com.hz.EnphaseCollectorApplication       : Started EnphaseCollectorApplication in 6.111 seconds (process running for 6.479)
2023-04-23T08:01:14.763-07:00  INFO 1 --- [           main] com.hz.services.LocalDBService           : Summarising Event table prior to 2023-04-23T00:00
2023-04-23T08:01:27.453-07:00  INFO 1 --- [  XNIO-1 task-3] io.undertow.servlet                      : Initializing Spring DispatcherServlet 'dispatcherServlet'
2023-04-23T08:01:27.453-07:00  INFO 1 --- [  XNIO-1 task-3] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2023-04-23T08:01:27.459-07:00  INFO 1 --- [  XNIO-1 task-3] o.s.b.a.w.s.WelcomePageHandlerMapping    : Adding welcome page template: index
2023-04-23T08:01:27.992-07:00  INFO 1 --- [  XNIO-1 task-3] c.hz.components.ReleaseInfoContributor   : Application Release version is unreleased
2023-04-23T08:01:28.034-07:00  INFO 1 --- [  XNIO-1 task-3] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 6 endpoint(s) beneath base path '/actuator'
2023-04-23T08:01:28.155-07:00  INFO 1 --- [  XNIO-1 task-3] o.s.web.servlet.DispatcherServlet        : Completed initialization in 702 ms
dlmcpaul commented 1 year ago

I made the code changes to this file https://github.com/dlmcpaul/EnphaseCollector/blob/master/src/main/java/com/hz/utils/MetricCalculatorStandard.java and just log out what is in the 2 battery lists.

My first thought is that neither array is being populated.

Are your batteries Enphase or another brand?

drache42 commented 1 year ago

Enphase batteries.

Is there a curl command I can send to my gateway to get a list of batteries. I've been looking for some REST API docs for it, but no luck so far.

I also haven't found the api endpoints in your code.

On Sun, Apr 23, 2023, 08:13 dlmcpaul @.***> wrote:

I made the code changes to this file https://github.com/dlmcpaul/EnphaseCollector/blob/master/src/main/java/com/hz/utils/MetricCalculatorStandard.java and just log out what is in the 2 battery lists.

My first thought is that neither array is being populated.

Are your batteries Enphase or another brand?

— Reply to this email directly, view it on GitHub https://github.com/dlmcpaul/EnphaseCollector/issues/55#issuecomment-1519090111, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD2PAI4J373U3GEM5U7RBEDXCVBIFANCNFSM6AAAAAAXH4E3BQ . You are receiving this because you authored the thread.Message ID: @.***>

drache42 commented 1 year ago

The api/v1/production/inverters is not returning any devices with devType of 11 https://github.com/dlmcpaul/EnphaseCollector/blob/master/src/main/java/com/hz/models/envoy/json/InvertersType.java#L19

[
  {
    "serialNumber": "202302042912",
    "lastReportDate": 1682263705,
    "devType": 1,
    "lastReportWatts": 21,
    "maxReportWatts": 365
  },
  {
    "serialNumber": "202302044880",
    "lastReportDate": 1682263706,
    "devType": 1,
    "lastReportWatts": 20,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302046469",
    "lastReportDate": 1682263825,
    "devType": 1,
    "lastReportWatts": 21,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302047576",
    "lastReportDate": 1682263886,
    "devType": 1,
    "lastReportWatts": 17,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302042190",
    "lastReportDate": 1682263726,
    "devType": 1,
    "lastReportWatts": 21,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302045301",
    "lastReportDate": 1682263727,
    "devType": 1,
    "lastReportWatts": 22,
    "maxReportWatts": 365
  },
  {
    "serialNumber": "202302044116",
    "lastReportDate": 1682263826,
    "devType": 1,
    "lastReportWatts": 21,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302047204",
    "lastReportDate": 1682263887,
    "devType": 1,
    "lastReportWatts": 18,
    "maxReportWatts": 369
  },
  {
    "serialNumber": "202302045316",
    "lastReportDate": 1682263730,
    "devType": 1,
    "lastReportWatts": 22,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302043614",
    "lastReportDate": 1682263827,
    "devType": 1,
    "lastReportWatts": 19,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302043111",
    "lastReportDate": 1682263731,
    "devType": 1,
    "lastReportWatts": 22,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302045647",
    "lastReportDate": 1682263849,
    "devType": 1,
    "lastReportWatts": 19,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302040087",
    "lastReportDate": 1682263765,
    "devType": 1,
    "lastReportWatts": 22,
    "maxReportWatts": 365
  },
  {
    "serialNumber": "202302044904",
    "lastReportDate": 1682263766,
    "devType": 1,
    "lastReportWatts": 22,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302042798",
    "lastReportDate": 1682263850,
    "devType": 1,
    "lastReportWatts": 19,
    "maxReportWatts": 365
  },
  {
    "serialNumber": "202302045605",
    "lastReportDate": 1682263889,
    "devType": 1,
    "lastReportWatts": 18,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302044785",
    "lastReportDate": 1682263851,
    "devType": 1,
    "lastReportWatts": 18,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302049188",
    "lastReportDate": 1682263585,
    "devType": 1,
    "lastReportWatts": 16,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302030516",
    "lastReportDate": 1682263852,
    "devType": 1,
    "lastReportWatts": 18,
    "maxReportWatts": 365
  },
  {
    "serialNumber": "202302043888",
    "lastReportDate": 1682263586,
    "devType": 1,
    "lastReportWatts": 15,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302036929",
    "lastReportDate": 1682263767,
    "devType": 1,
    "lastReportWatts": 20,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302044503",
    "lastReportDate": 1682263587,
    "devType": 1,
    "lastReportWatts": 16,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302033607",
    "lastReportDate": 1682263768,
    "devType": 1,
    "lastReportWatts": 21,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302042826",
    "lastReportDate": 1682263770,
    "devType": 1,
    "lastReportWatts": 22,
    "maxReportWatts": 365
  },
  {
    "serialNumber": "202302047819",
    "lastReportDate": 1682263588,
    "devType": 1,
    "lastReportWatts": 16,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302040880",
    "lastReportDate": 1682263589,
    "devType": 1,
    "lastReportWatts": 15,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302049339",
    "lastReportDate": 1682263590,
    "devType": 1,
    "lastReportWatts": 15,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302041033",
    "lastReportDate": 1682263591,
    "devType": 1,
    "lastReportWatts": 18,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302044873",
    "lastReportDate": 1682263607,
    "devType": 1,
    "lastReportWatts": 16,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302042463",
    "lastReportDate": 1682263593,
    "devType": 1,
    "lastReportWatts": 18,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302047611",
    "lastReportDate": 1682263594,
    "devType": 1,
    "lastReportWatts": 16,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302050398",
    "lastReportDate": 1682263892,
    "devType": 1,
    "lastReportWatts": 19,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302047170",
    "lastReportDate": 1682263595,
    "devType": 1,
    "lastReportWatts": 17,
    "maxReportWatts": 366
  },
  {
    "serialNumber": "202302043097",
    "lastReportDate": 1682263893,
    "devType": 1,
    "lastReportWatts": 18,
    "maxReportWatts": 366
  }
]
drache42 commented 1 year ago

I may have found something, but I have family obligations so I can't continue to investigate.

from this: https://enphase.com/download/accessing-iq-gateway-local-apis-or-local-ui-token-based-authentication I found api: /ivp/meters/readings

[
    {
        "eid": 704643328,
        "timestamp": 1682264885,
        "actEnergyDlvd": 459059.463,
        "actEnergyRcvd": 0.190,
        "apparentEnergy": 582709.688,
        "reactEnergyLagg": 152072.409,
        "reactEnergyLead": 325.245,
        "instantaneousDemand": 872.703,
        "activePower": 872.703,
        "apparentPower": 1204.716,
        "reactivePower": 671.868,
        "pwrFactor": 0.745,
        "voltage": 242.584,
        "current": 10.031,
        "freq": 60.000,
        "channels": [
            {
                "eid": 1778385169,
                "timestamp": 1682264885,
                "actEnergyDlvd": 229532.586,
                "actEnergyRcvd": 0.097,
                "apparentEnergy": 291444.089,
                "reactEnergyLagg": 76042.029,
                "reactEnergyLead": 173.369,
                "instantaneousDemand": 435.478,
                "activePower": 435.478,
                "apparentPower": 601.536,
                "reactivePower": 335.378,
                "pwrFactor": 0.750,
                "voltage": 121.165,
                "current": 5.015,
                "freq": 60.000
            },
            {
                "eid": 1778385170,
                "timestamp": 1682264885,
                "actEnergyDlvd": 229526.877,
                "actEnergyRcvd": 0.094,
                "apparentEnergy": 291265.599,
                "reactEnergyLagg": 76030.380,
                "reactEnergyLead": 151.877,
                "instantaneousDemand": 437.225,
                "activePower": 437.225,
                "apparentPower": 603.180,
                "reactivePower": 336.491,
                "pwrFactor": 0.741,
                "voltage": 121.420,
                "current": 5.016,
                "freq": 60.000
            },
            {
                "eid": 1778385171,
                "timestamp": 1682264885,
                "actEnergyDlvd": 0.000,
                "actEnergyRcvd": 0.000,
                "apparentEnergy": 0.000,
                "reactEnergyLagg": 0.000,
                "reactEnergyLead": 0.000,
                "instantaneousDemand": 0.000,
                "activePower": 0.000,
                "apparentPower": 0.000,
                "reactivePower": 0.000,
                "pwrFactor": 0.000,
                "voltage": 0.000,
                "current": 0.000,
                "freq": 60.000
            }
        ]
    },
    {
        "eid": 704643584,
        "timestamp": 1682264885,
        "actEnergyDlvd": 449090.493,
        "actEnergyRcvd": 203752.271,
        "apparentEnergy": 824486.568,
        "reactEnergyLagg": 1042.231,
        "reactEnergyLead": 251978.617,
        "instantaneousDemand": 618.232,
        "activePower": 618.232,
        "apparentPower": 1408.171,
        "reactivePower": -1073.496,
        "pwrFactor": 0.438,
        "voltage": 242.498,
        "current": 11.692,
        "freq": 60.000,
        "channels": [
            {
                "eid": 1778385425,
                "timestamp": 1682264885,
                "actEnergyDlvd": 194757.081,
                "actEnergyRcvd": 107160.256,
                "apparentEnergy": 390797.378,
                "reactEnergyLagg": 830.993,
                "reactEnergyLead": 136218.048,
                "instantaneousDemand": 355.750,
                "activePower": 355.750,
                "apparentPower": 738.657,
                "reactivePower": -571.304,
                "pwrFactor": 0.485,
                "voltage": 121.120,
                "current": 6.128,
                "freq": 60.000
            },
            {
                "eid": 1778385426,
                "timestamp": 1682264885,
                "actEnergyDlvd": 254333.412,
                "actEnergyRcvd": 96592.015,
                "apparentEnergy": 433689.189,
                "reactEnergyLagg": 211.238,
                "reactEnergyLead": 115760.569,
                "instantaneousDemand": 262.482,
                "activePower": 262.482,
                "apparentPower": 669.514,
                "reactivePower": -502.192,
                "pwrFactor": 0.387,
                "voltage": 121.378,
                "current": 5.565,
                "freq": 60.000
            },
            {
                "eid": 1778385427,
                "timestamp": 1682264885,
                "actEnergyDlvd": 0.000,
                "actEnergyRcvd": 0.000,
                "apparentEnergy": 0.000,
                "reactEnergyLagg": 0.000,
                "reactEnergyLead": 0.000,
                "instantaneousDemand": 0.000,
                "activePower": 0.000,
                "apparentPower": 0.000,
                "reactivePower": 0.000,
                "pwrFactor": 0.000,
                "voltage": 0.000,
                "current": 0.000,
                "freq": 60.000
            }
        ]
    }
]
drache42 commented 1 year ago

Oh, I have 2 batteries btw

drache42 commented 1 year ago

A little bit more digging, and I found /ivp/livedata/status that seems to give me current battery power usage under meters.storage.agg_p_mw - I think at least. I'm looking at my app and trying to compare numbers in the json directly to what the app is telling me

drache42 commented 1 year ago

Under meters.grid.agg_p_mw it goes into the negative when when I'm exporting to grid, and positive when I'm pulling from the grid. So that's a good sign.

drache42 commented 1 year ago

I found this, which might be helpful. Given how much work you've already done, you may have already known about it. https://github.com/Matthew1471/Enphase-API/blob/main/README.adoc

dlmcpaul commented 1 year ago

Hmm, I read the envoy expecting either a list of storage devices (type acb) or an entry in the inverters (devtype 11). This page gives a little info on the production endpoint and it thinks storage devices should show up https://github.com/Matthew1471/Enphase-API/blob/main/Documentation/IQ%20Gateway%20API/General/Production.adoc

Can you get the raw data from /production.json?details=1

drache42 commented 1 year ago

Here is the output.

Note, it's currently dark out and my batteries are set to full backup and are fully charged, and are not supplying any power to the house.

Also, some other facts about my house 34 panels, in across 2 buildings. 2 batteries, with 3 cells each

I can get you the output again tomorrow, with the batteries charging/discharging etc.

/production.json?details=1

{
  "production": [
    {
      "type": "inverters",
      "activeCount": 34,
      "readingTime": 1682305324,
      "wNow": 11,
      "whLifetime": 490131
    },
    {
      "type": "eim",
      "activeCount": 1,
      "measurementType": "production",
      "readingTime": 1682305324,
      "wNow": 0,
      "whLifetime": 487660.539,
      "varhLeadLifetime": 385.95,
      "varhLagLifetime": 157507.846,
      "vahLifetime": 614619.943,
      "rmsCurrent": 5.722,
      "rmsVoltage": 243.62,
      "reactPwr": 691.625,
      "apprntPwr": 697.014,
      "pwrFactor": 0,
      "whToday": 29257.539,
      "whLastSevenDays": 458404.539,
      "vahToday": 38378.943,
      "varhLeadToday": 61.95,
      "varhLagToday": 11591.846,
      "lines": [
        {
          "wNow": 0,
          "whLifetime": 243821.591,
          "varhLeadLifetime": 206.886,
          "varhLagLifetime": 78754.495,
          "vahLifetime": 307392.528,
          "rmsCurrent": 2.86,
          "rmsVoltage": 122.058,
          "reactPwr": 346.905,
          "apprntPwr": 349.025,
          "pwrFactor": 0,
          "whToday": 14616.591,
          "whLastSevenDays": 229205.591,
          "vahToday": 19179.528,
          "varhLeadToday": 33.886,
          "varhLagToday": 5789.495
        },
        {
          "wNow": 0,
          "whLifetime": 243838.948,
          "varhLeadLifetime": 179.063,
          "varhLagLifetime": 78753.351,
          "vahLifetime": 307227.415,
          "rmsCurrent": 2.862,
          "rmsVoltage": 121.561,
          "reactPwr": 344.72,
          "apprntPwr": 347.989,
          "pwrFactor": 0,
          "whToday": 14640.948,
          "whLastSevenDays": 229198.948,
          "vahToday": 19199.415,
          "varhLeadToday": 28.063,
          "varhLagToday": 5802.351
        }
      ]
    }
  ],
  "consumption": [
    {
      "type": "eim",
      "activeCount": 1,
      "measurementType": "total-consumption",
      "readingTime": 1682305324,
      "wNow": 13268.967,
      "whLifetime": 727739.318,
      "varhLeadLifetime": 262157.469,
      "varhLagLifetime": 158557.79,
      "vahLifetime": 859863.349,
      "rmsCurrent": 115.179,
      "rmsVoltage": 243.62,
      "reactPwr": -1444.503,
      "apprntPwr": 28059.759,
      "pwrFactor": 0.47,
      "whToday": 36013.318,
      "whLastSevenDays": 692131.318,
      "vahToday": 52328.349,
      "varhLeadToday": 19875.469,
      "varhLagToday": 11599.79,
      "lines": [
        {
          "wNow": 6289.291,
          "whLifetime": 328201.883,
          "varhLeadLifetime": 141777.927,
          "varhLagLifetime": 79585.654,
          "vahLifetime": 407621.094,
          "rmsCurrent": 54.597,
          "rmsVoltage": 122.059,
          "reactPwr": -895.763,
          "apprntPwr": 6664.014,
          "pwrFactor": 0.94,
          "whToday": 17526.883,
          "whLastSevenDays": 310863.883,
          "vahToday": 25416.094,
          "varhLeadToday": 10736.927,
          "varhLagToday": 5789.654
        },
        {
          "wNow": 6979.676,
          "whLifetime": 399537.434,
          "varhLeadLifetime": 120379.542,
          "varhLagLifetime": 78972.136,
          "vahLifetime": 452242.255,
          "rmsCurrent": 60.582,
          "rmsVoltage": 121.56,
          "reactPwr": -548.74,
          "apprntPwr": 7364.373,
          "pwrFactor": 0.95,
          "whToday": 18486.434,
          "whLastSevenDays": 381267.434,
          "vahToday": 26912.255,
          "varhLeadToday": 9138.542,
          "varhLagToday": 5810.136
        }
      ]
    },
    {
      "type": "eim",
      "activeCount": 1,
      "measurementType": "net-consumption",
      "readingTime": 1682305324,
      "wNow": 13268.967,
      "whLifetime": 460659.108,
      "varhLeadLifetime": 261771.519,
      "varhLagLifetime": 1049.945,
      "vahLifetime": 859863.349,
      "rmsCurrent": 109.456,
      "rmsVoltage": 243.62,
      "reactPwr": -752.879,
      "apprntPwr": 13326.122,
      "pwrFactor": 1,
      "whToday": 0,
      "whLastSevenDays": 0,
      "vahToday": 0,
      "varhLeadToday": 0,
      "varhLagToday": 0,
      "lines": [
        {
          "wNow": 6289.291,
          "whLifetime": 199803.425,
          "varhLeadLifetime": 141571.041,
          "varhLagLifetime": 831.159,
          "vahLifetime": 407621.094,
          "rmsCurrent": 51.736,
          "rmsVoltage": 122.06,
          "reactPwr": -548.859,
          "apprntPwr": 6308.796,
          "pwrFactor": 1,
          "whToday": 0,
          "whLastSevenDays": 0,
          "vahToday": 0,
          "varhLeadToday": 0,
          "varhLagToday": 0
        },
        {
          "wNow": 6979.676,
          "whLifetime": 260855.682,
          "varhLeadLifetime": 120200.479,
          "varhLagLifetime": 218.785,
          "vahLifetime": 452242.255,
          "rmsCurrent": 57.72,
          "rmsVoltage": 121.56,
          "reactPwr": -204.02,
          "apprntPwr": 7017.326,
          "pwrFactor": 0.99,
          "whToday": 0,
          "whLastSevenDays": 0,
          "vahToday": 0,
          "varhLeadToday": 0,
          "varhLagToday": 0
        }
      ]
    }
  ],
  "storage": [
    {
      "type": "acb",
      "activeCount": 0,
      "readingTime": 0,
      "wNow": 0,
      "whNow": 0,
      "state": "idle"
    }
  ]
}
dlmcpaul commented 1 year ago

Looks likes the batteries are not wired into the enphase controller.

Is that something you can ask your installer about?

"storage": [
    {
      "type": "acb",
      "activeCount": 0,
      "readingTime": 0,
      "wNow": 0,
      "whNow": 0,
      "state": "idle"
    }
  ]

Also no percentFull value which I was expecting.

drache42 commented 1 year ago

Interesting. Seems like they have a bug...

/inventory.json returns a lot of stuff, but ACB is empty

  {
    "type": "ACB",
    "devices": []
  },

However, /ivp/livedata/status returns storage data

"storage": {
            "agg_p_mw": -86000,
            "agg_s_mva": -633922,
            "agg_p_ph_a_mw": -86000,
            "agg_p_ph_b_mw": 0,
            "agg_p_ph_c_mw": 0,
            "agg_s_ph_a_mva": -633922,
            "agg_s_ph_b_mva": 0,
            "agg_s_ph_c_mva": 0
        },

The enphase app clearly shows my battery status, so it's physically hooked up and working. Seems like these APIs aren't returning the correct data through. I'll open a support ticket with enphase to see what's up.

dlmcpaul commented 1 year ago

Yes, the perils of trying to reverse engineer the data :-)

If the app is showing the battery data then it should be in there somewhere but will enphase tell us.

using the /ivp/livedata/status does agg_p_mw match anything you know about your batteries?

86000 = ?KWh

drache42 commented 1 year ago

I've asked enphase to see if I can get more documentation on the APIs... we'll see.

I have the enphase IQ10 batteries. 2 of them, which should hold 20kWh https://enphase.com/download/iq-battery-10-data-sheet

They are currently fully charged and /ivp/livedata/status return

"storage": { "agg_p_mw": -99000, "agg_s_mva": -99000, "agg_p_ph_a_mw": -99000, "agg_p_ph_b_mw": 0, "agg_p_ph_c_mw": 0, "agg_s_ph_a_mva": -99000, "agg_s_ph_b_mva": 0, "agg_s_ph_c_mva": 0 },

No idea what -99000 means though :(

-Steffen

On Mon, Apr 24, 2023 at 3:21 PM dlmcpaul @.***> wrote:

Yes, the perils of trying to reverse engineer the data :-)

If the app is showing the battery data then it should be in there somewhere but will enphase tell us.

using the /ivp/livedata/status does agg_p_mw match anything you know about your batteries?

86000 = ?KWh

— Reply to this email directly, view it on GitHub https://github.com/dlmcpaul/EnphaseCollector/issues/55#issuecomment-1520902636, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD2PAIYP72Y56QQ2OVXKLDLXC34GTANCNFSM6AAAAAAXH4E3BQ . You are receiving this because you authored the thread.Message ID: @.***>

dlmcpaul commented 1 year ago

mw suggests watts mva suggest volt amps

Batteries are usually rated with kWh and kVA so that might line up.

But honestly my first thought was some sort of percentage figure. Fully charged could be 99% and the previous reading you sent could be 86%

Maybe just record the values and compare to what the app shows.

drache42 commented 1 year ago

Recording the value would be a great idea! If you treat it like other metrics and just write it to a debug table in the database, then we could graph it and compare to the app graphs. Or if that doesn't work. We can try to use the cloud api to download the same timeframe, and then see what the values compare against.

dlmcpaul commented 1 year ago

Hmm, it looks like /ivp/livedata/status does not exist for the V5 enphase release.

Before we get too far, maybe just manually record some values.

drache42 commented 1 year ago

It would have been too simple if the current cloud api version had the same API as the v1 local API.

I was hoping that, just maybe, one of the current APIs returned a similar JSON so we could compare, but I didn't see anything.

But we could use the current cloud API to record the battery state, and then compare that to what we're seeing, but it is a lot of extra effort.

If we just record the current value, then I'll get it up and running and I'll do a side by side graph comparison of the graphs to see if I can find a pattern.

On Mon, Apr 24, 2023, 20:16 dlmcpaul @.***> wrote:

Hmm, it looks like /ivp/livedata/status does not exist for the V5 enphase release.

Before we get too far, maybe just manually record some values.

— Reply to this email directly, view it on GitHub https://github.com/dlmcpaul/EnphaseCollector/issues/55#issuecomment-1521100779, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD2PAI564XX4B62HV42GJX3XC46XBANCNFSM6AAAAAAXH4E3BQ . You are receiving this because you authored the thread.Message ID: @.***>

drache42 commented 1 year ago

After asking enphase very nicely over and over again, I finally got some documentation on the REST API Endpoints :)

I think the following endpoint may get us battery power, but I haven't had a chance to try it yet. /ivp/sc/status

ENVOY-Envoy REST Interface-150523-095555.pdf

dlmcpaul commented 1 year ago

Ok, when you do let me know the response you get and I can see about creating a model.

Matthew1471 commented 1 year ago

Hello. I just wanted to add that from my learning of the API there is a difference between:

.. ACB couldn't be deployed outside, talked directly to the Gateway via PowerLine Communications (PLC) just like the microinverters and ..

.. IQ Battery talks to an IQ System Controller (over Zigbee in Gen2, Wired CAN bus in Gen3). The IQ System Controller (formerly "Enpower MID R2" or microgrid interconnection device) contains the grid relays too.

The IQ Combiner contains an IQ Gateway and the network communication (to bring Internet connectivity etc). The IQ Combiner for Gen2 will chat to the IQ System Controller also over ZigBee.. and for Gen3 hardwired CAN bus again.

The production.json of a IQ Gateway/IQ Combiner is of the era of AC Battery.. but for any battery stats of the IQ Battery one must call the Enpower/Encharge/Ensemble APIs.

Enpower = "old name for IQ System Controller" Encharge = "old name for battery storage" Ensemble = "old name for Energy System" (batteries, combiner, smart switch etc)

This page is helpful to work out the terms : https://support.enphase.com/s/article/What-are-the-new-names-for-Enphase-products-And-why

The meters may also have sight of "battery data" from the CT clamps. I don't have Enphase battery storage as they do not appear to offer it in my country (United Kingdom). However my Telsa PowerWalls are being fitted this week. I'm happy to update my API documentation with other people's API response data however. The document from https://github.com/dlmcpaul/EnphaseCollector/issues/55#issuecomment-1548116709 looks like it's straight out of the Enphase Confluence, which is kind of fascinating considering their documentation seems more patchy than mine. I'm still trying to find out what "IVP" (in the URLs) stands for as part of my goal with the API documentation was to expand any terms referenced.

dlmcpaul commented 4 months ago

Well I now have Enphase IQ Battery 5P added to my system so I have been poking about a bit.

I can see the total battery availability and the current state of charge but have yet to find the usage values. Will look more at the /ivp/livedata/status endpoint.