Can't get Docker Build to Complete

[Doug910]

I downloaded Docker Desktop 4.12.0 on PC just to run this collector, did docker build, it goes through quite a way but then gets an error:

[builder 4/4] RUN "/usr/lib/jvm/zulu17-ca/bin/jar" -xf app.jar:

10 0.770 java.io.FileNotFoundException: app.jar (Is a directory)

Anyone know how to get past this?

[dlmcpaul]

If you want to build from scratch you should use the Dockerfile-build file with a command like docker build -f Dockerfile-build .

You probably also want to supply a tag with the -t param

The default dockerfile is used after the jar is built externally and is passed as an argument

[Doug910]

That build worked, thanks for the quick response, there are quite a few Dockerfiles and I didn't know which one to use, I have it running now. It is dark here currently so that may be a problem but all my readings are coming in as 0 even Production today and Highest output so far today, only the time is showing non zero values. My Envoy is on version D5.0.55.

[dlmcpaul]

The default docker file is there for my build server. Perhaps I should make the build from source one the default.

Consumption should be non zero I would think if everything is working. Be aware that building from source puts you on the bleeding edge. I am currently working on the new V7 authentication so may have broken stuff.

Check logs obviously and post any error messages.

[Doug910]

It is daylight now and I'm still getting 0 readings, I have Envoy S, so no consumption readings and I'm also using Ethernet so no WiFi configuration. I'm getting an "Enphase data collection failed" message so EnvoyService is not OK. Looking at the code any readings that fail will set the EnvoyService to not OK, is that correct.

I can check the log files, where do I find them.

[dlmcpaul]

Yes, any failure when requesting data will set that flag.

If you are running docker then docker logs should show them (use docker help logs to see all the options)

Another option is when you run the container to add the -it flags so all logs will be written to your console.

[dlmcpaul]

With envoy-s I think the most likely issue is the lack of power and device endpoints or they are returning no data.

but if there is an error in the log we have a chance to figure it out.

Please also add the command line used to run the container. Might be as simple as unable to find the envoy.

[Doug910]

I used this command to run the collector, I added the -it additional flag: docker run -it -e TZ=Europe/London -p 8080:8080 enphasecollector

After the initialisation, I'm not seeing anything come out on the console until I refresh the /solar web page, then I get:

2022-09-20 10:42:31.740 INFO 1 --- [ scheduling-1] o.apache.http.impl.execchain.RetryExec : I/O exception (java.net.UnknownHostException) caught when processing request to {}->http://envoy.local:80: envoy.local: Try again 2022-09-20 10:42:31.742 INFO 1 --- [ scheduling-1] o.apache.http.impl.execchain.RetryExec : Retrying request to {}->http://envoy.local:80 2022-09-20 10:42:51.750 INFO 1 --- [ scheduling-1] o.apache.http.impl.execchain.RetryExec : I/O exception (java.net.UnknownHostException) caught when processing request to {}->http://envoy.local:80: envoy.local: Try again 2022-09-20 10:42:51.750 INFO 1 --- [ scheduling-1] o.apache.http.impl.execchain.RetryExec : Retrying request to {}->http://envoy.local:80 2022-09-20 10:43:11.757 INFO 1 --- [ scheduling-1] o.apache.http.impl.execchain.RetryExec : I/O exception (java.net.UnknownHostException) caught when processing request to {}->http://envoy.local:80: envoy.local: Try again 2022-09-20 10:43:11.758 INFO 1 --- [ scheduling-1] o.apache.http.impl.execchain.RetryExec : Retrying request to {}->http://envoy.local:80 2022-09-20 10:44:04.243 INFO 1 --- [ scheduling-1] o.apache.http.impl.execchain.RetryExec : I/O exception (java.net.UnknownHostException) caught when processing request to {}->http://envoy.local:80: envoy.local: Try again 2022-09-20 10:44:04.243 INFO 1 --- [ scheduling-1] o.apache.http.impl.execchain.RetryExec : Retrying request to {}->http://envoy.local:80

I can use http://envoy.local on my browser and that does connect to the Envoy home page.

[dlmcpaul]

Ah welcome to the wonderful world of containers. Docker containers by default don't connect to DNS or similar network naming services. So you will need to supply the ip address of your envoy

eg docker run -it -e TZ=Europe/London -e ENVOY_CONTROLLER_HOST=envoy-ip -p 8080:8080 enphasecollector

Also note this will create a database inside the container. Not a problem unless you want to keep the data long term.

[Doug910]

I used this command to run a new instance: docker run -it -e TZ=Europe/London -e ENVOY_CONTROLLER_HOST=192.168.1.233 -p 8080:8080 enphasecollector

When I refresh the browser I'm not getting the host exceptions that I saw before, so that is a good sign, but I'm still getting all 0 readings and no log output on the console to say why.

[dlmcpaul]

Hmm, you should be getting something in your logs even if it is just the startup logging.

Ok lets try the following. Send me the results of

docker info

docker images

docker pull dlmcpaul/enphasecollector:0.25

docker run -it -e TZ=Europe/London -e ENVOY_CONTROLLER_HOST=192.168.1.233 -p 8080:8080 dlmcpaul/enphasecollector:0.25

And what OS you using.

[Doug910]

OS is Windows 10 Pro build 21H2. Running the 0.25 collector gave the same result, 0 readings on the web page and no log output. I have attached text files for the other outputs you wanted to see:

docker_info.txt docker_images.txt docker_pull.txt

I will try putting prints in the code to see where it is going, that is probably an easier way to debug it.

[dlmcpaul]

It is weird that no logs are being produced. Does this only happen to the enphasecollector container?

You could also try looking under \wsl$\docker-desktop-data\version-pack-data\community\docker\containers for the logs

[Doug910]

When I look in \wsl$\docker-desktop-data I don't see a version-pack-data directory, just the regular Unix directories bin, data, dev, etc.

I noticed this line in the start up console output: Reading from installer Envoy controller endpoint http://192.168.1.233/stream/meter

On my Envoy (D5.0.55) /stream/meter does not produce any output, I tried it on a browser (logged in as installer), on my own Python scripts and on the Python package envoy-utils that has a function to read this stream, in each case the command is sent but the response has no data in it.

[dlmcpaul]

Ok so you are getting logs. Can you please post them.

I configure a reader for that endpoint but it should never be called. It is a long term goal to allow for real time reads.

[Doug910]

I'm getting log output for start up, but after it says Completed initialization, nothing else comes out.

0.25_startup_log.txt latest_startup_log.txt

[dlmcpaul]

Ok that's good it gives us something to work with.

Looks like we are reading ok from home.json but not getting to the inverters part /api/v1/production/inverters

I have pushed a test version of EnvoyService with more logging. Build a new image and try it out.

[Doug910]

It did produce extra output, lots of lines saying "System Read time Optional.empty"

latest_debug_log.txt

On my Envoy I have to log in as installer to get /api/v1/production/inverters to work.

[dlmcpaul]

Ok that makes some sort of sense if you don't have a production eim. Have pushed 2 changes so please rebuild and try again.

Might need more work though I may have more places I expect a production eim to exist.

[dlmcpaul]

More fixes. Sorry

[Doug910]

I downloaded about the same time as your last change, so I did another download just now, it is getting further and now says Envoy Data Collection Successful, but then there is a null list exception.

latest_debug3_log.txt

[dlmcpaul]

Ok, lets go back to basics. Could you attach the output of these urls (using curl or wget)

http://192.168.1.233/production.json?details=1 http://192.168.1.233/inventory.json?deleted=1

Then I can do some local testing to see what data you have available there.

[Doug910]

I have generated the outputs from a test system with two panels, so not too much data to go through:

production.txt Inventory.txt

The crash was when processing consumption data, you will not get any readings for that, since this Envoy has production metering only.

[dlmcpaul]

Thanks, I have used the data to create a test case and fix other issue I could find.

Please test the new code.

[Doug910]

All looks good now, the production numbers and panel data are showing up, the chart is being updated every minute. Voltage is 0, but I guess that is the battery. Because consumption is 0, it thinks all my solar power is being exported to the grid, maybe that could be disabled.

[dlmcpaul]

I look for Voltage in the PowerMeter list (from /ivp/meters/readings) which is not available on your device.

I then check for a "production" eim entry from the productions endpoint (/production.json) but you only seem to have "inverters" and "storage"

If you know another place it might be set I can look for it.

I will try some code to zero out export etc if consumption is 0. Look out for a checkin.

[Doug910]

The export values are gone now, thanks for that. The collection time is being updated ok, but the Last communication to Enphase is only set once when the program starts, but then doesn't change. I don't like the way the status list keeps moving around, it would be easier to read if the numeric values stayed in the same place, I can see you have a lot of items to show and you would have to make them smaller or use two columns to get them all on the screen. Alternatively if the user could configure which ones they want to see, for example I would only show the production numbers and collection time.

[dlmcpaul]

Last comm to enphase should be fixed soon. I am experimenting with caching calls to the device to not overload it but need to rethink it a bit.

The status list refreshes every 60s but since adding the individual panel values the complete list is rather large. I am hoping to reduce the size by grouping the panels and for your use case removing entries that don't make sense.

If I can think of an easy way to configure the available stats I will try to add it.

[Doug910]

I noticed that the chart has 1 minute data points, but the production value only changes every 5 mins, I guess this is taken from the Envoy home page. It might be good to have a high resolution mode where it uses the installer overview this gives a total production power update every minute.

[dlmcpaul]

Each envoy device updates at different rates. You can set the poll rate I read things from the default of 60s.

The installer endpoint is supposed to be realtime but I find I never needed it.

I have pushed some more fixes for you to try out.

• Fixed bug of caching system object that held last communication value
• Reduced the stats shown to the bare minimum for a device with no consumption values and turned off shuffle.

[Doug910]

I'm seeing the connection to Enphase being updated now, so I must have the latest version, but I don't see any difference in the the status list, it still contains all the consumption entries like Grid import today and Cost today for grid usage and it is still being shuffled.

[dlmcpaul]

Oops, left my test reversed while testing. Try again

[Doug910]

The consumption data is gone, which is good, but there are still changes to the status list, when started it showed a list of panel power values at the top, then after 5 mins these were gone and it added "solar panels producing max" at the bottom, this moved everything else up. With a 2 panel system this is not too bad, but with 8 panels it would be more distracting, maybe you only need to show the max and the min panel power all the time.

[dlmcpaul]

I think I know what you are saying but perhaps some screenshots will help me understand?

[Doug910]

It alternates between 3 different layouts of the status list, so the values in the middle move around a bit, but not too much with my small system, probably not worth changing.

[dlmcpaul]

Ah ok. So the stats panel alternates between the original system summary stats and an individual panel production stats which tries to group similar producing panels together. Currently 5W buckets. Still trying to decide if such data is useful to know or not.

Having only 2 panels does not make it very useful :-(

Well if you are happy with what is working now. I will close this issue off. There will likely be a release in the near future but you know how to build your own images so no rush I expect.

[Doug910]

If I download a zip from github onto my PC drive, do you have a dockerfile that would build from this local copy, rather than pulling from github directly. I tried replacing the git pull with a "COPY pom.xml ." and that builds most of the way, but then fails when trying to push into a docker copy of git.

[dlmcpaul]

Maybe try something like this and pass the zip file as an argument to the docker build command

FROM azul/zulu-openjdk-alpine:17 as builder
LABEL maintainer="dlmcpaul@gmail.com"

RUN apk update && apk add git maven unzip
ARG ZIP_FILE
COPY ${ZIP_FILE} /source.zip
RUN unzip -d ./EnphaseCollector /source.zip 

WORKDIR EnphaseCollector
RUN mvn package -Dmaven.test.skip
RUN cp target/enphasecollector-development-SNAPSHOT.jar /app.jar
WORKDIR /

# Generate a JDK class data share
RUN "$JAVA_HOME/bin/java" -Xshare:dump

# Explode Uber jar into lib jars and classes
RUN "$JAVA_HOME/bin/jar" -xf app.jar

FROM azul/zulu-openjdk-alpine:17-jre-headless
LABEL maintainer="dlmcpaul@gmail.com"

COPY --from=builder "./BOOT-INF/lib" /app/lib
COPY --from=builder "./META-INF" /app/META-INF
COPY --from=builder "./BOOT-INF/classes" /app
COPY --from=builder "${JAVA_HOME}/lib/server/classes.jsa" "${JAVA_HOME}/lib/server"

ENV SPRING_DATASOURCE_URL=jdbc:h2:/internal_db/solar_stats_db
RUN mkdir "/properties"
RUN touch "/properties/application.properties"

ENTRYPOINT ["java", "-cp", "app:app/lib/*", "-Xshare:auto", "-Djava.security.egd=file:/dev/./urandom", "-Dspring.jmx.enabled=false", "com.hz.EnphaseCollectorApplication", "--spring.config.additional-location=file:/properties/application.properties"]

EXPOSE 8080

VOLUME /internal_db /properties

[dlmcpaul]

Added more documentation on the docker images. If more is needed please let me know