Closed CapnJackOff closed 4 years ago
Thanks for taking an interest. I knew there were other products that use the same protocol. I'll add that into the readme sometime. Do you happen to know the name of that Chinese company?
Ah yes, it looks like I didn't change all of the references to systemd/install.sh
when I moved it to other/
. I will change that.
Right now when messing with anything in /opt/solarthing
, only root has permission, so you have to use sudo
with everything. If you want to, you can run other/linux/create_group.sh
and other/linux/update_perms.sh
and then add yourself to the solarthing
group and log out and back in, but it's a lot easier to just do everything with sudo
. I plan to change this in the future since it's not the best that SolarThing runs with root privileges all the time.
When running java -jar solarthing.jar base/config.json
, it will attempt to create log files in program/logs
. This also requires root permission, so you have to run it with sudo. Running that command directly is a great way to test out stuff, but it might be simpler to just run ./solarthing.sh
which will work just the same with slightly less typing.
Also, except logging, the command java -jar solarthing.jar
worked as expected because you didn't pass the base config argument. You should instead be running java -jar solarthing.jar config/base.json
, or just run ./solarthing.sh
like I said above. You can see at the very bottom of the log it says (Fatal)Incorrect args
. Also know that if the service is running at the same time to try to run it yourself, there might be problems. You can stop it by using sudo systemctl stop solarthing
, and also stop it from starting on boot by using sudo systemctl disable solarthing
. Once you get everything setup, I recommend enabling the service. The service is a great way to make running and restarting your program very easy, especially while running it long term.
The files in config_templates
are meant to be copied to program/config
and then edited to your liking, so use them as you see fit. I don't know if this is intentional, but rover_dummy_setup_template.json
is used for testing SolarThing without a real charge controller to test with (hence dummy). My guess is that you want to use rover_template.json
so you can copy that to the program/config
directory and name it base.json
.
You will then copy a database config to program/config
and then make sure the databases
section in your base.json
links to it correctly.
Are you planning on setting up Grafana? If yes you'll probably run the command sudo cp ../config_templates/databases/influxdb_template.json config/influxdb.json
while in the program
directory.
If any of this is confusing please let me know. Your feedback has already helped me decide how to make improvements to the documentation, so thank you.
All good information, and thanks so much for your help. I confess I'm pretty new to RPi and I couldn't wait to get it connected and get some feedback. That is why I went with the dummy config to get started. I will try running sudo java instead. The 7" screen I bought went kaput within hours so I'll have to wait a few days to test the whole thing with the charge controller out in the backyard. Since I haven't actually tested the cable, I'll also have to plug that in to a Windows box with the Renogy Solar Monitoring program to test to make sure it works. I have bought RJ to USB cables on the internet which do not work, so I believe what you say is true about that proprietary pinout and the necessity of either getting a Renogy cable or making one. Eliminating that as a source of error will speed things along I hope.
The Chinese company is called SRNE Solar. https://www.srnesolar.com/ Interestingly, there is a link to them in some of those forum threads you posted in your documentation about how to make a cable. I would not be surprised to see those links die soon since those threads are being moved, archived, pruned etc. Grape Solar has no interest whatsoever in supporting features involving that serial port unless it is related to a device they sell such as the bluetooth communication module, the remote display etc. It would not surprise me if the folks at Renogy felt the same. The good thing is, I doubt that port will disappear in future models since it is needed for a remote display, however, I have seen "generic" models being sold around the internet without that port. Same is true of the EP Solar Tracer.
I do want to use Grafana - that is a thing of beauty! But I will probably not install it on the pi since I want public access to the data. My hope is that I can install the DB and Grafana on my shared webserver, and the pi will upload to them. I will hopefully install the right config files for that when the time comes to pump some data!
Since you just want to test it out without a database, you could probably use something like this as your base.json:
{
"type": "rover",
"source": "default",
"fragment": 2,
"unique": 30,
"databases": [],
"io": "../config_templates/io/default_linux_serial.json",
"modbus": 1
}
By using that config, the program should spit out data from your charge controller about every 5 seconds. Since the databases
field is an empty array, it won't put it anywhere.
Once you're ready to get it working on Grafana, you will probably add the InfluxDB "database". There's a template for it in config_templates/databases
.
I'm going to go ahead and close this issue. Once you're able to work on it again I'd love to hear if you're able to get this working. Feel free to open this back up if you still need help.
Ok, thanks! I got my Pi screen last night and I will set it up for real on the Zenith this week.
-----Original Message-----
From: Joshua Shannon notifications@github.com
Sent: Monday, May 04, 2020 3:34 PM
To: wildmountainfarms/solarthing solarthing@noreply.github.com
Cc: CapnJackOff amspay@greenfreedom.net; Author author@noreply.github.com
Subject: Re: [wildmountainfarms/solarthing] Could
I'm going to go ahead and close this issue. Once you're able to work on it again I'd love to hear if you're able to get this working. Feel free to open this back up if you still need help.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wildmountainfarms/solarthing/issues/5#issuecomment-623742963 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBL2NIUWOTD5TQ7RVGCFK3RP47EJANCNFSM4MN2V3OQ . https://github.com/notifications/beacon/AIBL2NLKPTB2DYUAXO2T7WTRP47EJA5CNFSM4MN2V3O2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEUWY74Y.gif
Awesome. Just to let you know, I just made huge changes today which affect old clones of the repository. I recommend that you clone the repository again with this command:
sudo git clone --single-branch https://github.com/wildmountainfarms/solarthing
A change this big (hopefully) will never happen again. I had to remove old jar files and git really doesn't like when that happens.
The main difference that you'll see is faster clone times, you'll have to run ./download_solarthing.sh to download the jar file, and the structure of the program directory has changed.
If you previously configured stuff, you'll have to backup some of that config, reclone, then copy it into program/rover/config. This also means that some of what I said above is outdated now, but it shouldn't be too difficult to figure out because you'll mostly just be putting stuff in program/rover rather than just program/.
I was wondering if you were able to test out your charge controller and if you got it working at all. If you had trouble with SolarThing it'd be helpful for me to know if there's anything I can improve or anything you got stuck on.
I was wondering if you were able to test out your charge controller and if you got it working at all. If you had trouble with SolarThing it'd be helpful for me to know if there's anything I can improve or anything you got stuck on.
Thanks so much for getting in touch. I am still trying to navigate the github messaging system. Somehow I am able to reply to a closed topic. Yesterday, I tested the cable with a PC and the Rover (Zenith), and it is working so pleased about that. No need to worry about the physical connection. I'm going through your docs again and seeing that A LOT has changed. You mentioned that a big rewrite had happened and that I should re-fork. I'm making progress. I have got the new files downloaded and the rover service installed. I installed and started InfluxDB. Also copied the config files to rover/config and edited base.json with only the influxdb entry. Ran sudo ./run. Yay, something happened! It seems to be hunting for Couchdb files. Not sure if that's a problem. Not connected to a rover yet. I am not sure if I need to configure any tables in Influx. Here is the output (without Rover)
pi@raspberrypi:~ $ sudo apt install influxdb
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
influxdb
0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded.
Need to get 3,729 kB of archives.
After this operation, 13.9 MB of additional disk space will be used.
Get:1 http://mirror.pit.teraswitch.com/raspbian/raspbian buster/main armhf influxdb armhf 1.6.4-1 [3,729 kB]
Fetched 3,729 kB in 2s (1,589 kB/s)
Selecting previously unselected package influxdb.
(Reading database ... 94967 files and directories currently installed.)
Preparing to unpack .../influxdb_1.6.4-1_armhf.deb ...
Unpacking influxdb (1.6.4-1) ...
Setting up influxdb (1.6.4-1) ...
Adding system user influxdb' (UID 110) ... Adding new user
influxdb' (UID 110) with group nogroup' ... Not creating home directory
/var/lib/influxdb'.
Adding group influxdb' (GID 115) ... Done. Adding user
influxdb' to group `influxdb' ...
Adding user influxdb to group influxdb
Done.
Created symlink /etc/systemd/system/influxd.service → /lib/systemd/system/influxdb.service.
Created symlink /etc/systemd/system/multi-user.target.wants/influxdb.service → /lib/systemd/system/influxdb.service.
Processing triggers for systemd (241-7~deb10u3+rpi1) ...
Processing triggers for man-db (2.8.5-2) ...
pi@raspberrypi:~ $ sudo systemctl unmask influxdb
pi@raspberrypi:~ $ sudo systemctl enable influxdb
Synchronizing state of influxdb.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable influxdb
pi@raspberrypi:~ $ sudo systemctl start influxdb
pi@raspberrypi:~ $ cd /opt/solarthing/program/rover
pi@raspberrypi:/opt/solarthing/program/rover $ cp ../../config_templates/io/default_linux_serial.json config/
cp: cannot create regular file 'config/default_linux_serial.json': Permission denied
pi@raspberrypi:/opt/solarthing/program/rover $ sudo cp ../../config_templates/io/default_linux_serial.json config/
pi@raspberrypi:/opt/solarthing/program/rover $ sudo cp ../../config_templates/base/rover_template.json config/base.json
pi@raspberrypi:/opt/solarthing/program/rover $ sudo cp ../../config_templates/databases/influxdb_template.json config/influxdb.json
pi@raspberrypi:/opt/solarthing/program/rover $ sudo ./run.sh
Using base config = config/base.json
Going to download version: 2020.2.1
--2020-05-29 14:47:00-- https://github.com/wildmountainfarms/solarthing/releases/download/v2020.2.1/solarthing-2020.2.1.jar
Resolving github.com (github.com)... 192.30.255.113
Connecting to github.com (github.com)|192.30.255.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/195026020/0c169080-9acd-11ea-9061-b1dead86ea92?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200529%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200529T214701Z&X-Amz-Expires=300&X-Amz-Signature=2ab6732f101ac07ae5a13d4f1a30ff0687610152f594b749a1c1bf24a896c560&X-Amz-SignedHeaders=host&actor_id=0&repo_id=195026020&response-content-disposition=attachment%3B%20filename%3Dsolarthing-2020.2.1.jar&response-content-type=application%2Foctet-stream [following]
--2020-05-29 14:47:01-- https://github-production-release-asset-2e65be.s3.amazonaws.com/195026020/0c169080-9acd-11ea-9061-b1dead86ea92?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200529%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200529T214701Z&X-Amz-Expires=300&X-Amz-Signature=2ab6732f101ac07ae5a13d4f1a30ff0687610152f594b749a1c1bf24a896c560&X-Amz-SignedHeaders=host&actor_id=0&repo_id=195026020&response-content-disposition=attachment%3B%20filename%3Dsolarthing-2020.2.1.jar&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.163.59
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.163.59|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17756677 (17M) [application/octet-stream]
Saving to: ‘solarthing-2020.2.1.jar’
solarthing-2 100% 16.93M 1.32MB/s in 11s
2020-05-29 14:47:12 (1.55 MB/s) - ‘solarthing-2020.2.1.jar’ saved [17756677/17756677]
Using SolarThing Version: 2020.2.1
2020-05-29 14:47:19.071 [main] INFO me.retrodaredevil.solarthing.program.SolarMain - [LOG] Beginning main
[stdout] Beginning main
2020-05-29 14:47:19.666 [main] INFO me.retrodaredevil.solarthing.program.RoverMain - Beginning rover program
2020-05-29 14:47:19.669 [main] ERROR me.retrodaredevil.solarthing.program.SolarMain - (Fatal)Got throwable
java.lang.RuntimeException: java.io.FileNotFoundException: config/couchdb.json (No such file or directory)
at me.retrodaredevil.solarthing.program.SolarMain.getDatabaseConfig(SolarMain.java:119) ~[solarthing-2020.2.1.jar:?]
at me.retrodaredevil.solarthing.program.SolarMain.getDatabaseConfigs(SolarMain.java:110) ~[solarthing-2020.2.1.jar:?]
at me.retrodaredevil.solarthing.program.RoverMain.connectRover(RoverMain.java:114) ~[solarthing-2020.2.1.jar:?]
at me.retrodaredevil.solarthing.program.SolarMain.doMain(SolarMain.java:194) [solarthing-2020.2.1.jar:?]
at me.retrodaredevil.solarthing.program.SolarMain.main(SolarMain.java:210) [solarthing-2020.2.1.jar:?]
Caused by: java.io.FileNotFoundException: config/couchdb.json (No such file or directory)
at java.io.FileInputStream.open0(Native Method) ~[?:?]
at java.io.FileInputStream.open(FileInputStream.java:219) ~[?:?]
at java.io.FileInputStream.
Caused by: java.io.FileNotFoundException: config/couchdb.json
This is likely caused by there being a reference to couchdb.json in your config/base.json
file. If you try editing config/base.json
, there should be something like this:
{
"type": "rover",
"source": "default",
"fragment": 2,
"unique": 30,
"databases": [
"config/couchdb.json",
"config/influxdb.json"
],
"io": "config/default_linux_serial.json",
"modbus": 1
}
If I'm correct, then all you need to do is remove "config/couchdb.json",
and it won't go hunting for a couchdb config. If I'm not correct, then please post your base.json config file.
The final base.json config should look like this:
{
"type": "rover",
"source": "default",
"fragment": 2,
"unique": 30,
"databases": [
"config/influxdb.json"
],
"io": "config/default_linux_serial.json",
"modbus": 1,
"bulk_request": true
}
I added bulk_request to be true because I've found it makes querying the rover take a lot less time. It will likely become the default option in a future release.
Also, solarthing doesn't have to be run with sudo if you run these commands. (This is optional. You can just use sudo for now until you get everything working)
sudo other/linux/create_user.sh # create solarthing user and group
sudo other/linux/update_perms.sh # update perms so solarthing group can edit
sudo usermod -a -G solarthing,dialout,tty,video $USER # add yourself to a few groups
SolarThing should automatically create an InfluxDB database, so you shouldn't have to configure influxdb much.
apparently I did not save base.json after I made edits. I'm getting so many permissions errors. I'm not able to save that file or even the properties governing permissions. Owner is root, and group is solarthing. I will have to research how to use sudo chmod.
If you use sudo for everything (like when you edit files), you shouldn't get any permission errors. Also by running the commands I put above, logging out and back in, you shouldn't have to use sudo at all. I edited my response to show that you should use sudo when running those commands. You could give those commands a try if you want. Once you run those commands, you shouldn't have to use sudo.
Ok fixed that. Now my base.json looks correct. Running it again.
also ran these sudo other/linux/create_user.sh # create solarthing user and group sudo other/linux/update_perms.sh # update perms so solarthing group can edit sudo usermod -a -G solarthing,dialout,tty,video $USER # add yourself to a few groups
Humm..Well, this is better. Time to hook up the rover and see what happens!
pi@raspberrypi:/opt/solarthing/program/rover $ sudo ./run.sh
Using base config = config/base.json
2020-05-29 16:57:33.008 [main] INFO me.retrodaredevil.solarthing.program.SolarMain - [LOG] Beginning main
[stdout] Beginning main
2020-05-29 16:57:33.564 [main] INFO me.retrodaredevil.solarthing.program.RoverMain - Beginning rover program
2020-05-29 16:57:33.825 [main] INFO me.retrodaredevil.solarthing.analytics.AnalyticsManager - Google Analytics is ENABLED!
2020-05-29 16:57:33.848 [main] DEBUG me.retrodaredevil.solarthing.analytics.AnalyticsManager - Using Analytics UUID: 1bedd713-32b5-4e06-b63c-4722d4326608
2020-05-29 16:57:35.376 [main] INFO me.retrodaredevil.solarthing.analytics.AnalyticsManager - Sending program type to Google Analytics
2020-05-29 16:57:35.593 [main] ERROR me.retrodaredevil.solarthing.program.RoverMain - (Fatal)Got exception!
me.retrodaredevil.io.serial.SerialPortException: Was unsuccessful while trying to open port: ttyUSB0 descriptive name: User-Specified Port description: User-Specified Port
at me.retrodaredevil.io.serial.JSerialIOBundle.
I think that serial error will go away when something is connected, right?
I think that serial error will go away when something is connected, right?
Yes exactly. If you don't have anything plugged in, that error is expected to happen. Once you plug your USB to serial thing in, it usually shows up as /dev/ttyUSB0, which is what it tried connecting to.
If you plug it in and still get this error that means it's not at /dev/ttyUSB0 or you don't have permission (which you should since you ran the necessary commands). So if you need to, you can run ls /dev/ttyUSB*
to figure out what it comes up as. In the rare case that it doesn't come up as /dev/ttyUSBX, then you can take a look at this: https://askubuntu.com/a/408831
It's working, hooray! Now this is hilarious! "packetType" : "RENOGY_ROVER_STATUS" That is coming from a so-called Grape Solar charge controller. LOL
Oh, just realized that is probably an alias you wrote into the program file.
I'm glad you got it working. Yeah, that I made that constant a while back when I didn't know other charge controllers used this same protocol. That's how it'll be referred to for the end of time. However, when displaying stuff in places like the Android application (that needs CouchDB), I'm trying to make it display the correct name of the charge controller.
The packet type is used internally in many places in SolarThing, so you can just ignore it lol. Let me know if you need help with anything else.
Next step is to make it log to a remote installation of InfluxDB rather than a local one. This is probably possible by editing the influxdb config file, right?
Yep exactly. In the config of influxdb.json, you would change the url
field to something besides http://localhost:8086
such as http://192.168.1.100:8086
.
Cool, thanks so much! This is so awesome!
Well, it seems to run just fine with ./run.sh but the service is shown as failed even after a manual start. . Could this be a permissions problem? Also, I have installed Grafana, but I'm it seems that solarthing sets a default retention policy of 1h which I would like to change to something like a few days. I'm not sure where the config options are for that. I'm a total noob when it comes to setting up queries. Someone posted a feature request a while ago for a Grafana dashboard.. Is that available somewhere for a plug and play way to get started with Solarthing and Grafana?
The Grafana dashboard is not plug and play, you'll probably have to mess with it a bit. This dashboard is designed for Outback devices, but also uses Rover packets in a few of the queries: https://github.com/wildmountainfarms/solarthing/blob/master/other/grafana/dashboards/solar_influxdb_basic.json
It looks like that's the solarthing service from the previous installation. It doesn't work with the new setup so you'll have to follow setup instructions here: https://github.com/wildmountainfarms/solarthing/tree/master/other/systemd Basically just run this:
sudo other/systemd/install.sh rover
That will install the solarthing-rover
service.
If you wanted to, you could completely remove the old solarthing
service:
sudo systemctl disable solarthing
sudo rm /etc/systemd/system/solarthing
solarthing sets a default retention policy of 1h which I would like to change to something like a few days
That's not exactly correct. SolarThing uses 3 different retention policies (with the default setup anyway) 29 times per hour it uses a policy that keeps it for 4 weeks, 7 times per hour it keeps it forever, and otherwise it keeps the rest that are uploaded for one hour.
The retention policies solarthing uses are a little weird. Anyway, you can edit your config/influxdb.json
file to change retention policies. The default file: https://github.com/wildmountainfarms/solarthing/blob/009c2f5ec5339c75e605624db16a1d0e0e2f6bf4/config_templates/databases/influxdb_template.json can be changed so the retention policies are something like this:
...
"status_retention_policies": [
{
"frequency": null,
"name": "autogen",
"set_as_default": true
}
],
...
That will use InfluxDB's default retention policy and it'll make sure it's set as default. Now if you want to use a more advanced configuration (you probably don't), you'll notice that each retention policy has a "frequency" field. This doesn't have anything to do with InfluxDB, but basically it means that it will use that retention policy X many times per hour. If it's null, then it will use it infinitely many times per hour, which is fine, but I don't know how much space that will take up on InfluxDB if you also have the retention set to INF
.
I know the default retention policies aren't the best thing in the world. I think I might change it to just use autogen later.
Yay, that worked! I see where I went wrong. I was trying to start the old service. This should have been something like
systemctl start solarthing-<mate|rover|graphql|pvoutput>
systemctl enable solarthing-<mate|rover|graphql|pvoutput>
I saw your examples for Grafana queries. I'll start with those and some Grafana tutorials. I noticed there are some recognizable terms in the picklists there so there must be a table being accessed.
Just to let you know, I updated the default influxdb config: https://github.com/wildmountainfarms/solarthing/blob/master/config_templates/databases/influxdb_template.json
That would just use autogen as the retention policy.
Also, for a pretty simple display of the battery voltage, this is what I use:
SELECT "batteryVoltage" FROM "medium_duration_retention"."solarthing" WHERE $timeFilter GROUP BY "identifier"
(You can switch into raw query mode to paste this in in Grafana) You'll notice that I explicity state the retention policy I'm getting data from. I don't remember what happens if you don't explicitly say what retention policy you want. I think it should go with your default, but I'm not sure.
Woo Hoo! Ok, it's a work in progress.
Hello, and thanks for sharing Solarthing. I had been trying to unravel the serial connection mystery of how to connect my Grape Solar charge controller to the internet since 2016 and now I have found your great project. By they way, the Renogy Rover and the GS Zenith series are made by the same Chinese company surprise, surprise! Don't ask me how I know that, but I do. It's not too hard to figure out by looking at them and the documentation. Anyway, I had a little trouble installing the service on my RPi 3b and a few other errors worth mentioning during setup.
pi@raspberrypi:/opt/solarthing $ sudo systemd/install.sh sudo: systemd/install.sh: command not found
It's a small thing, but it was necessary to "cd other" to get to those files. pi@raspberrypi:/opt/solarthing $ cd other pi@raspberrypi:/opt/solarthing/other $ sudo systemd/install.sh pi@raspberrypi:/opt/solarthing/other $
Success! Next it was necessary to copy config template files, but I got "permission denied" so I prefaced all my cp commands with sudo, and it worked. I guess I can download your files but they aren't really mine. I used the rover_dummy_setup_template file. Then I had to install java and ran "java -jar solarthing.jar" I'm not sure if what happened next is by design or what. I have attached a copy of the output. solarthingoutput.txt