Can't access influxdb database

[flammagenitus]

Is there any reason to think that solarthing is incompatible with InfluxDB OSS 2.0? It just came out about 10 days ago. I didn't have solarthing working with InfluxDB 1.x either so I don't know if that's the problem.

I followed these instructions to set up InfluxDB: https://docs.influxdata.com/influxdb/v2.0/get-started/ The server is running because I can access http://localhost:8086 and can log in. I used the InfluxDB configuration wizard to create a new user, password, organization, and bucket. I copied the user name and password into influxdb.json.

When I run setup.sh I see data from the Renogy plus this error:

2020-12-02 11:07:40.115 [main] INFO  org.influxdb - --> POST http://localhost:8086/query?q=CREATE+DATABASE+default_database (0-byte body)
2020-12-02 11:07:40.166 [main] INFO  org.influxdb - <-- 401 Unauthorized http://localhost:8086/query?q=CREATE+DATABASE+default_database (50ms, 48-byte body)
2020-12-02 11:07:40.175 [main] ERROR me.retrodaredevil.solarthing.program.PacketListReceiverHandler - Couldn't packet collection id: 2020,12,02,11,(04/30),[e6f29dcc] dateMillis: 1606928859542
me.retrodaredevil.solarthing.packets.handling.PacketHandleException: Unable to query the database!

Here is my base.json:

{
  "analytics_enabled": false,
  "type": "rover",
  "source": "default",
  "fragment": 2,
  "unique": 30,
  "databases": [
    "config/influxdb.json"
  ],
  "io": "config/default_linux_serial.json",
  "modbus": 1
}

Here is my influxdb.json:

{
  "type": "influxdb",
  "config": {
    "url": "http://localhost:8086",
    "username": "<redacted>",
    "password": "<redacted>",
    "database": "default_database",
    "measurement": null,

    "status_retention_policies": [
      {
        "frequency": 120,
        "name": "autogen"
      }
    ],

    "event_retention_policy": {
      "name": "autogen"
    }
  }
}

Any ideas on why solarthing can't access my InfluxDB database?

[retrodaredevil]

It looks like InfluxDB 2.0 isn't compatible with InfluxDB 1.x. I will have to look into implementing https://github.com/influxdata/influxdb-client-java, which shouldn't be too difficult (I hope).

[flammagenitus]

OK. I'm going to switch back to InfluxDB 1.x in the meantime and see if that works. Maybe some security setting specific to my particular Ubuntu setup is causing the problem.

[retrodaredevil]

Yeah that's possible. I'm not at all familiar with InfluxDB 2.0 and it looks like there have been a lot of changes so I can't say for sure how easy it will be to support it. I'm also not using InfluxDB right now so it's not a high priority for me.

[flammagenitus]

I've got solarthing working nicely via an old Raspberry Pi that connects to InfluxDB 1.7 running on a different server. My database experience is only with sqlite and mysql; I didn't know anything about CouchDB or InfluxDB but I'm learning. Hopefully InfluxDB 1.7 is around for a long time. No worries about InfluxDB 2.0 and thanks again for sharing solarthing.

[retrodaredevil]

I'm glad to hear you got it working. Let me know if you have any more issues or suggestions.

Currently InfluxDB is definitely the easier one to set up, but I might end up trying to make some more documentation for using Grafana+GraphQL+CouchDB since that's what I'm currently using and because it's more customizable. I don't think anyone besides me is actually using CouchDB as the database for SolarThing right now.

I think InfluxDB 1.X will be around for a while. I imagine there are plenty of people that won't be upgrading any time soon.

[retrodaredevil]

I just got InfluxDB 2.0 support working. You'll have to update your config file, and you can't migrate from your 1.X database, so it's up to you if you want to upgrade and start over with new data.

I also don't have example queries for InfluxDB 2.0 yet.

[flammagenitus]

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

Good deal! I recently updated solarthing so I noticed that you had changed some files related to databases. I am also migrating between computers and managed to screw up my Thunderbird profile so I didn't see this email for a while. InfluxDB 1.x has been working great so I hate to change anything right now. Sooner or later I will install a "real" solar power system and then I'll make the jump to 2.x. Good to know it's available. Dave

On 1/31/21 6:35 PM, Joshua Shannon wrote:

I just got InfluxDB 2.0 support working. You'll have to update your config file, and you can't migrate from your 1.X database, so it's up to you if you want to upgrade and start over with new data.

I also don't have example queries for InfluxDB 2.0 yet.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wildmountainfarms/solarthing/issues/12#issuecomment-770484429, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR2JW6DDASW5MSFLZQ6EOY3S4XZNNANCNFSM4UKZUP3Q.

-----BEGIN PGP SIGNATURE----- Version: ProtonMail

wsBmBAEBCAAQBQJgGvS4CRA78k/NMG4+tQAKCRA78k/NMG4+tTgJB/49jQDE HEe0DCbNZaIwY/f0jQi49mkmCkO9Rm21DKvgDX/QEMBBnOrg90up3/2nEgLR luj/cfdo90ui7e9Y+jUHsickNUQHwj0mhlzQe7L8YLDC0KiCONB3mfQzumEN TEQZiCGkT7LoD8ICStcjxOTvL8XDXsne30vk3VDJL06kE7CAJIPHZTqMvO6u ZBtw3GUtmYmOOq7KuUo3+sJvGAUZDt15rSdg29SidMNTi8Xj+me9/NB0ODTN f0mIUqjX0DgQrxh5Wz1h092+ofA3Ndz2qliWZZ4B8b/FU3+yjyc8n9Ih8CC0 iGGcpcK11wZ1RMgR78SR7KGFyqkuoZd7Ll1B =4UYO -----END PGP SIGNATURE-----

[flammagenitus]

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

Sorry if this is documented somewhere obvious but I haven't seen it yet. Is there an easy way to decrease the solarthing reporting frequency? It seems to be logging data once every few seconds which would be fine but my poor rpi that runs influxdb (along with other services) is also getting bombarded with collectd data and it's overloaded. influxdb seems to be the main culprit. I don't really need such high precision data from my PV system.

Thanks, Dave

On 2/3/21 1:08 PM, Dave Schmidt wrote:

Good deal! I recently updated solarthing so I noticed that you had changed some files related to databases. I am also migrating between computers and managed to screw up my Thunderbird profile so I didn't see this email for a while. InfluxDB 1.x has been working great so I hate to change anything right now. Sooner or later I will install a "real" solar power system and then I'll make the jump to 2.x. Good to know it's available. Dave

On 1/31/21 6:35 PM, Joshua Shannon wrote:

I just got InfluxDB 2.0 support working. You'll have to update your config file, and you can't migrate from your 1.X database, so it's up to you if you want to upgrade and start over with new data.

I also don't have example queries for InfluxDB 2.0 yet.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wildmountainfarms/solarthing/issues/12#issuecomment-770484429, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR2JW6DDASW5MSFLZQ6EOY3S4XZNNANCNFSM4UKZUP3Q.

-----BEGIN PGP SIGNATURE----- Version: ProtonMail

wsCUBAEBCAAnBQJgQW03CRA78k/NMG4+tRYhBKpso4s8xQJc7OEqljvyT80w bj61ACEJEDvyT80wbj61FiEEqmyjizzFAlzs4SqWO/JPzTBuPrXL1Qf+MdPg Ah97ZSYfMIAP8U+vNnuaU5gVO4iC2w5hnCfAeVsoYjnPPt01T/4pHen9O/UH hZxuQ1HWjN2nVaszvnEX+wj+dUfAwyS8u4Xsj0HtD1nJ40zcvhofzM4bHFRS Kueh8HcxNy20l7uLxDHNwVGXLYuMN8TwPyBYt1HrdqTwnj6B3xnEaL0M/kOc Jm7t5w2B1Yt/TJAymxd5XLGLFa7GWk2//2oFQUtcLGAkUaapXxUCBF5SU1VW WCdo0T/89omJRyoacwVMGR32WSWzNYQ1ovzM4N63/LDr7alnG9mQXKDTZRmo djXE7019XZOy7CbDNGClMElTtZQleuDFzh2RNw== =IU6D -----END PGP SIGNATURE-----

[retrodaredevil]

If you're using InfluxDB 1.X, there some advanced configuration to use different retention policies and stuff, but I don't recommend that since it makes the queries more complex. Using CouchDB doesn't have this problem because data is replaced in CouchDB except for a single datapoint that is kept (usually 1 per minute or 2 per every 2 minutes), so the data doesn't take up too much space. Another option is to make the retention policy you're using delete data after a certain amount of time, which you may or may not want. If you want to do that, you can Google how to do that, as that's something you can configure in InfluxDB. (Technically it can be configured through SolarThing for InfluxDB 1.X, but it's been a while since I've looked at that code)

Now to actually decrease the reporting frequency, you just change the throttle factor (see CouchDB example), or what I recommend: changing the query period in base.json itself.

You can just add a period field to base.json and specify how often in seconds you want data to be queried. Example:

{
  ...,
  "period": 10
}

Also I should probably put this somewhere in the documentation, but I don't recommend that the database runs on the Pi, even though that's what most people who set up SolarThing do. On the system I monitor, the CouchDB database is taking up 2.2GB of space for 1.5 years worth of data. (Also keep in mind that I have multiple charge controller's data in here)