PostgreSQL service module

[blacklab128]

Module sends (uploads) weather data to a PostgreSQL database. Can be adapted for other databases easily.

Signed-off-by: Mike mike@homenet.kits.local

[jim-easterbrook]

Can you update the documentation so potential users will become aware of this module? And explain why you'd use it instead of an alternative data storage backend such as pywws.sqlite3data.

[blacklab128]

Hi, thanks for the reply.

Well, I have written the script for a earlier version of pywws and just adapted it to run with the current one. I haven't realized there is already a sqlite module in the current version. Just thought maybe this is useful for other users of pywws who also want to store the data in a PostgreSQL database as I do (more quick and dirty than well planned).

I will have a look at it after work -- maybe it is easier to extend the existing sqlite module to be able to use PostgreSQL and MySQL instead of writing a whole new service module (or it already does?). Also I will have a look at pywws.conversions.

Best regards, Mike

On Monday, January 14, 2019 09:07 CET, Jim Easterbrook notifications@github.com wrote:

jim-easterbrook commented on this pull request.

+#illuminance "'solarradiation': '%.2f'," "" "illuminance_wm2(x)"# +#uv "'UV' : '%d',"# +""" +

• @contextmanager
• def session(self):
• yield None
• def fahrenheit2celsius(self, fahrenheit):
• """Convert fahrenheit to celsius"""
• return (float(fahrenheit) - 32) * 5.0 / 9.0
• def mph2kmh(self, mph):
• """Convert mph to km/h"""
• return float(mph) / 0.62137119

Why not use the function from pywws.conversions?

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/jim-easterbrook/pywws/pull/78#pullrequestreview-192041648

[blacklab128]

Hi Jim,

after reading the code I see many things I wrote are already existing. So I am going to write an alternative storage backend for PostgreSQL -- instead of a service module.

My first intention with the service module was to store only the (preconverted) data I need inside the PostgreSQL database. It might be a better idea to use the PostgreSQL backend and export the pywws.conversions as PostgreSQL functions. I think this might give good flexibility as users can directly get any kind of raw/converted/interpreted value from the database with whatever program/script/language they are querying the database.

Something like: SELECT temp_f(temp_in) FROM raw;

If you think this is a good idea too, I would start with that work. The current pull request could be closed/rejected.

Best regards, Mike

On Mon, 2019-01-14 at 00:10 -0800, Jim Easterbrook wrote:

Can you update the documentation so potential users will become aware of this module? And explain why you'd use it instead of an alternative data storage backend such as pywws.sqlite3data.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c5 5493e4bb","name":"GitHub"},"entity":{"external_key":"github/jim- easterbrook/pywws","title":"jim-easterbrook/pywws","subtitle":"GitHub repository","main_image_url":" https://github.githubassets.com/images/email/message_cards/header.png","avatar_image_url":"https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":" https://github.com/jim-easterbrook/pywws"}},"updates":{"snippets":[{"icon":"PERSON","message":"@jim-easterbrook in #78: Can you update the documentation so potential users will become aware of this module? And explain why you'd use it instead of an alternative data storage backend such as pywws.sqlite3data."}],"action":{"name":"View Pull Request","url":" https://github.com/jim-easterbrook/pywws/pull/78#issuecomment-453925816 "}}} [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": " https://github.com/jim-easterbrook/pywws/pull/78#issuecomment-453925816 ", "url": " https://github.com/jim-easterbrook/pywws/pull/78#issuecomment-453925816 ", "name": "View Pull Request" }, "description": "View this Pull Request on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[jim-easterbrook]

I have no personal interest in (or knowledge of) database stuff, so do what you think will be useful. The SQLite3 database alternative storage was submitted by Mark Jarvis (@jarvisms) so you might want to discuss your ideas with him.

[jarvisms]

Hi Mike, I think my SQLite3 module can be easily adapted to support PostgreSQL (or any other SQL platform for that matter). But the underlying database is still just a load of "dumb-tables" with no calculated values, but I did originally want to see the workload of conversions and the various summaries off-loaded into SQL which I think is what you want to do too. I think it's actually a little more challenging than it first appears if full compatibility is to be retained with the rest of pywws but I'd be interested in contributing if it can be done with both SQLite3 and PostgreSQL. Mark

[blacklab128]

Hi Mark, I have a first version of the database structure but I still need to write the PgSQL module.When everything goes well I might have a first working and tested version until Saturday.

Best regards,Mike On Tue, 2019-01-15 at 00:23 -0800, Mark Jarvis wrote:

Hi Mike,

I think my SQLite3 module can be easily adapted to support PostgreSQL (or any other SQL platform for that matter). But the underlying database is still just a load of "dumb-tables" with no calculated values, but I did originally want to see the workload of conversions and the various summaries off-loaded into SQL which I think is what you want to do too. I think it's actually a little more challenging than it first appears if full compatibility is to be retained with the rest of pywws but I'd be interested in contributing if it can be done with both SQLite3 and PostgreSQL.

Mark

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c5 5493e4bb","name":"GitHub"},"entity":{"external_key":"github/jim- easterbrook/pywws","title":"jim-easterbrook/pywws","subtitle":"GitHub repository","main_image_url":" https://github.githubassets.com/images/email/message_cards/header.png","avatar_image_url":"https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":" https://github.com/jim-easterbrook/pywws"}},"updates":{"snippets":[{"icon":"PERSON","message":"@jarvisms in #78: Hi Mike,\r\nI think my SQLite3 module can be easily adapted to support PostgreSQL (or any other SQL platform for that matter). But the underlying database is still just a load of \"dumb-tables\" with no calculated values, but I did originally want to see the workload of conversions and the various summaries off-loaded into SQL which I think is what you want to do too. I think it's actually a little more challenging than it first appears if full compatibility is to be retained with the rest of pywws but I'd be interested in contributing if it can be done with both SQLite3 and PostgreSQL.\r\nMark"}],"action":{"name":"View Pull Request","url":" https://github.com/jim-easterbrook/pywws/pull/78#issuecomment-454304993 "}}} [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": " https://github.com/jim-easterbrook/pywws/pull/78#issuecomment-454304993 ", "url": " https://github.com/jim-easterbrook/pywws/pull/78#issuecomment-454304993 ", "name": "View Pull Request" }, "description": "View this Pull Request on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]