winds-mobi / winds-mobi-providers

Real-time weather observations: weather data providers
https://winds.mobi
GNU Affero General Public License v3.0
8 stars 12 forks source link

feat: pmcjoder provider & neuhuus weather station #90

Closed MichalBryxi closed 3 months ago

MichalBryxi commented 3 months ago

Fixes: #89

MichalBryxi commented 3 months ago

Note that I'm not 100% sure what I'm doing, but running the provider locally seems to be passing:

providers-1  | 2024-06-15 08:31:06+0000 INFO [neuhuus] | Processing MyProvider data...
providers-1  | 1718447160
providers-1  | 2024-06-15 08:31:06+0000 INFO [neuhuus] | ⏱ 24-06-15 12:26:00+02:00 (24-06-15 10:26:00+00:00) 'neuhuus'/'Segelclub Neuhaus-Interlaken' (neuhuus-segelclub-neuhaus-interlaken): 1 values inserted
providers-1  | 2024-06-15 08:31:06+0000 INFO [neuhuus] | ...Done !

And seems to be writing data:

mongodb-1  | {"t":{"$date":"2024-06-15T08:36:59.459+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"WTCheckpointThread","msg":"WiredTiger message","attr":{"message":"[1718440619:459565][1:0x7f0ffc070700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 571, snapshot max: 571 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 25"}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.212+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.18.0.4:36780","connectionId":82,"connectionCount":1}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.212+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn82","msg":"client metadata","attr":{"remote":"172.18.0.4:36780","client":"conn82","doc":{"driver":{"name":"PyMongo","version":"4.2.0"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"6.6.22-linuxkit"},"platform":"CPython 3.10.11.final.0"}}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.214+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.18.0.4:36794","connectionId":83,"connectionCount":2}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.214+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.18.0.4:36796","connectionId":84,"connectionCount":3}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.214+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn83","msg":"client metadata","attr":{"remote":"172.18.0.4:36794","client":"conn83","doc":{"driver":{"name":"PyMongo","version":"4.2.0"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"6.6.22-linuxkit"},"platform":"CPython 3.10.11.final.0"}}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.214+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn84","msg":"client metadata","attr":{"remote":"172.18.0.4:36796","client":"conn84","doc":{"driver":{"name":"PyMongo","version":"4.2.0"},"os":{"type":"Linux","name":"Linux","architecture":"x86_64","version":"6.6.22-linuxkit"},"platform":"CPython 3.10.11.final.0"}}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.910+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"conn84","msg":"createCollection","attr":{"namespace":"winds.pmcjoder-segelclub-neuhaus-interlaken","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"3c33f445-708d-45ab-a26f-c19bd0ca3dbb"}},"options":{"capped":true,"size":500224,"max":5000}}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.922+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"conn84","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"winds.pmcjoder-segelclub-neuhaus-interlaken","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:02.930+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn84","msg":"Connection ended","attr":{"remote":"172.18.0.4:36796","connectionId":84,"connectionCount":2}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:03.216+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn82","msg":"Interrupted operation as its client disconnected","attr":{"opId":178332}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:03.217+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn82","msg":"Connection ended","attr":{"remote":"172.18.0.4:36780","connectionId":82,"connectionCount":1}}
mongodb-1  | {"t":{"$date":"2024-06-15T08:37:03.217+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn83","msg":"Connection ended","attr":{"remote":"172.18.0.4:36794","connectionId":83,"connectionCount":0}}

From the README I can't tell how can I verify that the correct data has been correctly written into the DB. I just printed everything in the provider and manually inspected before saving. Is there an easy way to just look at what my local data shows now?

MichalBryxi commented 3 months ago

I did not manage to make the JS client work on my localhost (requires python2 for some reason), but looking directly into the mongodb seems to show good results:

Screenshot 2024-06-15 at 11 43 01 Screenshot 2024-06-15 at 11 38 19

MichalBryxi commented 3 months ago

Not sure what is the process in this project, so humble ping @ysavary

ysavary commented 3 months ago

Hi @MichalBryxi,

Thank you for your PR 👍 I pushed some fixes to your branch. Build is now passing. Please review my changes and we can merge.

did not manage to make the JS client work on my localhost (requires python2 for some reason)

Weird 🤔 Additionally, to run the js client locally, you need to deploy the API service and point the js client to it!

MichalBryxi commented 3 months ago

Yup, makes sense to me:

👍🏾 from me

ysavary commented 3 months ago

@MichalBryxi Thank you for this new provider 👏

Your code is deployed: https://winds.mobi/stations/list/pmcjoder-segelclub-neuhaus-interlaken It would be cool if you could verify that everything is correct!

Unfortunately, the monitoring tool is use, https://sentry.io, is only allowing one user with the free plan. If an issue occurs, I will link the sentry issue with the GitHub issue.

MichalBryxi commented 3 months ago

Obviously I bumped into a facepalm-y issue right after it has been deployed :) PR already exists.