search

Ingenjorsarbete-For-Klimatet / ifk-smhi

SMHI climate data client.
https://ingenjorsarbeteforklimatet.se/ifk-smhi/
MIT License
8 stars 1 forks source link

Version change Mesan #98

Closed docNord closed 3 months ago

docNord commented 4 months ago

Closes #97

Main contributions:

docNord commented 4 months ago

https://opendata-download-metanalys.smhi.se/api/category/mesan2g/version/1/geotype/point/lon/16/lat/57/data.json

This works. To me, it seems this is the link which is used in the code. Will now try to find where the fault comes.

docNord commented 4 months ago 
from smhi import mesan
client = mesan.Mesan()
client.get_point(57,16)

Works. Thus, the basic levels of the API are now working with the naïve version update I introduced. I will now go through the failing tests to try to piece together what is wrong and why.

docNord commented 3 months ago

Two strange errors I'm getting @mgcth : 1) Metobs integration test is failing even though I haven't touched those files 2) I have made only minor changes, but lint is failing under the headline "format with ruff" for almost all files in the library

docNord commented 3 months ago

This is what it says on my computer, regarding the ruff problem:

>>ruff format --check src tests
15 files already formatted
mgcth commented 3 months ago
  1. Metobs integration test is failing even though I haven't touched those files

This is probably relevant here.

docNord commented 3 months ago

Ok, will look into it when I get back from Vasaloppet 😊

docNord commented 3 months ago

The problem in metobs integration seems to be that dates are mismatching. The way dates for reference are parsed is somewhat wishwash - it seems there is an assumption "corrected-archive" will always measure up until a set date ahead of today, which no longer seems to be the case. I will ponder a solution to the problem.

docNord commented 3 months ago

Will use direct webb call to get the data from the corrected archive and use the actual date from the actual data to parse in the test, something like: data = pd.read_csv("https://opendata-download-metobs.smhi.se/api/version/latest/parameter/1/station/188800/period/corrected-archive/data.csv",skiprows=9, usecols=["Datum","Tid (UTC)"],sep=";")

docNord commented 3 months ago

My updated code works for test1 however the problem remains even there. This is due to the fact that the latest date of the actual data in this file is indeed different from what i get from the ifk-smhi code.

docNord commented 3 months ago

...and that is because I'm not reading the header but the latest data point.

docNord commented 3 months ago

Alright @mgcth, I've simplified and fixed also the Metobs integration tests now, but I am unable to locally reproduce, or even find a meaningful answer as to what ruff finds appalling with the formatting of 14 out of 15 files, so I'm not getting any further with solving the lint issues. Care to weigh in?

mgcth commented 3 months ago

Alright @mgcth, I've simplified and fixed also the Metobs integration tests now, but I am unable to locally reproduce, or even find a meaningful answer as to what ruff finds appalling with the formatting of 14 out of 15 files, so I'm not getting any further with solving the lint issues. Care to weigh in?

Locally, you probably have a different ruff version. I had 0.1.* which I expect because we use compatible releases (https://peps.python.org/pep-0440/#compatible-release) in our pyproject.toml file. If you look at the build for lint in this PR you will see it installs 0.3.2 (https://github.com/Ingenjorsarbete-For-Klimatet/ifk-smhi/actions/runs/8219939531/job/22478471060#step:4:39). I don't know why. It does so locally for me when I recreate the env, too.

Looks like the lint run in main respected the compatible release version: https://github.com/Ingenjorsarbete-For-Klimatet/ifk-smhi/actions/runs/7680658453/job/20932961862#step:4:41

The run on main uses pip 23.0.1 while here it is using 24.0. However, even with 23.0.1 locally, I can't pin ruff to 0.1.*. Maybe it's related to setuptools too?

docNord commented 3 months ago

I changed my local ruff to 0.1.* to match our pyproject. Will change again to match what is running then. Could you have a look at this and try to have github actions run with the defined versions of our pyproject file @mgcth ?

mgcth commented 3 months ago

I changed my local ruff to 0.1.* to match our pyproject. Will change again to match what is running then. Could you have a look at this and try to have github actions run with the defined versions of our pyproject file @mgcth ?

As you can tell from my comment above, I don't know why this is happening. ~= isn't respected any more.