khaeru / sdmx

SDMX information model and client in Python
https://sdmx1.readthedocs.io
Apache License 2.0
24 stars 18 forks source link

Add support for data availability queries #161

Closed FedorYatsenko closed 7 months ago

FedorYatsenko commented 8 months ago

Add support for data availability queries (availableconstraint) defined by the SDMX 2.1 standard

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (69aa84a) 98.50% compared to head (ebe2953) 96.45%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #161 +/- ## ========================================== - Coverage 98.50% 96.45% -2.06% ========================================== Files 90 90 Lines 7110 7128 +18 ========================================== - Hits 7004 6875 -129 - Misses 106 253 +147 ``` | [Files](https://app.codecov.io/gh/khaeru/sdmx/pull/161?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Paul+Natsuo+Kishimoto) | Coverage Δ | | |---|---|---| | [sdmx/client.py](https://app.codecov.io/gh/khaeru/sdmx/pull/161?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Paul+Natsuo+Kishimoto#diff-c2RteC9jbGllbnQucHk=) | `92.97% <100.00%> (-4.87%)` | :arrow_down: | | [sdmx/rest.py](https://app.codecov.io/gh/khaeru/sdmx/pull/161?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Paul+Natsuo+Kishimoto#diff-c2RteC9yZXN0LnB5) | `100.00% <100.00%> (ø)` | | | [sdmx/source/\_\_init\_\_.py](https://app.codecov.io/gh/khaeru/sdmx/pull/161?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Paul+Natsuo+Kishimoto#diff-c2RteC9zb3VyY2UvX19pbml0X18ucHk=) | `100.00% <ø> (ø)` | | | [sdmx/testing/\_\_init\_\_.py](https://app.codecov.io/gh/khaeru/sdmx/pull/161?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Paul+Natsuo+Kishimoto#diff-c2RteC90ZXN0aW5nL19faW5pdF9fLnB5) | `99.28% <ø> (ø)` | | | [sdmx/tests/reader/test\_reader\_xml\_v21.py](https://app.codecov.io/gh/khaeru/sdmx/pull/161?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Paul+Natsuo+Kishimoto#diff-c2RteC90ZXN0cy9yZWFkZXIvdGVzdF9yZWFkZXJfeG1sX3YyMS5weQ==) | `100.00% <100.00%> (ø)` | | ... and [18 files with indirect coverage changes](https://app.codecov.io/gh/khaeru/sdmx/pull/161/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Paul+Natsuo+Kishimoto)
khaeru commented 8 months ago

Hi @FedorYatsenko —thanks for this contribution, much appreciated!

A couple of questions:

These would help me confirm the code works, and guide you in adding tests. Thanks in advance.

FedorYatsenko commented 7 months ago

Hi @khaeru,

I'm afraid, I can't yet share a link to SDMX web services that will give responses to such queries. As @sychsergiy mentioned in the khaeru/sdmx#164, our data provider is hidden by a private network.

Consequently, I'm not sure if this help you, but here is the code we are using:

keys_dict = {
    'INDICATOR': ['PCPIHA_IX', 'PCPIHA_PC_CP_A_PT', 'PCPI_PC_CP_A_PT'],
    'COUNTRY':   ['111', '134'],
    'FREQUENCY': []
}
resp2 = sdmx_client.availableconstraint(
    'CPI',
    key=keys_dict,
    dsd=cpi_dsd
)

It generates the following link: .../availableconstraint/CPI/111+134.PCPIHA_IX+PCPIHA_PC_CP_A_PT+PCPI_PC_CP_A_PT.*. And the following XML response: khaeru/sdmx-test-data#1

I am going to push a test for this file in this branch. Please let me know if I should change something.

khaeru commented 7 months ago

Thanks for adding the specimen in the other repo. On the next push the tests on this branch should find it.

I need to at some point add a PR template with a check-list for contributors. For now, while you're working on this draft, could you please:

  1. Add a line to doc/whatsnew.rst, following the style of the existing entries.
  2. Rebase the branch on the latest main, resolving any conflicts (don't need to do this repeatedly, only when ready to review/merge).

I'll come back to review when you indicate I should.

FedorYatsenko commented 7 months ago

Thank you for merging the specimen.

As for the whatsnew and updates to the latest main, I'll do that soon.

FedorYatsenko commented 7 months ago

Thank you very much for approving these changes.

khaeru commented 7 months ago

By the way: I see on the commit 31a7ba12b1f3f32df02d64cdbe4dba04e8c64687 that the associated username is "Fedir_Yatsenko", whereas your GitHub username appears to be "FedorYatsenko". I don't know why there is a mismatch (maybe difference between your local configuration and GitHub account), but as a result GitHub does not correctly associate the commit with your account, and you don't show up yet in the list of contributors: https://github.com/khaeru/sdmx/graphs/contributors 🥲

I'm sorry for not noticing this earlier. I'll add a :thanks: to doc/whatsnew.rst when I prepare the next release.

FedorYatsenko commented 7 months ago

No worries. I just misspelled my name a little when creating my GitHub account.