sckott / habanero

client for Crossref search API
https://habanero.readthedocs.io
MIT License
195 stars 30 forks source link

Library returns wrong values #159

Closed wisentini closed 5 months ago

wisentini commented 8 months ago

Environment

Python   3.8.18
habanero 1.2.3

Problem

When I make a request to https://api.crossref.org/journals/2448-1904 via my web browser, the subjects field is:

"subjects": [
    {
        "ASJC": 2505,
        "name": "Materials Chemistry"
    },
    {
        "ASJC": 2002,
        "name": "Economics and Econometrics"
    },
    {
        "ASJC": 2214,
        "name": "Media Technology"
    },
    {
        "ASJC": 1107,
        "name": "Forestry"
    }
]

But when I make the same request via code:

crossref = Crossref(
    mailto=config['crossref']['mailto'],
    ua_string=config['crossref']['user_agent']
)

response = crossref.journals(ids='2448-1904')

print(json.dumps(response['message']['subjects'], indent=4))

The result is:

[
    {
        "ASJC": 2700,
        "name": "General Medicine"
    }
]

Why is that?

sckott commented 8 months ago

Thanks for the issue. I was able to replicate this in python3.8 and 3.11. However, it's not a habanero issue that I know of.

E.g., try it in a curl request on the command line (jq is here if you want to use that, otherwise just do the curl part)

without polite pool

 curl https://api.crossref.org/journals/2448-1904 | jq .message.subjects

[
  {
    "ASJC": 2505,
    "name": "Materials Chemistry"
  },
  {
    "ASJC": 2002,
    "name": "Economics and Econometrics"
  },
  {
    "ASJC": 2214,
    "name": "Media Technology"
  },
  {
    "ASJC": 1107,
    "name": "Forestry"
  }
]

with polite pool

curl -H 'User-Agent: (mailto:myemail@gmail.com)' https://api.crossref.org/journals/2448-1904 | jq .message.subjects

[
  {
    "ASJC": 2700,
    "name": "General Medicine"
  }
]

To explain, setting your email in mailto puts you in the polite pool . Without your email set you are in the non-polite pool. that link describes what polite pool does.

What I think is happening is that somehow the data backing each of the pools is different somehow for this API request. I'll see if I can track it down with Crossref

sckott commented 8 months ago

forum question linkback https://community.crossref.org/t/polite-pol-and-non-polite-pool-data-out-of-sync-possibly/4871

sckott commented 8 months ago

@wisentini see response from crossref https://community.crossref.org/t/polite-pol-and-non-polite-pool-data-out-of-sync-possibly/4871/2?u=sckott

for now if you could do non polite pool for the data you need - then use polite pool again when they have fixed it

sckott commented 5 months ago

@wisentini see update from Crossref https://community.crossref.org/t/polite-pol-and-non-polite-pool-data-out-of-sync-possibly/4871/4 they're no longer returning subjects, so im closing this