search

zhiyzuo / python-scopus

PyScopus
http://zhiyzuo.github.io/python-scopus/
MIT License
23 stars 29 forks source link

HTTP Error #10

Closed zdarktknight closed 7 years ago

zdarktknight commented 7 years ago

`papers_count = scopus.search_venue('Computers and Chemical Engineering', count=7048, year_range=(1977,2016), show=False)'

I get an error saying: raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 400: Bad Request

zhiyzuo commented 7 years ago

I tried to search “Computers and Chemical Engineering” without specifying the count parameter and it worked well. I also tried to set count=100, which also returned the results well.

Try to change your count to smaller number. I suspect that your count param is too big.

zhiyzuo commented 7 years ago

1000 also worked out well.

zdarktknight commented 7 years ago

`papers_dict, total = scopus.search_venue('Computers and Chemical Engineering', year_range=(1999,2000), show=False)' I get another error saying:

title = entry.find('dc:title').text AttributeError: 'NoneType' object has no attribute 'text'

zhiyzuo commented 7 years ago

I cannot reproduce your problem. image

zhiyzuo commented 7 years ago

BTW, why you are returning dict and total? search_venue only return a dictionary.

zdarktknight commented 7 years ago

After I install it. There are 3 types of errors in my machine:

  1. In scopus.py, 'trunc' function is not imported. (I import it by myself)
  2. title = entry.find('dc:title').text AttributeError: 'NoneType' object has no attribute 'text' (I still have this error)
  3. Encoding issue, ascii cannot encode/decode....out of range.... (i have to ignore it)

I want to return all the searched results, so I modified your source code a little bit.

zhiyzuo commented 7 years ago

  1. For the trunc function, it may be the bug where I forgot to add import in the .py file. This does not matter though -- you can set show=False so that trunc would not be used.

  2. I did not expect that a paper could be retrieved but there is no title for it. Besides, I run your code but cannot reproduce your problem.

  3. I think I changed the coding to be able to handle UTF-8. I use my package a lot and never encountered such problem.

If you want to search for all the result, I suggest you send the http request from your browser. There will be an entry for the total number of papers and you can use that.

However, you should acquaint yourself with the API specifications: I think my implementation only returns scopus id and title. To retrieve more detailed information such as date, authors, citation, or even abstracts, you have to know your api limits. Otherwise, you will not be able to get all the information you wanted,

zhiyzuo commented 7 years ago

New release should not have any problem now.