scholarly-python-package / scholarly

Retrieve author and publication information from Google Scholar in a friendly, Pythonic way without having to worry about CAPTCHAs!
https://scholarly.readthedocs.io/
The Unlicense
1.29k stars 292 forks source link

No documentation on scraperAPI integration #301

Closed samuelhaysom closed 2 years ago

samuelhaysom commented 2 years ago

I am interested in using scholarly in combination with scraperAPI however there doesn't seem to be any documentation on how this can be done even though the feature is present in the code. I am particularly confused about how to get test_module to work with scraperAPI so I can see if the package is working as planned without being immediately blocked by google.

ipeirotis commented 2 years ago
from scholarly import scholarly, ProxyGenerator
pg = ProxyGenerator()
pg = pg.ScraperAPI(YOUR_SCRAPER_API_KEY)
scholarly.use_proxy(pg)

search_query = scholarly.search_pubs('Perception of physical stability and center of mass of 3D objects')
r = next(search_query)
scholarly.pprint(r)

returns

{'author_id': ['4bahYMkAAAAJ', 'ruUKktgAAAAJ', ''],
 'bib': {'abstract': 'Humans can judge from vision alone whether an object is '
                     'physically stable or not. Such judgments allow observers '
                     'to predict the physical behavior of objects, and hence '
                     'to guide their motor actions. We investigated the visual '
                     'estimation of physical stability of 3-D objects (shown '
                     'in stereoscopically viewed rendered scenes) and how it '
                     'relates to visual estimates of their center of mass '
                     '(COM). In Experiment 1, observers viewed an object near '
                     'the edge of a table and adjusted its tilt to the '
                     'perceived critical angle, ie, the tilt angle at which '
                     'the object',
         'author': ['SA Cholewiak', 'RW Fleming', 'M Singh'],
         'pub_year': '2015',
         'title': 'Perception of physical stability and center of mass of 3-D '
                  'objects',
         'venue': 'Journal of vision'},
 'citedby_url': '/scholar?cites=15736880631888070187&as_sdt=5,33&sciodt=0,33&hl=en',
 'eprint_url': 'https://jov.arvojournals.org/article.aspx?articleID=2213254',
 'filled': False,
 'gsrank': 1,
 'num_citations': 22,
 'pub_url': 'https://jov.arvojournals.org/article.aspx?articleID=2213254',
 'source': 'PUBLICATION_SEARCH_SNIPPET',
 'url_add_sclib': '/citations?hl=en&xsrf=&continue=/scholar%3Fq%3DPerception%2Bof%2Bphysical%2Bstability%2Band%2Bcenter%2Bof%2Bmass%2Bof%2B3D%2Bobjects%26hl%3Den%26as_sdt%3D0,33&citilm=1&json=&update_op=library_add&info=K8ZpoI6hZNoJ&ei=H6_4YLXlEsiWywSAkLmYDA',
 'url_related_articles': '/scholar?q=related:K8ZpoI6hZNoJ:scholar.google.com/&scioq=Perception+of+physical+stability+and+center+of+mass+of+3D+objects&hl=en&as_sdt=0,33',
 'url_scholarbib': '/scholar?q=info:K8ZpoI6hZNoJ:scholar.google.com/&output=cite&scirp=0&hl=en'}
arunkannawadi commented 2 years ago

Reopening this issue until this example is reflected in the readthedocs page.

arunkannawadi commented 2 years ago

ScraperAPI APIs are documented in v1.4.2. Closing the issue.