muschellij2 / rscopus

Scopus Database API Interface to R
75 stars 16 forks source link

Add a "sleep" parameter in scopus_search.R when getting more than 25/200 items #17

Closed dkremmydas closed 5 years ago

dkremmydas commented 5 years ago

It would be useful if the user could specify a Sys.sleep time that would be applied between consecutive retrievals that are called internally from the scopus_search.R function.

In my case I try to download ~5000 items in a single _scopussearch call, and I get a HTTP 400 (Bad Request) Response, somewhere in the middle of retrieving the items. I suspect the API has a limit of requests per second. Thus setting such a parameter would force the _scopussearch to maintain a maximum request rate.

If there is no other workaround, I will try to modify the scopus_search.R and add this functionality.

muschellij2 commented 5 years ago

Please send a pull request. John

On Wed, Feb 6, 2019 at 11:42 AM Dimitris Kremmydas notifications@github.com wrote:

It would be useful if the user could specify a Sys.sleep time that would be applied between consecutive retrievals that are called internally from the scopus_search.R function.

In my case I try to download ~5000 items in a single scopus_search call, and I get a HTTP 400 (Bad Request) Response, somewhere in the middle of retrieving the items. I suspect there is a limit of requests per second. Thus setting such a parameter would force the scopus_search to maintain a maximum request rate.

If there is no other workaround, I will try to modify the scopus_search.R and add this functionality.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/muschellij2/rscopus/issues/17, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBnrr8i79jD3dxLnomHqQt9pki_LJ_Vks5vK0xBgaJpZM4amKuQ .

dkremmydas commented 5 years ago

Shall I fork, do the changes in the local repo and create the pull request, or shall I edit file directly here and create a pull request.

I did it the second way. I it ok ?

dimitris

muschellij2 commented 5 years ago

Fixed via https://github.com/muschellij2/rscopus/blob/master/R/scopus_search.R#L53