M3SOulu / TrendMining

Trends and Text mining scripts used in teaching and in research. Fetches data from Scopus, StackOverflow and Twitter. Performs Latent Dirichlet Allocation, Wordcloud, etc.
MIT License
9 stars 21 forks source link

Accessing scopus without VPN #10

Open arpadgabor opened 2 years ago

arpadgabor commented 2 years ago

As it is a rather cumbersome process of getting access to scopus without VPN (because you need to log in with a finnish bank account that one might not have), the following is a proposal implementation for making requests to scopus without VPN but with the university account.

in GetScopusData.r

my_articles = get_scopus_papers(
  my_query_string,
  api_url = "https://api-elsevier-com.pc124152.oulu.fi:9443", # optional
  cookie = "cookie header" # optional
)

in FunctionsScopusApi.r

inst_cookie_header = function(cookie) {
  x = c("Cookie" = cookie)
  class(x) = "cookie"
  x
}

get_scopus_papers = function (
  query_string,
  api_url = "https://api.elsevier.com",
  cookie = NULL
){
  # other code

  if(!is.null(cookie)) {
    head = inst_cookie_header(cookie)
  }

  resp = generic_elsevier_api(
    root_http = api_url,
    query=query_string,
    type="search",
    search_type="scopus",
    cursor=cursor_value,
    view="COMPLETE",
    headers = head
  )
}

And the cookie value can be easily used when accessing the API URL for search after logging in with the university account (https://api-elsevier-com.pc124152.oulu.fi:9443/content/search/scopus).

image

mmantyla commented 2 years ago

Looks like a nice workaround for people without VPN-access.