small difference in Scopus standard online search and ft_search results

[ghost]

Hello,

When I do the following search

ft_search(query="TITLE-ABS-KEY(ecology)",from="scopus")

I get 300321 records

and when I do a search for "ecology" using the standard Scopus online database

I get 300366 records

Which is pretty similar, but it took me a while to figure out that the standard Scopus online database search uses TITLE-ABS-KEY as a default and it looks like the ft_search function use the ALL field as a default or maybe Scopus applies this as a default if you don't specify a field? If so, perhaps it could be worth adding something like this to the function description?

Might good to include some more advanced query examples for this function too, but it looks like a useful pkg.

Thanks.

[sckott]

thanks for this @lucymerobinson and thanks for trying the pkg

That's right, we use don't specify ALL , but that's the default they use, so I guess that's what we're using too.

I agree we should document this more, including more advanced examples

[ghost]

Thanks for clarifying @sckott - I am happy to add some examples to the ft_search fcn if that would help. I'm also trying to get the facets param working in ft_search. I have tried using the scopusopts = list("ver=facetexpand") and scopusopts = list("facets=subjarea(count=5)"), but it doesn't return any additional fields. Would you have any advice on this?

[sckott]

A pull request would be great if you have the time.

I'll add facet suppor

[sckott]

@lucymerobinson reinstall and try examples in https://github.com/ropensci/fulltext/blob/master/R/scopus_utils.R#L36-L54 - and https://github.com/ropensci/fulltext/blob/master/R/ft_search.R#L107-L113 for ft_search - you can yet get the facet data with ft_search - do you want the facet data, or is it just that you want to use facets to limit your search results? from the docs https://dev.elsevier.com/tecdoc_api_facets.html it seems like using facets limits search results in addition to providing facets results.

[ghost]

Thanks @sckott I have responded to your facets questions under #170. I have updated the Scopus examples under ft_search to include a query with title, abstract and keyword fields, but removed examples with low limits (e.g. 5) and those with the start param as I wasn't sure why you needed this?

[sckott]

but removed examples with low limits (e.g. 5) and those with the start param as I wasn't sure why you needed this?

can you clarify, not sure what you're talking about?

[ghost]

sorry I wasn't clear. I'm just not sure why someone who is doing a lit search via API would only want to return 5 articles out of the 1000's that are likely to be found, so I was just questioning whether that was a relevant example to include? I also wasn't sure why you would want to use the start param for scopus searches - I guess this could be handy if you want to update a lit search a few years later and you have already collected articles from several years ago, but if that was the case perhaps it would be better to specify a date range in the query param?

[sckott]

The small limit (e.g., 5) is so that examples run quickly so users can get a quick sense of what the function does and returns. Does that make sense?

✔️ The example with start should go away - as i switched to using scopus_search_loop internally within ft_search for scopus searches, so it loops for the user, so no reason to use start anymore when using ft_search