search

mannau / tm.plugin.webmining

Retrieve structured, textual data from various web sources.
34 stars 10 forks source link

Not able to use NYTimesSource #15

Open ekoepplin opened 8 years ago

ekoepplin commented 8 years ago

Hello,

thank you for providing this nice library.

I'm trying to use:

nytimes <- WebCorpus(NYTimesSource("Microsoft", appid = nytimes_appid))

with the API key for search articles:

nytimes_appid = '6d9f........................h432' from http://developer.nytimes.com/

However, I always get: Error in json$response : $ operator is invalid for atomic vectors

May I ask for your assistance

sessionInfo() R version 3.2.1 (2015-06-18) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 7 x64 (build 7601) Service Pack 1

locale: [1] LC_COLLATE=German_Germany.1252 LC_CTYPE=German_Germany.1252
[3] LC_MONETARY=German_Germany.1252 LC_NUMERIC=C
[5] LC_TIME=German_Germany.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] RJSONIO_1.3-0 diezeit_0.1-0 SnowballC_0.5.1
[4] tm.plugin.webmining_1.3 tm_0.6-2 NLP_0.1-9

loaded via a namespace (and not attached): [1] XML_3.98-1.3 bitops_1.0-6 slam_0.1-32 R6_2.1.1
[5] jsonlite_0.9.21 httr_1.1.0 brew_1.0-6 tools_3.2.1
[9] boilerpipeR_1.3 RCurl_1.95-4.7 parallel_3.2.1 rJava_0.9-7`

GoranMilovanovic commented 8 years ago

Hi,

try by sending params = list() explicitly to NYTimesSource before applying WebCorpus, e.g.

#NOTE: nyTimesid is yours NYT Search API ID query <- "Microsoft" NYTimesNews <- WebCorpus(NYTimesSource(query, n = 100, appid = nyTimesid, # your NYT search API ID sleep = 1, params = list(# format = "json", q = query, begin_date = bd, end_date = ed, 'api-key' = nyTimesid) ) )

Let me know whether this works for you. I keep facing similar problems with NYTimesSource in tm.plugin.webmining.

zross commented 8 years ago

This is a really great package. The other sources (Google, Yahoo) seem to work. I had the same issue as above, then made the changes you suggested but I'm not getting anything. It does not give an error but also returns no documents. I'm not sure what the date format should be. The ID I'm using is the one from the NYT developers site.

NYTimesNews <- WebCorpus(NYTimesSource("Microsoft", n = 1000, appid = nytimes_appid, sleep = 1, params = list( format = "json", q = "Microsoft", begin_date = "2013-01-01", end_date = "2016-01-01", 'api-key' = nytimes_appid)))