search

tc / RMongo

R client to interface with MongoDB
102 stars 34 forks source link

RMONGO with streaming data #18

Open louisdv1 opened 11 years ago

louisdv1 commented 11 years ago

Hi , I am pulling data out of an API with R and it seems that the insert to mongodb keeps failing? This is the message I am having: Error in .jcall(rmongo.object@javaMongo, "S", "dbInsertDocument", collection, : com.mongodb.util.JSONParseException:

tc commented 11 years ago

What query command are you running? what does your mongo collection look like?

louisdv1 commented 11 years ago

Hi Tommy and thank you for the fast response. I am pulling data out of stocktwits and pushing it to mongodb. Sometimes I have a parsing problem but the most problematic is that the json feed representing several messages seem to be bundled up together so that mongo only recognizes one message per call . Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url="
https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo =
"cacert.pem")},error=function(e){})

if (length(result)>0)  dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much On 1 May 2013 17:26, "Tommy Chheng" notifications@github.com wrote:

What query command are you running? what does your mongo collection look like?

— Reply to this email directly or view it on GitHubhttps://github.com/tc/RMongo/issues/18#issuecomment-17290176 .

tc commented 11 years ago

What is z in dbInsertDocument(mongo, "stocktwits7", z)? It looks like an unreferenced variable.

tc commented 11 years ago

You can also use rjson to extract the messages array out of the url response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 notifications@github.com wrote:

Hi Tommy and thank you for the fast response. I am pulling data out of stocktwits and pushing it to mongodb. Sometimes I have a parsing problem but the most problematic is that the json feed representing several messages seem to be bundled up together so that mongo only recognizes one message per call . Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url=" https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo = "cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much On 1 May 2013 17:26, "Tommy Chheng" notifications@github.com wrote:

What query command are you running? what does your mongo collection look like?

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17290176> .

— Reply to this email directly or view it on GitHubhttps://github.com/tc/RMongo/issues/18#issuecomment-17296054 .

Tommy Chheng

louisdv1 commented 11 years ago

Stocktwits is just the name of tge collection and z is a json response fetched by rcurl On 1 May 2013 20:39, "Tommy Chheng" notifications@github.com wrote:

You can also use rjson to extract the messages array out of the url response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 notifications@github.com wrote:

Hi Tommy and thank you for the fast response. I am pulling data out of stocktwits and pushing it to mongodb. Sometimes I have a parsing problem but the most problematic is that the json feed representing several messages seem to be bundled up together so that mongo only recognizes one message per call . Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url=" https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo = "cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much On 1 May 2013 17:26, "Tommy Chheng" notifications@github.com wrote:

What query command are you running? what does your mongo collection look like?

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17290176> .

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17296054> .

Tommy Chheng

— Reply to this email directly or view it on GitHubhttps://github.com/tc/RMongo/issues/18#issuecomment-17301621 .

tc commented 11 years ago

Try printing out the z variable to see if it's malformed. That could be the reason why it's not inserting correctly.

On Wed, May 1, 2013 at 12:42 PM, louisdv1 notifications@github.com wrote:

Stocktwits is just the name of tge collection and z is a json response fetched by rcurl On 1 May 2013 20:39, "Tommy Chheng" notifications@github.com wrote:

You can also use rjson to extract the messages array out of the url response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 notifications@github.com wrote:

Hi Tommy and thank you for the fast response. I am pulling data out of stocktwits and pushing it to mongodb. Sometimes I have a parsing problem but the most problematic is that the json feed representing several messages seem to be bundled up together so that mongo only recognizes one message per call . Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url=" https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo = "cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much On 1 May 2013 17:26, "Tommy Chheng" notifications@github.com wrote:

What query command are you running? what does your mongo collection look like?

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17290176> .

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17296054> .

Tommy Chheng

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17301621> .

— Reply to this email directly or view it on GitHubhttps://github.com/tc/RMongo/issues/18#issuecomment-17301832 .

Tommy Chheng