search

albertosantini / node-rio

Integration with Rserve, a TCP/IP server for R framework
https://github.com/albertosantini/node-conpa
MIT License
176 stars 35 forks source link

expect callback(true) when parsing error happens #9

Closed ghost closed 10 years ago

ghost commented 10 years ago

Rserve disconnect from client because of parsing errors, so no response value comes back.

albertosantini commented 10 years ago

Maybe I got it. Do you mean "parsing errors" in the R script?

ghost commented 10 years ago

yes, when you start with a new R environment, ext2.js goes wrong.

albertosantini commented 10 years ago

Following an hint in Rserve FAQ, I have been trying to catch the parse exception on server side.

http://rforge.net/Rserve/faq.html#errors

Below the details I have been working on. I hope to support the issue before the end of the week.

rio.evaluate("1 // 3"); // syntax error
function evaluate(cmd, options) {
    var command = "try(eval(parse(text=\"" + cmd + "\")), silent=TRUE)";
    sendAction(command, CMD_TYPE.EVAL, "Sending command to Rserve", options);
}
Connected to Rserve
Supported capabilities --------------

Sending command to Rserve
00000000: 0300 0000 3400 0000 0000 0000 0000 0000  ....4...........
00000010: 0430 0000 7472 7928 6576 616c 2870 6172  .0..try(eval(par
00000020: 7365 2874 6578 743d 2231 202f 2f20 3322  se(text="1.//.3"
00000030: 2929 2c20 7369 6c65 6e74 3d54 5255 4529  )),.silent=TRUE)
00000040: 0001 0101                                ....

Data packet
00000000: a230 0100 15dc 0000 220c 0000 7472 792d  "0...\.."...try-
00000010: 6572 726f 7200 0101 1308 0000 636c 6173  error.......clas
00000020: 7300 0000 90ac 0000 154c 0000 2210 0000  s....,...L.."...
00000030: 6d65 7373 6167 6500 6361 6c6c 0001 0101  message.call....
00000040: 1308 0000 6e61 6d65 7300 0000 221c 0000  ....names..."...
00000050: 7369 6d70 6c65 4572 726f 7200 6572 726f  simpleError.erro
00000060: 7200 636f 6e64 6974 696f 6e00 1308 0000  r.condition.....
00000070: 636c 6173 7300 0000 222c 0000 3c74 6578  class...",..<tex
00000080: 743e 3a31 3a34 3a20 756e 6578 7065 6374  t>:1:4:.unexpect
00000090: 6564 2027 2f27 0a31 3a20 3120 2f2f 0a20  ed.'/'.1:.1.//..
000000a0: 2020 2020 2020 5e00 1728 0000 1308 0000  ......^..(......
000000b0: 7061 7273 6500 0000 0000 0000 2208 0000  parse......."...
000000c0: 3120 2f2f 2033 0001 1308 0000 7465 7874  1.//.3......text
000000d0: 0000 0000 130c 0000 636f 6e64 6974 696f  ........conditio
000000e0: 6e00 0000 4572 726f 7220 696e 2070 6172  n...Error.in.par
000000f0: 7365 2874 6578 7420 3d20 2231 202f 2f20  se(text.=."1.//.
00000100: 3322 2920 3a20 3c74 6578 743e 3a31 3a34  3").:.<text>:1:4
00000110: 3a20 756e 6578 7065 6374 6564 2027 2f27  :.unexpected.'/'
00000120: 0a31 3a20 3120 2f2f 0a20 2020 2020 2020  .1:.1.//........
00000130: 5e0a 0001                                ^...

Type SEXP 162
Type 162 is currently not implemented
Rserve call failed
Disconnected from Rserve
Closed from Rserve
ghost commented 10 years ago

suppose I do not install RJSONIO yet，run node ex2.js It just closed, with no response value.

Connected to Rserve
Supported capabilities --------------

Sending command to Rserve
00000000: 0300 0000 a804 0000 0000 0000 0000 0000  ....�...........
00000010: 04a4 0400 7265 7175 6972 6528 7473 6572  .$......ire(....
00000020: 6965 7329 0a72 6571 7569 7265 2852 4a53  ies).require(RJS
00000030: 4f4e 494f 290a 0a68 6163 6b5a 6572 6f65  ONIO)..hackZeroe
00000040: 7320 3c2d 2066 756e 6374 696f 6e20 2878  s.<-.function.(x
00000050: 2920 7b0a 2020 2020 666f 7220 2869 2069  ).{.....for.(i.i
00000060: 6e20 7768 6963 6828 7820 3d3d 2030 2929  n.which(x.==.0))
00000070: 207b 0a20 2020 2020 2020 2069 6620 2869  .{.........if.(i
00000080: 203d 3d20 3129 207b 0a20 2020 2020 2020  .==.1).{........
00000090: 2020 2020 2063 203d 2069 0a20 2020 2020  .....c.=.i......
000000a0: 2020 2020 2020 2077 6869 6c65 2028 785b  .......while.(x[
000000b0: 635d 203d 3d20 3029 207b 0a20 2020 2020  c].==.0).{......
000000c0: 2020 2020 2020 2020 2020 2063 203d 2063  ...........c.=.c
000000d0: 202b 2031 0a20 2020 2020 2020 2020 2020  .+.1............
000000e0: 207d 0a20 2020 2020 2020 2020 2020 2078  .}.............x
000000f0: 5b69 5d20 3c2d 2078 5b63 5d0a 2020 2020  [i].<-.x[c].....
00000100: 2020 2020 7d20 656c 7365 207b 0a20 2020  ....}.else.{....
00000110: 2020 2020 2020 2020 2078 5b69 5d20 3c2d  .........x[i].<-
00000120: 2078 5b69 202d 2031 5d0a 2020 2020 2020  .x[i.-.1].......
00000130: 2020 7d0a 2020 2020 7d0a 2020 2020 780a  ..}.....}.....x.
00000140: 7d0a 0a67 6574 5265 7475 726e 7320 3c2d  }..getReturns.<-
00000150: 2066 756e 6374 696f 6e20 2873 796d 626f  .function.(symbo
00000160: 6c29 207b 0a20 2020 2061 7373 6574 5072  l).{.....assetPr
00000170: 6963 6520 3c2d 2061 732e 7665 6374 6f72  ice.<-.as.vector
00000180: 2867 6574 2e68 6973 742e 7175 6f74 6528  (get.hist.quote(
00000190: 7379 6d62 6f6c 2c0a 2020 2020 2020 2020  symbol,.........
000001a0: 7374 6172 743d 2232 3030 392d 382d 3622  start="2009-8-6"
000001b0: 2c20 656e 643d 2232 3031 312d 382d 3622  ,.end="2011-8-6"
000001c0: 2c0a 2020 2020 2020 2020 636f 6d70 7265  ,.........compre
000001d0: 7373 696f 6e3d 2277 222c 2071 756f 7465  ssion="w",.quote
000001e0: 3d22 436c 6f73 6522 2c20 7175 6965 743d  ="Close",.quiet=
000001f0: 5452 5545 2929 0a20 2020 2061 7373 6574  TRUE)).....asset
00000200: 5072 6963 6520 3c2d 2068 6163 6b5a 6572  Price.<-.hackZer
00000210: 6f65 7328 6173 7365 7450 7269 6365 290a  oes(assetPrice).
00000220: 2020 2020 6173 7365 7452 6574 7572 6e73  ....assetReturns
00000230: 203c 2d20 6469 6666 286c 6f67 2861 7373  .<-.diff(log(ass
00000240: 6574 5072 6963 655b 313a 286c 656e 6774  etPrice[1:(lengt
00000250: 6828 6173 7365 7450 7269 6365 292d 3129  h(assetPrice)-1)
00000260: 5d29 290a 0a20 2020 2072 6574 7572 6e28  ]))......return(
00000270: 6173 7365 7452 6574 7572 6e73 290a 7d0a  assetReturns).}.
00000280: 0a67 6574 4f70 7469 6d61 6c50 6f72 7466  .getOptimalPortf
00000290: 6f6c 696f 203c 2d20 6675 6e63 7469 6f6e  olio.<-.function
000002a0: 2028 6a73 6f6e 4f62 6a29 207b 0a20 2020  .(jsonObj).{....
000002b0: 2078 203c 2d20 6328 290a 0a20 2020 206f  .x.<-.c()......o
000002c0: 203d 2066 726f 6d4a 534f 4e28 6a73 6f6e  .=.fromJSON(json
000002d0: 4f62 6a29 0a20 2020 2073 796d 626f 6c73  Obj).....symbols
000002e0: 203c 2d20 6f24 7072 6f64 730a 0a20 2020  .<-.o$prods.....
000002f0: 2066 6f72 2028 6173 7365 7420 696e 2073  .for.(asset.in.s
00000300: 796d 626f 6c73 2920 7b0a 2020 2020 2020  ymbols).{.......
00000310: 2020 7265 7473 203d 2067 6574 5265 7475  ..rets.=.getRetu
00000320: 726e 7328 6173 7365 7429 3b0a 2020 2020  rns(asset);.....
00000330: 2020 2020 7820 3c2d 2063 6269 6e64 2878  ....x.<-.cbind(x
00000340: 2c20 7265 7473 290a 2020 2020 7d0a 0a20  ,.rets).....}...
00000350: 2020 2023 2054 6172 6765 7420 5265 7475  ...#.Target.Retu
00000360: 726e 2028 706d 2920 6973 2077 6565 6b6c  rn.(pm).is.weekl
00000370: 790a 2020 2020 7265 7320 3c2d 2070 6f72  y.....res.<-.por
00000380: 7466 6f6c 696f 2e6f 7074 696d 2878 2c20  tfolio.optim(x,.
00000390: 706d 203d 206d 6561 6e28 7829 2c0a 2020  pm.=.mean(x),...
000003a0: 2020 2020 2020 7265 736c 6f77 203d 2063  ......reslow.=.c
000003b0: 2830 2c20 302c 2030 292c 2072 6573 6869  (0,.0,.0),.reshi
000003c0: 6768 203d 2063 2831 2c20 312c 2031 2929  gh.=.c(1,.1,.1))
000003d0: 0a0a 2020 2020 7265 7475 726e 2874 6f4a  ......return(toJ
000003e0: 534f 4e28 7265 7329 290a 7d0a 0a23 2061  SON(res)).}..#.a
000003f0: 7267 7320 3c2d 2027 7b22 7072 6f64 7322  rgs.<-.'{"prods"
00000400: 3a5b 2249 424d 222c 2259 484f 4f22 2c22  :["IBM","YHOO","
00000410: 4d53 4654 225d 7d27 3b0a 2320 7265 7320  MSFT"]}';.#.res.
00000420: 3c2d 2067 6574 4f70 7469 6d61 6c50 6f72  <-.getOptimalPor
00000430: 7466 6f6c 696f 2861 7267 7329 0a23 2066  tfolio(args).#.f
00000440: 726f 6d4a 534f 4e28 7265 7329 2470 770a  romJSON(res)$pw.
00000450: 0a23 204f 7074 696d 616c 2077 6569 6768  .#.Optimal.weigh
00000460: 7473 3a20 302e 3237 3130 372c 302e 3236  ts:.0.27107,0.26
00000470: 3838 2c30 2e34 3630 3133 0a0a 0a0a 6765  88,0.46013....ge
00000480: 744f 7074 696d 616c 506f 7274 666f 6c69  tOptimalPortfoli
00000490: 6f28 277b 2270 726f 6473 223a 5b22 4942  o('{"prods":["IB
000004a0: 4d22 2c22 5948 4f4f 222c 224d 5346 5422  M","YHOO","MSFT"
000004b0: 5d7d 2729 0001 0101                      ]}')....

Disconnected from Rserve
Closed from Rserve
ghost commented 10 years ago

This was the debug info from Rserve

> library(Rserve)
> Rserve(TRUE)
Starting Rserve...
 "c:\Users\chipeng.xu\R\WIN-LI~1\3.0\Rserve\libs\i386\Rserve_d.exe"
> Rserve 1.7-3 () (C)Copyright 2002-2013 Simon Urbanek
$Id$

Loading config file Rserv.cfg
Failed to find config file Rserv.cfg
Rserve: Ok, ready to answer queries.
 - create_server(port = 6311, socket = <NULL>, mode = 0, flags = 0x4000)
INFO: adding server 02D55A98 (total 1 servers)
INFO: accepted connection for server 02D55A98, calling connected
connection accepted.
sending ID string.

header read result: 16
DUMP [16]: 03 00 00 00 a8 04 00 00 00 00 00 00 00 00 00 00  |................
loading buffer (awaiting 1192 bytes)
parsing parameters (buf=02DD6BE8, len=1192)
DUMP [1192]: 04 a4 04 00 72 65 71 75 69 72 65 28 74 73 65 72 69 65 73 29 0a 72 6
5 71 75 69 72 65 28 52 4a 53 4f 4e 49 4f 29 0a 0a 68 61 63 6b 5a 65 72 6f 65 73
20 3c 2d 20 66 75 6e 63 74 69 6f 6e 20 28 78 29 20 7b 0a 20 20 20 20 66 6f 72 20
 28 69 20 69 6e 20 77 68 69 63 68 28 78 20 3d 3d 20 30 29 29 20 7b 0a 20 20 20 2
0 20 20 20 20 69 66 20 28 69 20 3d 3d 20 31 29 20 7b 0a 20 20 20 20 20 20 20 20
...  |....require(tseries).require(RJSONIO)..hackZeroes <- function (x) {.    fo
r (i in which(x == 0)) {.        if (i == 1) {.
PAR[0]: 000004a8 (PAR_LEN=1188, PAR_TYPE=4, large=no, c=02DD6BE8, ptr=02DD6BEC)
CMD=00000003, pars=1
parseString("require(tseries)
require(RJSONIO)

hackZeroes <- function (x) {
    for (i in which(x == 0)) {
        if (i == 1) {
            c = i
            while (x[c] == 0) {
                c = c + 1
            }
            x[i] <- x[c]
        } else {
            x[i] <- x[i - 1]
        }
    }
    x
}

getReturns <- function (symbol) {
    assetPrice <- as.vector(get.hist.quote(symbol,
        start="2009-8-6", end="2011-8-6",
        compression="w", quote="Close", quiet=TRUE))
    assetPrice <- hackZeroes(assetPrice)
    assetReturns <- diff(log(assetPrice[1:(length(assetPrice)-1)]))

    return(assetReturns)
}

getOptimalPortfolio <- function (jsonObj) {
    x <- c()

    o = fromJSON(jsonObj)
    symbols <- o$prods

    for (asset in symbols) {
        rets = getReturns(asset);
        x <- cbind(x, rets)
    }

    # Target Return (pm) is weekly
    res <- portfolio.optim(x, pm = mean(x),
        reslow = c(0, 0, 0), reshigh = c(1, 1, 1))

    return(toJSON(res))
}

# args <- '{"prods":["IBM","YHOO","MSFT"]}';
# res <- getOptimalPortfolio(args)
# fromJSON(res)$pw

# Optimal weights: 0.27107,0.2688,0.46013

getOptimalPortfolio('{"prods":["IBM","YHOO","MSFT"]}')")
buffer parsed, stat=1, parts=57
result type: 20, length: 6
R_tryEval(xp,R_GlobalEnv,&Rerror);
Calling R_tryEval for expression 1 [type=6] ...
tseries
Expression 1, error code: 0
Calling R_tryEval for expression 2 [type=6] ...
RJSONIO
Expression 2, error code: 0
Calling R_tryEval for expression 3 [type=6] ...
Expression 3, error code: 0
Calling R_tryEval for expression 4 [type=6] ...
Expression 4, error code: 0
Calling R_tryEval for expression 5 [type=6] ...
Expression 5, error code: 0
Calling R_tryEval for expression 6 [type=6] ...
Error in getOptimalPortfolio("{\"prods\":[\"IBM\",\"YHOO\",\"MSFT\"]}") :
  could not find function "fromJSON"
In addition: Warning messages:
1: In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return
= TRUE,  :
  'tseries'
2: In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return
= TRUE,  :
  'RJSONIO'
Expression 6, error code: 1
>> early error, aborting further evaluations
expression(s) evaluated (Rerror=1).
OUT.sendRespData
HEAD DUMP [16]: 02 00 01 7f 00 00 00 00 00 00 00 00 00 00 00 00  |.............
..
(no body)
reply sent.
Connection closed by peer.
done.
INFO: accepted connection for server 02D55A98, calling connected
connection accepted.
sending ID string.

header read result: 16
DUMP [16]: 03 00 00 00 a8 04 00 00 00 00 00 00 00 00 00 00  |................
loading buffer (awaiting 1192 bytes)
parsing parameters (buf=02DD6BE8, len=1192)
DUMP [1192]: 04 a4 04 00 72 65 71 75 69 72 65 28 74 73 65 72 69 65 73 29 0a 72 6
5 71 75 69 72 65 28 52 4a 53 4f 4e 49 4f 29 0a 0a 68 61 63 6b 5a 65 72 6f 65 73
20 3c 2d 20 66 75 6e 63 74 69 6f 6e 20 28 78 29 20 7b 0a 20 20 20 20 66 6f 72 20
 28 69 20 69 6e 20 77 68 69 63 68 28 78 20 3d 3d 20 30 29 29 20 7b 0a 20 20 20 2
0 20 20 20 20 69 66 20 28 69 20 3d 3d 20 31 29 20 7b 0a 20 20 20 20 20 20 20 20
...  |....require(tseries).require(RJSONIO)..hackZeroes <- function (x) {.    fo
r (i in which(x == 0)) {.        if (i == 1) {.
PAR[0]: 000004a8 (PAR_LEN=1188, PAR_TYPE=4, large=no, c=02DD6BE8, ptr=02DD6BEC)
CMD=00000003, pars=1
parseString("require(tseries)
require(RJSONIO)

hackZeroes <- function (x) {
    for (i in which(x == 0)) {
        if (i == 1) {
            c = i
            while (x[c] == 0) {
                c = c + 1
            }
            x[i] <- x[c]
        } else {
            x[i] <- x[i - 1]
        }
    }
    x
}

getReturns <- function (symbol) {
    assetPrice <- as.vector(get.hist.quote(symbol,
        start="2009-8-6", end="2011-8-6",
        compression="w", quote="Close", quiet=TRUE))
    assetPrice <- hackZeroes(assetPrice)
    assetReturns <- diff(log(assetPrice[1:(length(assetPrice)-1)]))

    return(assetReturns)
}

getOptimalPortfolio <- function (jsonObj) {
    x <- c()

    o = fromJSON(jsonObj)
    symbols <- o$prods

    for (asset in symbols) {
        rets = getReturns(asset);
        x <- cbind(x, rets)
    }

    # Target Return (pm) is weekly
    res <- portfolio.optim(x, pm = mean(x),
        reslow = c(0, 0, 0), reshigh = c(1, 1, 1))

    return(toJSON(res))
}

# args <- '{"prods":["IBM","YHOO","MSFT"]}';
# res <- getOptimalPortfolio(args)
# fromJSON(res)$pw

# Optimal weights: 0.27107,0.2688,0.46013

getOptimalPortfolio('{"prods":["IBM","YHOO","MSFT"]}')")
buffer parsed, stat=1, parts=57
result type: 20, length: 6
R_tryEval(xp,R_GlobalEnv,&Rerror);
Calling R_tryEval for expression 1 [type=6] ...
tseries
Expression 1, error code: 0
Calling R_tryEval for expression 2 [type=6] ...
RJSONIO
Expression 2, error code: 0
Calling R_tryEval for expression 3 [type=6] ...
Expression 3, error code: 0
Calling R_tryEval for expression 4 [type=6] ...
Expression 4, error code: 0
Calling R_tryEval for expression 5 [type=6] ...
Expression 5, error code: 0
Calling R_tryEval for expression 6 [type=6] ...
Error in getOptimalPortfolio("{\"prods\":[\"IBM\",\"YHOO\",\"MSFT\"]}") :
  could not find function "fromJSON"
In addition: Warning messages:
1: In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return
= TRUE,  :
  'tseries'
2: In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return
= TRUE,  :
  'RJSONIO'
Expression 6, error code: 1
>> early error, aborting further evaluations
expression(s) evaluated (Rerror=1).
OUT.sendRespData
HEAD DUMP [16]: 02 00 01 7f 00 00 00 00 00 00 00 00 00 00 00 00  |.............
..
(no body)
reply sent.
Connection closed by peer.
done.
albertosantini commented 10 years ago

With parse and eval I catch the errors, but the side effect is the error returned as a string.

function evaluate(cmd, options) {
    var command = "res = try({" + cmd + "}, silent=TRUE);" +
        "unlist(res[sapply(res, function(x) !inherits(x, \"try-error\"))])\n";

    sendAction(command, CMD_TYPE.EVAL, "Sending command to Rserve", options);
}

So I followed your way. Anyway I noticed I forgot to add the check for the eval response and the data type.

Amending with this commit: https://github.com/albertosantini/node-rio/commit/5fcf7c7c4e11cb8ced8f4fbcc63fef765dde4f54

It should fix the issue.

Later I will build a new npm version depending on your feedback.

ghost commented 10 years ago

expecting the new version.