Conversion to JSON fails on Windows

bb4242 commented 9 years ago

After alavrik/piqi#47 was fixed (thanks!), it is now possible to successfully start the Erlang piqi application on Windows (via piqi:start().). However, trying to convert a piqi message to JSON fails. (In the examples below, server_response_piqi is a piqi module that has been generated with piqic-erlang).

Erlang/OTP 17 [erts-6.2] [64-bit] [smp:8:8] [async-threads:10]

Eshell V6.2  (abort with ^G)
1> piqi:start().
ok
2> MyMessage = server_response_piqi:default_server_response().
{server_response_server_response,topic_data,<<>>}
3> server_response_piqi:gen_server_response(MyMessage, pb).
<<8,0,18,0>>
4> server_response_piqi:gen_server_response(MyMessage, json).
** exception exit: {{piqi_tools_error,{port_command_exited,epipe}},
                    {gen_server,call,
                                [<0.38.0>,
                                 {rpc,server_response_piqi,
                                      <<10,7,99,111,110,118,101,114,116,18,45,
                                        10,29,83,101,114,118,101,114,...>>},
                                 5000]}}
     in function  gen_server:call/3 (gen_server.erl, line 190)
     in call from piqi_tools:call_server/1 (src/piqi_tools.erl, line 348)
     in call from piqi_tools:rpc/3 (src/piqi_tools.erl, line 312)
     in call from piqi_tools:convert/6 (src/piqi_tools.erl, line 425)
     in call from piqirun_ext:convert/6 (src/piqirun_ext.erl, line 69)

When running server_response_piqi:gen_server_response(MyMessage, json) repeatedly, the epipe error seems to be most common, but I also occasionally see a different error when running the same command:

5> server_response_piqi:gen_server_response(MyMessage, json).
** exception exit: {{{nocatch,{piqirun_error,not_enough_data}},
                     [{piqirun,throw_error,1,
                               [{file,"src/piqirun.erl"},{line,714}]},
                      {piqirun,parse_field,1,
                               [{file,"src/piqirun.erl"},{line,554}]},
                      {piqirun,parse_record_buf_ordered,2,
                               [{file,"src/piqirun.erl"},{line,620}]},
                      {piqirun,parse_variant,1,
                               [{file,"src/piqirun.erl"},{line,641}]},
                      {piqi_rpc_piqi,parse_rpc_error,1,
                                     [{file,"src/piqi_rpc_piqi.erl"},
                                      {line,253}]},
                      {piqi_rpc_piqi,parse_response,1,
                                     [{file,"src/piqi_rpc_piqi.erl"},
                                      {line,220}]},
                      {piqi_tools,handle_rpc_response,2,
                                  [{file,"src/piqi_tools.erl"},{line,249}]},
                      {piqi_tools,port_receive_loop,1,
                                  [{file,"src/piqi_tools.erl"},{line,179}]}]},
                    {gen_server,call,
                                [<0.436.0>,
                                 {rpc,server_response_piqi,
                                      <<10,7,99,111,110,118,101,114,116,18,45,
                                        10,29,83,101,114,118,101,114,...>>},
                                 5000]}}
     in function  gen_server:call/3 (gen_server.erl, line 190)
     in call from piqi_tools:call_server/1 (src/piqi_tools.erl, line 348)
     in call from piqi_tools:rpc/3 (src/piqi_tools.erl, line 312)
     in call from piqi_tools:convert/6 (src/piqi_tools.erl, line 425)
     in call from piqirun_ext:convert/6 (src/piqirun_ext.erl, line 69)

Similar errors occur when trying to convert to XML rather than JSON. The same commands above work as expected on linux.

alavrik commented 9 years ago

It looks like piqi server is crashing when converting from pb to json. Not sure why. Perhaps another windows compatibility issue.

I don't have a windows system nearby right now... There are two options: 1) figure out what's going wrong exactly and fix it -- I will need your help with this but I think it is worth a try; 2) build piqi under cygwin by following unix build instructions -- hopefully, cygwin build works better (I am assuming you are using cygwin, correct me if I am wrong).

If you decide to go with option 1, there are a couple of things to try:

try reproducing the problem outside of Erlang by using piqi call to test pb -> json conversion in piqi server: https://github.com/alavrik/piqi/blob/master/tests/server/test_piqi_server#L53
if that triggers a problem, export OCAMLRUNPARAM=b to print server's stacktrace on stderr and post it here.

Alternatively, modify this line to capture server's stderr to a file and also erlang:open_port arguments to set OCAMLRUNPARAM environment variable as indicated above.

bb4242 commented 9 years ago

Thanks Anton. I spent a bit of time seeing how far I could get with option 1. I read through the test_piqi_server script and tried to replicate the functionality on a native Windows console. Thus far, I've been able to see the following:

C:\Users\bbethke\piqi>set OCAMLRUNPARAM=b
C:\Users\bbethke\piqi>piqi.exe call "piqi.exe server/ping"
uncaught system error: Invalid argument
Fatal error: exception Sys_error("Invalid argument")
Called from file "pervasives.ml", line 482, characters 30-33
Called from file "pervasives.ml", line 487, characters 2-15
server exited with error code 2

I'm not an OCaml expert - is this information useful in debugging the issue?

alavrik commented 9 years ago

This is very helpful. Thank you! Let me find a windows machine and I'll figure out the rest. There are a couple of weird things and it would be easier to investigate further having a custom build.

bb4242 commented 9 years ago

Sounds good, thanks a lot! :) Let me know if there's any other debugging assistance I can provide.

On Sat, Jan 10, 2015 at 5:00 AM, Anton Lavrik notifications@github.com wrote:

This is very helpful. Thank you! Let me find a windows machine and I'll figure out the rest.

— Reply to this email directly or view it on GitHub https://github.com/alavrik/piqi-erlang/issues/16#issuecomment-69454958.

alavrik commented 9 years ago

Should be fixed now in piqi-0.6.11 -- there were a couple of Windows compatibility problems. Can you verify it is working? piqi-erlang is pointed to the newest piqi-binary

bb4242 commented 9 years ago

I can verify that switching to 0.6.11 fixes the problem for me. Thanks! :)

On Sun, Jan 25, 2015 at 12:55 AM, Anton Lavrik notifications@github.com wrote:

Should be fixed now in piqi-0.6.11 -- there were a couple of Windows compatibility problems. Can you verify it is working? piqi-erlang is pointed to the newest piqi-binary

— Reply to this email directly or view it on GitHub https://github.com/alavrik/piqi-erlang/issues/16#issuecomment-71361098.

alavrik / piqi-erlang

Conversion to JSON fails on Windows #16