to.casDataFrame returns json parsing error

[yennguyen248]

Hi, I'm trying to bring a CAS table into R Studio using RSWAT. The table has 382k rows and 35 columns. However, when I tried to use (obs=) in the statement, I encountered a Json error. Please see the attached file. When I removed obs=, the code ran successfully but only returned 32k instead of 382k rows for me which is not desirable. Is there any advice on how to fix this problem?

[kesmit13]

Can you wrap that code in the following? I'm not sure what would cause this and I'd like to see what the HTML is that's coming back.

httr::with_verbose( <your code here>, data_out=TRUE, data_in=TRUE, info=TRUE, ssl=TRUE)

[yennguyen248]

Hi Kevin, I got the same error even after wrapping the code in verbose.

Yen Nguyen

[kesmit13]

I guess I assumed that you were connecting to CAS using the HTTP interface, is that correct? If so, are you connecting directly to the CAS HTTP server, or is this in kubernetes with an ingress?

[yennguyen248]

I'm actually connecting to a Viya 3.4 environment (hosted by SAS)

[kesmit13]

I'm puzzled as to why the httr::with_verbose isn't showing anything. That should display the HTTP request and the full response. I have a feeling the server is failing somewhere and the proxy front end is returning an HTML response with the error message. Typically, httr::with_verbose would display the information. Just out of curiosity, what happens if you remove the casdf= part of the expression.

[yennguyen248]

That makes sense. What puzzled me though is that if I reduce obs to a small number (100 for example), then the code ran successfully. So maybe it has something to do with the data size?

[kesmit13]

You can try hitting the REST endpoint directly with something like curl as follows:

curl -X POST -u username:password -d '{"table":"<table-name>", "from":1, "to":100}' -H 'Content-Type: application/json' <server-url>/cas/sessions/<session-id>/actions/table.fetch

This is the same thing that the R client is doing. If you change the to parameter to a large number, maybe you can get the HTML output. BTW, you can get your session ID by printing the connection object in R.

[yennguyen248]

Hi Kevin,

Thank you so much for your instructions. I'm not so familiar with using REST though I will give it a try. However, I tried the same code outside the customer environment and was unable to reproduce her error. Would it be something to do with the connection? If so, is there any way to check it?

[kesmit13]

Typically, I just use the httr::with_verbose wrapper around whatever call is causing the problem and it shows you all of the HTTP requests in raw form. I don't know why it isn't working in your setup. I tried doing a very similar line of code on my machine and it does show all of the HTTP calls. If we could get that information to display, it would help a lot.

[yennguyen248]

Hi Kevin, I omitted casdf in httr::with_verbose wrapper and am now able to get the log out. Please see the attached file.

I hope this helps to troubleshoot what's gone wrong during the connection.

Thanks, YN

[kesmit13]

I've seen 502 errors in testing, but I've only seen them if the CAS controller goes down and the backup has to take over. I'm pretty sure that's not happening here. Unfortunately, it still isn't displaying the non-JSON result. Is there any chance you can get the server logs?

[yennguyen248]

I'm going to reach out to the environment admin to ask for the log now. This error seems to be consistent when I added obs. However, if I only got to.casDataFrame(casObj), then sometimes I got the error, sometimes not. This makes me think it might be a connection failure :(.

[kesmit13]

The log you sent shows a `HTTP/1.1 502 Proxy Error' message in it. So I'm still thinking there is either some error from the server, or the proxy itself is failing in some fashion.