search

insightsengineering / sasr

sasr
https://insightsengineering.github.io/sasr/
15 stars 2 forks source link

[Question]: table return values #48

Closed kkmann closed 2 months ago

kkmann commented 2 months ago

What is your question?

Hey,

I recently gave saspy and sasr a shot and ran into a problem with retrieving pandas data frames correctly, see log below.

Any idea of how to address this? Somehow the conversion of the returned pandas df through reticulate seems to fail.

Using saspy directly form python works :(

R version 4.3.2 (2023-10-31) -- "Eye Holes"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> reticulate::use_virtualenv("r-reticulate")
> # reticulate::virtualenv_install(packages = "saspy==5.100.2")
> # reticulate::virtualenv_install(packages = "pandas==2.2.2")
> # remotes::install_github(repo = 'insightsengineering/sasr@v0.1.2')
> 
> library(sasr)
> 
> sasr::get_sas_session()
Using SAS Config named: ssh
SAS Connection established. Subprocess id is 625659

Access Method         = SSH
SAS Config name       = ssh
SAS Config file       = sascfg_personal.py
WORK Path             = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
SAS Version           = 9.04.01M7P08062020
SASPy Version         = 5.100.2
Teach me SAS          = False
Batch                 = False
Results               = Pandas
SAS Session Encoding  = latin1
Python Encoding value = latin_1
SAS process Pid value = 1834400

> 
> mtcars |>
+   tibble::rownames_to_column() |>
+   df2sd("mt")
Libref  = WORK
Table   = mt
Dsopts  = {}
Results = Pandas

> 
> result <- run_sas("
+   proc freq data = mt;
+   run;
+ ")
> cat(result$LOG)

65   
66   
67     proc freq data = mt;
68     run;
NOTE: There were 32 observations read from the data set WORK.MT.
NOTE: The PROCEDURE FREQ printed page 1.
NOTE: PROCEDURE FREQ used (Total process time):
      real time           0.01 seconds
      cpu time            0.01 seconds

69   
70   > cat(result$LST)
                                                                                                                         The SAS System                                                                                   12:25 Thursday, September 12, 2024   1

                                                                                                                       The FREQ Procedure

                                                                                                                                            Cumulative    Cumulative
                                                                                            rowname                Frequency     Percent     Frequency      Percent
                                                                                            ------------------------------------------------------------------------
                                                                                            AMC Javelin                   1        3.13             1         3.13  
                                                                                            Cadillac Fleetwood            1        3.13             2         6.25  
                                                                                            Camaro Z28                    1        3.13             3         9.38  
                                                                                            Chrysler Imperial             1        3.13             4        12.50  
                                                                                            Datsun 710                    1        3.13             5        15.63  
                                                                                            Dodge Challenger              1        3.13             6        18.75  
                                                                                            Duster 360                    1        3.13             7        21.88  
                                                                                            Ferrari Dino                  1        3.13             8        25.00  
                                                                                            Fiat 128                      1        3.13             9        28.13  
                                                                                            Fiat X1-9                     1        3.13            10        31.25  
                                                                                            Ford Pantera L                1        3.13            11        34.38  
                                                                                            Honda Civic                   1        3.13            12        37.50  
                                                                                            Hornet 4 Drive                1        3.13            13        40.63  
                                                                                            Hornet Sportabout             1        3.13            14        43.75  
                                                                                            Lincoln Continental           1        3.13            15        46.88  
                                                                                            Lotus Europa                  1        3.13            16        50.00  
                                                                                            Maserati Bora                 1        3.13            17        53.13  
                                                                                            Mazda RX4                     1        3.13            18        56.25  
                                                                                            Mazda RX4 Wag                 1        3.13            19        59.38  
                                                                                            Merc 230                      1        3.13            20        62.50  
                                                                                            Merc 240D                     1        3.13            21        65.63  
                                                                                            Merc 280                      1        3.13            22        68.75  
                                                                                            Merc 280C                     1        3.13            23        71.88  
                                                                                            Merc 450SE                    1        3.13            24        75.00  
                                                                                            Merc 450SL                    1        3.13            25        78.13  
                                                                                            Merc 450SLC                   1        3.13            26        81.25  
                                                                                            Pontiac Firebird              1        3.13            27        84.38  
                                                                                            Porsche 914-2                 1        3.13            28        87.50  
                                                                                            Toyota Corolla                1        3.13            29        90.63  
                                                                                            Toyota Corona                 1        3.13            30        93.75  
                                                                                            Valiant                       1        3.13            31        96.88  
                                                                                            Volvo 142E                    1        3.13            32       100.00  

                                                                                                                                    Cumulative    Cumulative
                                                                                                    mpg    Frequency     Percent     Frequency      Percent
                                                                                                   ---------------------------------------------------------
                                                                                                   10.4           2        6.25             2         6.25  
                                                                                                   13.3           1        3.13             3         9.38  
                                                                                                   14.3           1        3.13             4        12.50  
                                                                                                   14.7           1        3.13             5        15.63  
                                                                                                     15           1        3.13             6        18.75  
                                                                                                   15.2           2        6.25             8        25.00  
                                                                                                   15.5           1        3.13             9        28.13  
                                                                                                   15.8           1        3.13            10        31.25  
                                                                                                   16.4           1        3.13            11        34.38  
                                                                                                   17.3           1        3.13            12        37.50  
                                                                                                   17.8           1        3.13            13        40.63  
                                                                                                   18.1           1        3.13            14        43.75  
                                                                                                   18.7           1        3.13            15        46.88  
                                                                                                   19.2           2        6.25            17        53.13  
                                                                                                   19.7           1        3.13            18        56.25  
                                                                                                     21           2        6.25            20        62.50  
                                                                                                   21.4           2        6.25            22        68.75  
                                                                                                   21.5           1        3.13            23        71.88  
                                                                                                   22.8           2        6.25            25        78.13  
                                                                                                   24.4           1        3.13            26        81.25  
                                                                                                     26           1        3.13            27        84.38  
                                                                                                   27.3           1        3.13            28        87.50  
                                                                                                   30.4           2        6.25            30        93.75  
                                                                                                   32.4           1        3.13            31        96.88  
                                                                                                   33.9           1        3.13            32       100.00  

                                                                                                                                    Cumulative    Cumulative
                                                                                                    cyl    Frequency     Percent     Frequency      Percent
                                                                                                    --------------------------------------------------------
                                                                                                      4          11       34.38            11        34.38  
                                                                                                      6           7       21.88            18        56.25  
                                                                                                      8          14       43.75            32       100.00  

                                                                                                                                     Cumulative    Cumulative
                                                                                                    disp    Frequency     Percent     Frequency      Percent
                                                                                                   ----------------------------------------------------------
                                                                                                    71.1           1        3.13             1         3.13  
                                                                                                    75.7           1        3.13             2         6.25  
                                                                                                    78.7           1        3.13             3         9.38  
                                                                                                      79           1        3.13             4        12.50  
                                                                                                    95.1           1        3.13             5        15.63  
                                                                                                     108           1        3.13             6        18.75  
                                                                                                   120.1           1        3.13             7        21.88  
                                                                                                   120.3           1        3.13             8        25.00  
                                                                                                     121           1        3.13             9        28.13  
                                                                                                   140.8           1        3.13            10        31.25  
                                                                                                     145           1        3.13            11        34.38  
                                                                                                   146.7           1        3.13            12        37.50  
                                                                                                     160           2        6.25            14        43.75  
                                                                                                   167.6           2        6.25            16        50.00  
                                                                                                     225           1        3.13            17        53.13  
                                                                                                     258           1        3.13            18        56.25  
                                                                                                   275.8           3        9.38            21        65.63  
                                                                                                     301           1        3.13            22        68.75  
                                                                                                     304           1        3.13            23        71.88  
                                                                                                     318           1        3.13            24        75.00  
                                                                                                     350           1        3.13            25        78.13  
                                                                                                     351           1        3.13            26        81.25  
                                                                                                     360           2        6.25            28        87.50  
                                                                                                     400           1        3.13            29        90.63  
                                                                                                     440           1        3.13            30        93.75  
                                                                                                     460           1        3.13            31        96.88  
                                                                                                     472           1        3.13            32       100.00  

                                                                                                                                    Cumulative    Cumulative
                                                                                                     hp    Frequency     Percent     Frequency      Percent
                                                                                                    --------------------------------------------------------
                                                                                                     52           1        3.13             1         3.13  
                                                                                                     62           1        3.13             2         6.25  
                                                                                                     65           1        3.13             3         9.38  
                                                                                                     66           2        6.25             5        15.63  
                                                                                                     91           1        3.13             6        18.75  
                                                                                                     93           1        3.13             7        21.88  
                                                                                                     95           1        3.13             8        25.00  
                                                                                                     97           1        3.13             9        28.13  
                                                                                                    105           1        3.13            10        31.25  
                                                                                                    109           1        3.13            11        34.38  
                                                                                                    110           3        9.38            14        43.75  
                                                                                                    113           1        3.13            15        46.88  
                                                                                                    123           2        6.25            17        53.13  
                                                                                                    150           2        6.25            19        59.38  
                                                                                                    175           3        9.38            22        68.75  
                                                                                                    180           3        9.38            25        78.13  
                                                                                                    205           1        3.13            26        81.25  
                                                                                                    215           1        3.13            27        84.38  
                                                                                                    230           1        3.13            28        87.50  
                                                                                                    245           2        6.25            30        93.75  
                                                                                                    264           1        3.13            31        96.88  
                                                                                                    335           1        3.13            32       100.00  

                                                                                                                                    Cumulative    Cumulative
                                                                                                   drat    Frequency     Percent     Frequency      Percent
                                                                                                   ---------------------------------------------------------
                                                                                                   2.76           2        6.25             2         6.25  
                                                                                                   2.93           1        3.13             3         9.38  
                                                                                                      3           1        3.13             4        12.50  
                                                                                                   3.07           3        9.38             7        21.88  
                                                                                                   3.08           2        6.25             9        28.13  
                                                                                                   3.15           2        6.25            11        34.38  
                                                                                                   3.21           1        3.13            12        37.50  
                                                                                                   3.23           1        3.13            13        40.63  
                                                                                                   3.54           1        3.13            14        43.75  
                                                                                                   3.62           1        3.13            15        46.88  
                                                                                                   3.69           1        3.13            16        50.00  
                                                                                                    3.7           1        3.13            17        53.13  
                                                                                                   3.73           1        3.13            18        56.25  
                                                                                                   3.77           1        3.13            19        59.38  
                                                                                                   3.85           1        3.13            20        62.50  
                                                                                                    3.9           2        6.25            22        68.75  
                                                                                                   3.92           3        9.38            25        78.13  
                                                                                                   4.08           2        6.25            27        84.38  
                                                                                                   4.11           1        3.13            28        87.50  
                                                                                                   4.22           2        6.25            30        93.75  
                                                                                                   4.43           1        3.13            31        96.88  
                                                                                                   4.93           1        3.13            32       100.00  

                                                                                                                                     Cumulative    Cumulative
                                                                                                      wt    Frequency     Percent     Frequency      Percent
                                                                                                   ----------------------------------------------------------
                                                                                                   1.513           1        3.13             1         3.13  
                                                                                                   1.615           1        3.13             2         6.25  
                                                                                                   1.835           1        3.13             3         9.38  
                                                                                                   1.935           1        3.13             4        12.50  
                                                                                                    2.14           1        3.13             5        15.63  
                                                                                                     2.2           1        3.13             6        18.75  
                                                                                                    2.32           1        3.13             7        21.88  
                                                                                                   2.465           1        3.13             8        25.00  
                                                                                                    2.62           1        3.13             9        28.13  
                                                                                                    2.77           1        3.13            10        31.25  
                                                                                                    2.78           1        3.13            11        34.38  
                                                                                                   2.875           1        3.13            12        37.50  
                                                                                                    3.15           1        3.13            13        40.63  
                                                                                                    3.17           1        3.13            14        43.75  
                                                                                                    3.19           1        3.13            15        46.88  
                                                                                                   3.215           1        3.13            16        50.00  
                                                                                                   3.435           1        3.13            17        53.13  
                                                                                                    3.44           3        9.38            20        62.50  
                                                                                                    3.46           1        3.13            21        65.63  
                                                                                                    3.52           1        3.13            22        68.75  
                                                                                                    3.57           2        6.25            24        75.00  
                                                                                                    3.73           1        3.13            25        78.13  
                                                                                                    3.78           1        3.13            26        81.25  
                                                                                                    3.84           1        3.13            27        84.38  
                                                                                                   3.845           1        3.13            28        87.50  
                                                                                                    4.07           1        3.13            29        90.63  
                                                                                                    5.25           1        3.13            30        93.75  
                                                                                                   5.345           1        3.13            31        96.88  
                                                                                                   5.424           1        3.13            32       100.00  

                                                                                                                                     Cumulative    Cumulative
                                                                                                    qsec    Frequency     Percent     Frequency      Percent
                                                                                                   ----------------------------------------------------------
                                                                                                    14.5           1        3.13             1         3.13  
                                                                                                    14.6           1        3.13             2         6.25  
                                                                                                   15.41           1        3.13             3         9.38  
                                                                                                    15.5           1        3.13             4        12.50  
                                                                                                   15.84           1        3.13             5        15.63  
                                                                                                   16.46           1        3.13             6        18.75  
                                                                                                    16.7           1        3.13             7        21.88  
                                                                                                   16.87           1        3.13             8        25.00  
                                                                                                    16.9           1        3.13             9        28.13  
                                                                                                   17.02           2        6.25            11        34.38  
                                                                                                   17.05           1        3.13            12        37.50  
                                                                                                    17.3           1        3.13            13        40.63  
                                                                                                    17.4           1        3.13            14        43.75  
                                                                                                   17.42           1        3.13            15        46.88  
                                                                                                    17.6           1        3.13            16        50.00  
                                                                                                   17.82           1        3.13            17        53.13  
                                                                                                   17.98           1        3.13            18        56.25  
                                                                                                      18           1        3.13            19        59.38  
                                                                                                    18.3           1        3.13            20        62.50  
                                                                                                   18.52           1        3.13            21        65.63  
                                                                                                    18.6           1        3.13            22        68.75  
                                                                                                   18.61           1        3.13            23        71.88  
                                                                                                    18.9           2        6.25            25        78.13  
                                                                                                   19.44           1        3.13            26        81.25  
                                                                                                   19.47           1        3.13            27        84.38  
                                                                                                    19.9           1        3.13            28        87.50  
                                                                                                      20           1        3.13            29        90.63  
                                                                                                   20.01           1        3.13            30        93.75  
                                                                                                   20.22           1        3.13            31        96.88  
                                                                                                    22.9           1        3.13            32       100.00  

                                                                                                                                   Cumulative    Cumulative
                                                                                                    vs    Frequency     Percent     Frequency      Percent
                                                                                                    -------------------------------------------------------
                                                                                                     0          18       56.25            18        56.25  
                                                                                                     1          14       43.75            32       100.00  

                                                                                                                                   Cumulative    Cumulative
                                                                                                    am    Frequency     Percent     Frequency      Percent
                                                                                                    -------------------------------------------------------
                                                                                                     0          19       59.38            19        59.38  
                                                                                                     1          13       40.63            32       100.00  

                                                                                                                                    Cumulative    Cumulative
                                                                                                   gear    Frequency     Percent     Frequency      Percent
                                                                                                   ---------------------------------------------------------
                                                                                                      3          15       46.88            15        46.88  
                                                                                                      4          12       37.50            27        84.38  
                                                                                                      5           5       15.63            32       100.00  

                                                                                                                                    Cumulative    Cumulative
                                                                                                   carb    Frequency     Percent     Frequency      Percent
                                                                                                   ---------------------------------------------------------
                                                                                                      1           7       21.88             7        21.88  
                                                                                                      2          10       31.25            17        53.13  
                                                                                                      3           3        9.38            20        62.50  
                                                                                                      4          10       31.25            30        93.75  
                                                                                                      6           1        3.13            31        96.88  
                                                                                                      8           1        3.13            32       100.00  > 
> result <- run_sas("
+   ods output ParameterEstimates = parms;
+   proc reg data = mt;
+       model mpg = cyl;
+   run;
+ ")
> 
> sd2df("parms")
sys:1: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.
                Model     Dependent            Variable      DF
1 ['MODEL1' 'MODEL1'] ['mpg' 'mpg'] ['Intercept' 'cyl'] [1. 1.]
2                <NA>          <NA>                <NA>    <NA>
                   Estimate                  StdErr                    tValue
1 [37.88457649 -2.87579014] [2.07384361 0.32240888] [18.26780784 -8.91969885]
2                      <NA>                    <NA>                      <NA>
                            Probt
1 [8.36915530e-18 6.11268714e-10]
2                            <NA>
Warning message:
In format.data.frame(if (omit) x[seq_len(n0), , drop = FALSE] else x,  :
  corrupt data frame: columns will be truncated or padded with NAs
> 
> proc.time()
   user  system elapsed 
  1.996   0.307   7.312

Code of Conduct

Contribution Guidelines

Security Policy

kkmann commented 2 months ago

x) turns out I was using an outdated reticulate version and didn't realize it. updating that fixes the issue, sry