Limit of 110 lines on set-data-frame?

[edmizac]

Dear SR developers,

Following the guidance on an issue I have opened about NetLogo R extension, I have started adapting my code to the SR extension. However, my model was freezing as soon as I would use set-data-frame

Thus, I developed another (but simpler) program that reproduces my original model steps and schedule. The same bug was happening after lists of more than 111 entries/steps (controled by a slider) were being assigned to a dataframe with the set-data-frame function. It works up to 110, but it crashes after 111.

You can find the model attached here or check the .txt code below.

Thank you.

PS: In my original model with the R extension I was using r:putdataframe, which managed lists pretty ok. I have also tried other functions like sr:set-agent-data-frame of SimpleR but I get the same freezing problem.

extensions [sr  ]

breed [ monkeys monkey ]

monkeys-own [
  X_coords
  Y_coords
  day_list
  Name
]

to setupSR
  sr:setup
end

to test-calc-homerange
  ca
  create-monkeys 10
  ask monkeys [
    set X_coords []
    set Y_coords []
    set day_list []
    set Name [who] of self
  ]

  ask monkeys [
    repeat n_steps [
      move-to one-of patches
      set X_coords lput ( [xcor] of self ) X_coords
      set Y_coords lput [ycor] of self Y_coords
      set day_list lput 1 day_list
    ]
  ]

  ask one-of monkeys [
    print "==== SR debugging ==== "

;    set X_coords [ 0 2 3 5 6 ]
;    set Y_coords [ 1 2 3 4 5 ]
;    set day_list [ 1 2 3 4 5 ]

    let X_coords_sr [X_coords] of self    print X_coords_sr  ;print length X_coords_sr
    let Y_coords_sr [Y_coords] of self    print Y_coords_sr  ;print length Y_coords_sr
    let day_list_sr [day_list] of self    print day_list_sr  ;print length day_list_sr
    let Name_sr ( [Name] of self )            print Name_sr  ;print length Name_sr

    print "==== SR debugging 2 ==== "

;          stop

    (sr:set-data-frame "df1" "X_coords" X_coords_sr "Y_coords" Y_coords_sr "day_list" day_list_sr)
    sr:run "print(df1)"

    sr:set "Name_sr" Name_sr
    sr:run "df1 <- data.frame(df1, Name = Name_sr)"
    sr:run "print(df1)"

  ]

end
@#$#@#$#@
GRAPHICS-WINDOW
210
10
647
448
-1
-1
13.0
1
10
1
1
1
0
1
1
1
-16
16
-16
16
0
0
1
ticks
30.0

BUTTON
85
102
163
135
NIL
setupSR
NIL
1
T
OBSERVER
NIL
NIL
NIL
NIL
1

BUTTON
87
152
161
185
NIL
clear-all
NIL
1
T
OBSERVER
NIL
NIL
NIL
NIL
1

BUTTON
60
214
255
247
NIL
setupSR\ntest-calc-homerange
NIL
1
T
OBSERVER
NIL
NIL
NIL
NIL
1

SLIDER
64
268
236
301
n_steps
n_steps
1
200
105.0
1
1
NIL
HORIZONTAL

@#$#@#$#@
## WHAT IS IT?

(a general understanding of what the model is trying to show or explain)

## HOW IT WORKS

(what rules the agents use to create the overall behavior of the model)

## HOW TO USE IT

(how to use the model, including a description of each of the items in the Interface tab)

## THINGS TO NOTICE

(suggested things for the user to notice while running the model)

## THINGS TO TRY

(suggested things for the user to try to do (move sliders, switches, etc.) with the model)

## EXTENDING THE MODEL

(suggested things to add or change in the Code tab to make the model more complicated, detailed, accurate, etc.)

## NETLOGO FEATURES

(interesting or unusual features of NetLogo that the model uses, particularly in the Code tab; or where workarounds were needed for missing features)

## RELATED MODELS

(models in the NetLogo Models Library and elsewhere which are of related interest)

## CREDITS AND REFERENCES

(a reference to the model's URL on the web if it has one, as well as any other necessary credits, citations, and links)
@#$#@#$#@
default
true
0
Polygon -7500403 true true 150 5 40 250 150 205 260 250

airplane
true
0
Polygon -7500403 true true 150 0 135 15 120 60 120 105 15 165 15 195 120 180 135 240 105 270 120 285 150 270 180 285 210 270 165 240 180 180 285 195 285 165 180 105 180 60 165 15

arrow
true
0
Polygon -7500403 true true 150 0 0 150 105 150 105 293 195 293 195 150 300 150

box
false
0
Polygon -7500403 true true 150 285 285 225 285 75 150 135
Polygon -7500403 true true 150 135 15 75 150 15 285 75
Polygon -7500403 true true 15 75 15 225 150 285 150 135
Line -16777216 false 150 285 150 135
Line -16777216 false 150 135 15 75
Line -16777216 false 150 135 285 75

bug
true
0
Circle -7500403 true true 96 182 108
Circle -7500403 true true 110 127 80
Circle -7500403 true true 110 75 80
Line -7500403 true 150 100 80 30
Line -7500403 true 150 100 220 30

butterfly
true
0
Polygon -7500403 true true 150 165 209 199 225 225 225 255 195 270 165 255 150 240
Polygon -7500403 true true 150 165 89 198 75 225 75 255 105 270 135 255 150 240
Polygon -7500403 true true 139 148 100 105 55 90 25 90 10 105 10 135 25 180 40 195 85 194 139 163
Polygon -7500403 true true 162 150 200 105 245 90 275 90 290 105 290 135 275 180 260 195 215 195 162 165
Polygon -16777216 true false 150 255 135 225 120 150 135 120 150 105 165 120 180 150 165 225
Circle -16777216 true false 135 90 30
Line -16777216 false 150 105 195 60
Line -16777216 false 150 105 105 60

car
false
0
Polygon -7500403 true true 300 180 279 164 261 144 240 135 226 132 213 106 203 84 185 63 159 50 135 50 75 60 0 150 0 165 0 225 300 225 300 180
Circle -16777216 true false 180 180 90
Circle -16777216 true false 30 180 90
Polygon -16777216 true false 162 80 132 78 134 135 209 135 194 105 189 96 180 89
Circle -7500403 true true 47 195 58
Circle -7500403 true true 195 195 58

circle
false
0
Circle -7500403 true true 0 0 300

circle 2
false
0
Circle -7500403 true true 0 0 300
Circle -16777216 true false 30 30 240

cow
false
0
Polygon -7500403 true true 200 193 197 249 179 249 177 196 166 187 140 189 93 191 78 179 72 211 49 209 48 181 37 149 25 120 25 89 45 72 103 84 179 75 198 76 252 64 272 81 293 103 285 121 255 121 242 118 224 167
Polygon -7500403 true true 73 210 86 251 62 249 48 208
Polygon -7500403 true true 25 114 16 195 9 204 23 213 25 200 39 123

cylinder
false
0
Circle -7500403 true true 0 0 300

dot
false
0
Circle -7500403 true true 90 90 120

face happy
false
0
Circle -7500403 true true 8 8 285
Circle -16777216 true false 60 75 60
Circle -16777216 true false 180 75 60
Polygon -16777216 true false 150 255 90 239 62 213 47 191 67 179 90 203 109 218 150 225 192 218 210 203 227 181 251 194 236 217 212 240

face neutral
false
0
Circle -7500403 true true 8 7 285
Circle -16777216 true false 60 75 60
Circle -16777216 true false 180 75 60
Rectangle -16777216 true false 60 195 240 225

face sad
false
0
Circle -7500403 true true 8 8 285
Circle -16777216 true false 60 75 60
Circle -16777216 true false 180 75 60
Polygon -16777216 true false 150 168 90 184 62 210 47 232 67 244 90 220 109 205 150 198 192 205 210 220 227 242 251 229 236 206 212 183

fish
false
0
Polygon -1 true false 44 131 21 87 15 86 0 120 15 150 0 180 13 214 20 212 45 166
Polygon -1 true false 135 195 119 235 95 218 76 210 46 204 60 165
Polygon -1 true false 75 45 83 77 71 103 86 114 166 78 135 60
Polygon -7500403 true true 30 136 151 77 226 81 280 119 292 146 292 160 287 170 270 195 195 210 151 212 30 166
Circle -16777216 true false 215 106 30

flag
false
0
Rectangle -7500403 true true 60 15 75 300
Polygon -7500403 true true 90 150 270 90 90 30
Line -7500403 true 75 135 90 135
Line -7500403 true 75 45 90 45

flower
false
0
Polygon -10899396 true false 135 120 165 165 180 210 180 240 150 300 165 300 195 240 195 195 165 135
Circle -7500403 true true 85 132 38
Circle -7500403 true true 130 147 38
Circle -7500403 true true 192 85 38
Circle -7500403 true true 85 40 38
Circle -7500403 true true 177 40 38
Circle -7500403 true true 177 132 38
Circle -7500403 true true 70 85 38
Circle -7500403 true true 130 25 38
Circle -7500403 true true 96 51 108
Circle -16777216 true false 113 68 74
Polygon -10899396 true false 189 233 219 188 249 173 279 188 234 218
Polygon -10899396 true false 180 255 150 210 105 210 75 240 135 240

house
false
0
Rectangle -7500403 true true 45 120 255 285
Rectangle -16777216 true false 120 210 180 285
Polygon -7500403 true true 15 120 150 15 285 120
Line -16777216 false 30 120 270 120

leaf
false
0
Polygon -7500403 true true 150 210 135 195 120 210 60 210 30 195 60 180 60 165 15 135 30 120 15 105 40 104 45 90 60 90 90 105 105 120 120 120 105 60 120 60 135 30 150 15 165 30 180 60 195 60 180 120 195 120 210 105 240 90 255 90 263 104 285 105 270 120 285 135 240 165 240 180 270 195 240 210 180 210 165 195
Polygon -7500403 true true 135 195 135 240 120 255 105 255 105 285 135 285 165 240 165 195

line
true
0
Line -7500403 true 150 0 150 300

line half
true
0
Line -7500403 true 150 0 150 150

pentagon
false
0
Polygon -7500403 true true 150 15 15 120 60 285 240 285 285 120

person
false
0
Circle -7500403 true true 110 5 80
Polygon -7500403 true true 105 90 120 195 90 285 105 300 135 300 150 225 165 300 195 300 210 285 180 195 195 90
Rectangle -7500403 true true 127 79 172 94
Polygon -7500403 true true 195 90 240 150 225 180 165 105
Polygon -7500403 true true 105 90 60 150 75 180 135 105

plant
false
0
Rectangle -7500403 true true 135 90 165 300
Polygon -7500403 true true 135 255 90 210 45 195 75 255 135 285
Polygon -7500403 true true 165 255 210 210 255 195 225 255 165 285
Polygon -7500403 true true 135 180 90 135 45 120 75 180 135 210
Polygon -7500403 true true 165 180 165 210 225 180 255 120 210 135
Polygon -7500403 true true 135 105 90 60 45 45 75 105 135 135
Polygon -7500403 true true 165 105 165 135 225 105 255 45 210 60
Polygon -7500403 true true 135 90 120 45 150 15 180 45 165 90

sheep
false
15
Circle -1 true true 203 65 88
Circle -1 true true 70 65 162
Circle -1 true true 150 105 120
Polygon -7500403 true false 218 120 240 165 255 165 278 120
Circle -7500403 true false 214 72 67
Rectangle -1 true true 164 223 179 298
Polygon -1 true true 45 285 30 285 30 240 15 195 45 210
Circle -1 true true 3 83 150
Rectangle -1 true true 65 221 80 296
Polygon -1 true true 195 285 210 285 210 240 240 210 195 210
Polygon -7500403 true false 276 85 285 105 302 99 294 83
Polygon -7500403 true false 219 85 210 105 193 99 201 83

square
false
0
Rectangle -7500403 true true 30 30 270 270

square 2
false
0
Rectangle -7500403 true true 30 30 270 270
Rectangle -16777216 true false 60 60 240 240

star
false
0
Polygon -7500403 true true 151 1 185 108 298 108 207 175 242 282 151 216 59 282 94 175 3 108 116 108

target
false
0
Circle -7500403 true true 0 0 300
Circle -16777216 true false 30 30 240
Circle -7500403 true true 60 60 180
Circle -16777216 true false 90 90 120
Circle -7500403 true true 120 120 60

tree
false
0
Circle -7500403 true true 118 3 94
Rectangle -6459832 true false 120 195 180 300
Circle -7500403 true true 65 21 108
Circle -7500403 true true 116 41 127
Circle -7500403 true true 45 90 120
Circle -7500403 true true 104 74 152

triangle
false
0
Polygon -7500403 true true 150 30 15 255 285 255

triangle 2
false
0
Polygon -7500403 true true 150 30 15 255 285 255
Polygon -16777216 true false 151 99 225 223 75 224

truck
false
0
Rectangle -7500403 true true 4 45 195 187
Polygon -7500403 true true 296 193 296 150 259 134 244 104 208 104 207 194
Rectangle -1 true false 195 60 195 105
Polygon -16777216 true false 238 112 252 141 219 141 218 112
Circle -16777216 true false 234 174 42
Rectangle -7500403 true true 181 185 214 194
Circle -16777216 true false 144 174 42
Circle -16777216 true false 24 174 42
Circle -7500403 false true 24 174 42
Circle -7500403 false true 144 174 42
Circle -7500403 false true 234 174 42

turtle
true
0
Polygon -10899396 true false 215 204 240 233 246 254 228 266 215 252 193 210
Polygon -10899396 true false 195 90 225 75 245 75 260 89 269 108 261 124 240 105 225 105 210 105
Polygon -10899396 true false 105 90 75 75 55 75 40 89 31 108 39 124 60 105 75 105 90 105
Polygon -10899396 true false 132 85 134 64 107 51 108 17 150 2 192 18 192 52 169 65 172 87
Polygon -10899396 true false 85 204 60 233 54 254 72 266 85 252 107 210
Polygon -7500403 true true 119 75 179 75 209 101 224 135 220 225 175 261 128 261 81 224 74 135 88 99

wheel
false
0
Circle -7500403 true true 3 3 294
Circle -16777216 true false 30 30 240
Line -7500403 true 150 285 150 15
Line -7500403 true 15 150 285 150
Circle -7500403 true true 120 120 60
Line -7500403 true 216 40 79 269
Line -7500403 true 40 84 269 221
Line -7500403 true 40 216 269 79
Line -7500403 true 84 40 221 269

wolf
false
0
Polygon -16777216 true false 253 133 245 131 245 133
Polygon -7500403 true true 2 194 13 197 30 191 38 193 38 205 20 226 20 257 27 265 38 266 40 260 31 253 31 230 60 206 68 198 75 209 66 228 65 243 82 261 84 268 100 267 103 261 77 239 79 231 100 207 98 196 119 201 143 202 160 195 166 210 172 213 173 238 167 251 160 248 154 265 169 264 178 247 186 240 198 260 200 271 217 271 219 262 207 258 195 230 192 198 210 184 227 164 242 144 259 145 284 151 277 141 293 140 299 134 297 127 273 119 270 105
Polygon -7500403 true true -1 195 14 180 36 166 40 153 53 140 82 131 134 133 159 126 188 115 227 108 236 102 238 98 268 86 269 92 281 87 269 103 269 113

x
false
0
Polygon -7500403 true true 270 75 225 30 30 225 75 270
Polygon -7500403 true true 30 75 75 30 270 225 225 270
@#$#@#$#@
NetLogo 6.2.2
@#$#@#$#@
@#$#@#$#@
@#$#@#$#@
@#$#@#$#@
@#$#@#$#@
default
0.0
-0.2 0 0.0 1.0
0.0 1 1.0 0.0
0.2 0 0.0 1.0
link direction
true
0
Line -7500403 true 150 150 90 180
Line -7500403 true 150 150 210 180
@#$#@#$#@
0
@#$#@#$#@

[JamesHovet]

Thank you for your detailed steps to reproduce, @edmizac

I ran your model on my own machine however and did not face any issues, even with the slider set all the way to 200.

Can you share some more information about your machine? And can you also paste the error message (if any?) that you encounter?

For your reference, my machine is an intel macbook running macOS 13.3.1 and R version 4.3.1 (2023-06-16).

[edmizac]

Thank you James for the fast response.

That is weird. I could reproduce the error in other two machines (three machines in total, all of them with Windows). My co-advisor was able to run it on Linux, though.

Actually, it only freezes. No error is thrown.

My machine: ------

NetLogo 6.2.2 Windows 10 Home

Device name DESKTOP-R12V3D6 Processor Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz 1.99 GHz Installed RAM 16,0 GB (15,9 GB usable) Device ID 9E9736B3-A40B-43F1-A68E-CAD8A03966D1 Product ID 00326-10000-00000-AA961 System type 64-bit operating system, x64-based processor

> sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=Portuguese_Brazil.utf8  LC_CTYPE=Portuguese_Brazil.utf8   
[3] LC_MONETARY=Portuguese_Brazil.utf8 LC_NUMERIC=C                      
[5] LC_TIME=Portuguese_Brazil.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.2.1 tools_4.2.1 

Machine 2: -----

NetLogo 6.3 Windows 10 Pro

DESKTOP-9PSSDHR 12th Gen Intel(R) Core(TM) i9-12900KF 3.20 GHz RAM 64,0 GB (63,8 GB) Devide ID 3BE65676-C7BC-4A81-B074-4DBC649C2562 Product ID 00331-10000-00001-AA655 System type 64 bits (x64)

> sessionInfo()
R version 4.3.1 (2023-06-16 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=Portuguese_Brazil.utf8 
[2] LC_CTYPE=Portuguese_Brazil.utf8   
[3] LC_MONETARY=Portuguese_Brazil.utf8
[4] LC_NUMERIC=C                      
[5] LC_TIME=Portuguese_Brazil.utf8    

time zone: America/Sao_Paulo
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets 
[6] methods   base     

loaded via a namespace (and not attached):
 [1] digest_0.6.31    progressr_0.13.0 utf8_1.2.3      
 [4] R6_2.5.1         tidyselect_1.2.0 magrittr_2.0.3  
 [7] glue_1.6.2       stringr_1.5.0    tictoc_1.2      
[10] tibble_3.2.1     parallel_4.3.1   pkgconfig_2.0.3 
[13] dplyr_1.1.2      generics_0.1.3   lifecycle_1.0.3 
[16] cli_3.6.1        fansi_1.0.4      vctrs_0.6.3     
[19] compiler_4.3.1   rstudioapi_0.14  tools_4.3.1     
[22] pillar_1.9.0     rlang_1.1.1      stringi_1.7.12 

Machine 3: ------

NetLogo 6.3 Windows 10 Home

DESKTOP-71SL58S Processor Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz 3.40 GHz RAM 32,0 GB ID 6A58377A-C9CD-47B9-B046-B51368B95965 Product ID 00326-10000-00000-AA483 System 64 bits (x64)

> sessionInfo()
R version 4.3.0 (2023-04-21 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=Portuguese_Brazil.utf8  LC_CTYPE=Portuguese_Brazil.utf8   
[3] LC_MONETARY=Portuguese_Brazil.utf8 LC_NUMERIC=C                      
[5] LC_TIME=Portuguese_Brazil.utf8    

time zone: America/Sao_Paulo
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.3.0

[LaCuneta]

@edmizac Thanks for your detailed report, sample model, and the info from all the affected systems. I also could not reproduce when running on Linux, but I got ahold of a Windows 11 VM. The good news is the issue did occur there so I have a system I can work on. I'll try to track down a root cause and get it fixed; I'll post an update here once I have more info.

[LaCuneta]

Okay, so it took me longer than I expected to get a fix, but I've published version 2.0.2 of the Simple R Extension to the extensions manager. Please test it out if you get a chance and let me know if it works for you.

The technical bits: I found the BufferedReader that the core NetLogo Language Library was using was hanging on Windows when the given sample code was ran. I traced everything I could think of but could not find a cause. I decided to try making a simple custom line reader to drop in to maybe resolve the issue, or at least provide more debug information. That simple reader worked to resolve the problem. So I published the change and the Simple R extension as well. I'd love to know what the root cause was, but am happy to have something that might work.

[edmizac]

Thanks, Jeremy. However, the issue continues...

I have updated simpleR in all machines I have access to, but I still get the same error (frozen screen). The only new thing is this error when I try to end NetLogo through the Windows Manager:

Is there a possibility for you to test it on a W10 machine?

Thank you and sorry for the late response.

[LaCuneta]

@edmizac Thanks for letting me know. I checked the package and there was an issue with it (it contained the old interop library as well as the new one), but in testing the fixed version I found another issue. I'm working on that now and will update here once it's fixed.

[LaCuneta]

@edmizac I have isolated a root cause of the issue, but I haven't got a fix just yet. The core issue is with the way the Simple R Extension redirects R's output back to the NetLogo Command Center; it's causing a deadlock in some scenarios (but not all).

Knowing the cause does provide a workaround: avoiding the use of print() (and similar) statements in R code will avoid the deadlock. The easiest way to do that is to push the data back to NetLogo to display it, as in show sr:runresult "df1" instead of sr:run "print(df1)". If there is a lot of data to be displayed and pushing it all back to NetLogo isn't desirable, another alternative is to write it out to a text file.

To be clear, this is definitely a bug and I'll be working on getting use of those R output commands working properly. I just want to get you the workarounds if it allows you to move forward with your work. Let me know if you have questions on it, or if you see hangs occur when not using any R output commands.

[edmizac]

Thanks for your endeavor @LaCuneta

I should say I still get the same freezing even after running show sr:runresult "df1" though. I will do my best to circumvent all the debugging process without printing more than 110 lines.

Best,

[LaCuneta]

@edmizac Okay, I have a new version of the Simple R Extension, 2.0.3, posted to the extensions library. I found a bug with the way NetLogo installs extensions that causes an issue; basically it can leave older versions of dependencies behind went updating an extensions. To workaround, you'll need to uninstall the Simple R extension and then re-install it instead of just updating it. You'll need to open NetLogo fresh without starting a model that uses the Simple R extension to make sure it can be removed. Let me know if you have any issues with the remove and re-install.

The new Simple R version should allow for use of any console output from R code, and should resolve any other issues you were still seeing. Please do let me know if that is not the case.

The root cause was that the console stdout for the R process was becoming full before we had a chance to read it out, which caused the process to hang. The solution was to setup simple threads to read the stdout async as soon as possible and keep it empty.

[edmizac]

Thanks for the troubleshooting, Jeremy!

I am now printing more than 110 lines.

About installing/reinstalling: At first I got an unknown error from NetLogo 6.3.0 after trying to uninstall SimpleR. I reopened NetLogo (clean) and uninstalled sucessfully on the second try. So I guess no problems with it, also...

Thanks again!