Open umesh-timalsina opened 8 months ago
jdanish
commented
8 months ago Hello Umesh, I checked with Ben (one of the devs) and he said a) if you want to you can use a packet sniffer to see what data is being sent around, b) it's pretty low-level game data, and either way c) if you can provide some more description on what your system looks like and what you want it to do he can provide more useful guidance.
daveseah
commented
8 months ago Umesh, Another GEMSTEP dev here. Can you show me what you're logging (a snippet of the log file showing an example of data your capturing) and I can also provide options.
edavalosanaya
commented
8 months ago Hello!
I'm one of @umesh-timalsina colleague who works on ChimeraPy. From both the system logs and rtlogs, we mostly leveraged from the NET:DISPLAY_LIST logs. These logs were vital in reconstructing the game state, i.e. student's position and sprite-state -- making systematic analysis possible. Our objective is to collect the system logs and align them with other data streams, like audio and video.
Let me add some additional context to this issue. As @umesh-timalsina mentioned, we originally were approaching log collection via filesystem monitoring. However, this turned out to be computationally expensive due to the large volume of logs dumped by GEM-STEP. It started negatively impacting the macOS host computer and causing the game to lag.
Therefore, we wanted to migrate this computational load to another computer via a WS connection. Additionally, the NET:DISPLAY_LIST logs don't provide us the complete picture, such as game phase and restarts. It would be ideal if we could obtain this nuggets of information via the WS channel. These would be needed to develop an automated analysis method.
Here is an example of the NET:DISPLAY_LIST JSON dump:
[
{
"id": "154",
"_pool_id": 0,
"valid": true,
"x": 0,
"y": 0,
"zIndex": 0,
"visible": false,
"meterPosition": 1,
"mode": 3,
"flags": 0
},
{
"id": "22",
"_pool_id": 1,
"valid": true,
"skin": "PS_daytime.png",
"x": -8.8,
"y": 19.97,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.3,
"scaleY": 0.3
},
{
"id": "23",
"_pool_id": 2,
"valid": true,
"skin": "PS_dirt_ground.png",
"x": -8.04,
"y": 46.33,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.3,
"scaleY": 0.3
},
{
"id": "24",
"_pool_id": 3,
"valid": true,
"skin": "PS_plant_body.png",
"x": -68.98,
"y": -19.32,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.3,
"scaleY": 0.3
},
{
"id": "25",
"_pool_id": 4,
"valid": true,
"skin": "PS_root_system.png",
"x": -73.74,
"y": 346.73,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.25,
"scaleY": 0.25
},
{
"id": "26",
"_pool_id": 5,
"valid": true,
"skin": "PS_rabbit.png",
"x": -322.32,
"y": 193.53,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.5,
"scaleY": 0.5
},
{
"id": "27",
"_pool_id": 6,
"valid": true,
"skin": "PS_left_leaf.png",
"x": -188.53,
"y": -98.13,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.1,
"scaleY": 0.1
},
{
"id": "32",
"_pool_id": 7,
"valid": true,
"skin": "PS_water.png",
"x": 139.65,
"y": -371.08,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "33",
"_pool_id": 8,
"valid": true,
"skin": "PS_carbon_dioxide.png",
"x": -141.31,
"y": -368.49,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "34",
"_pool_id": 9,
"valid": true,
"skin": "PS_sugar.png",
"x": 341.71,
"y": -371.26,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.02,
"scaleY": 0.02
},
{
"id": "35",
"_pool_id": 10,
"valid": true,
"skin": "PS_oxygen.png",
"x": -369.04,
"y": -370.28,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "36",
"_pool_id": 11,
"valid": true,
"skin": "PS_waterdroplet.png",
"x": 64.04,
"y": 306.51,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "38",
"_pool_id": 12,
"valid": true,
"skin": "PS_waterdroplet.png",
"x": -186.04,
"y": 271.53,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "39",
"_pool_id": 13,
"valid": true,
"skin": "PS_waterdroplet.png",
"x": -115.88,
"y": 336.58,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "40",
"_pool_id": 14,
"valid": true,
"skin": "PS_waterdroplet.png",
"x": -184.89,
"y": 347.67,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "41",
"_pool_id": 15,
"valid": true,
"skin": "PS_waterdroplet.png",
"x": -81.18,
"y": 270.93,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "42",
"_pool_id": 16,
"valid": true,
"skin": "PS_waterdroplet.png",
"x": 70.1,
"y": 350.08,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "43",
"_pool_id": 17,
"valid": true,
"skin": "PS_waterdroplet.png",
"x": -2.75,
"y": 285.95,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "44",
"_pool_id": 18,
"valid": true,
"skin": "PS_waterdroplet.png",
"x": -69.14,
"y": 339.35,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.03,
"scaleY": 0.03
},
{
"id": "45",
"_pool_id": 19,
"valid": true,
"skin": "PS_right_leaf.png",
"x": 102.94,
"y": 14.29,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.25,
"scaleY": 0.25
},
{
"id": "46",
"_pool_id": 20,
"valid": true,
"skin": "PS_chloroplast.png",
"x": 229.3,
"y": -239.93,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.25,
"scaleY": 0.25
},
{
"id": "47",
"_pool_id": 21,
"valid": true,
"skin": "PS_chloroplast.png",
"x": 307.14,
"y": 210.3,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 3,
"flags": 0,
"scale": 0.25,
"scaleY": 0.25
},
{
"id": "pz6954",
"_pool_id": 22,
"valid": true,
"skin": "PS_oxygen.png",
"x": -242.528,
"y": -254.60000000000002,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 2,
"flags": 0,
"scale": 0.1,
"scaleY": 0.1
},
{
"id": "pz6942",
"_pool_id": 23,
"valid": true,
"skin": "PS_oxygen.png",
"x": -231.08799999999997,
"y": -236.512,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 2,
"flags": 0,
"scale": 0.1,
"scaleY": 0.1
},
{
"id": "pz6951",
"_pool_id": 24,
"valid": true,
"skin": "PS_oxygen.png",
"x": -233.58399999999997,
"y": -263.56800000000004,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 2,
"flags": 0,
"scale": 0.1,
"scaleY": 0.1
},
{
"id": "pz6953",
"_pool_id": 25,
"valid": true,
"skin": "PS_oxygen.png",
"x": -240.23999999999995,
"y": -220.4,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 2,
"flags": 0,
"scale": 0.1,
"scaleY": 0.1
},
{
"id": "pz219",
"_pool_id": 26,
"valid": true,
"skin": "PS_oxygen.png",
"x": -235.248,
"y": -234.08,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 2,
"flags": 0,
"scale": 0.1,
"scaleY": 0.1
},
{
"id": "pz6915",
"_pool_id": 27,
"valid": true,
"skin": "PS_oxygen.png",
"x": -238.15999999999997,
"y": -250.496,
"zIndex": 0,
"frame": 0,
"visible": true,
"meterPosition": 1,
"mode": 2,
"flags": 0,
"scale": 0.1,
"scaleY": 0.1
}
]
Nice to meet ya @edavalosanaya!
I cobbled together a proof-of-concept standalone URNET client that might be able to reduce your log processing overhead. I'll put up a pull request with the new VERY UNTESTED code if you want to see if it will work for you.
jdanish
commented
8 months ago Flagging @joycehorn as well.
daveseah
commented
8 months ago The WIP PR is #796 for early testing.
daveseah
commented
8 months ago There's a README file in the gem-srv/ur-peek that has a list of NET:MESSAGES that you can intercept that may serve your needs. There's also some information about how URNET works. Good luck! Ben is the owner of the simulation start/stop logic so he may have other insights as to how to trap certain data, or perhaps arrange for it to be signaled.
excerpt from readme
'NET:BLUEPRINT_DELETE'
'NET:BLUEPRINTS_UPDATE'
'NET:BPNAMESLIST_UPDATE'
'NET:DISPLAY_LIST'
'NET:GEM_CHARCTRLAPP'
'NET:GEM_COMPILERAPP'
'NET:GEM_FAKETRACKAPP'
'NET:GEM_HOMEAPP'
'NET:HACK_INSPECTOR_UPDATE'
'NET:HACK_SIM_COSTUMES'
'NET:HACK_SIM_END'
'NET:HACK_SIM_NEXTROUND'
'NET:HACK_SIM_START'
'NET:HACK_SIM_STOP'
'NET:HELLO'
'NET:INSPECTOR_REGISTER'
'NET:INSPECTOR_UNREGISTER'
'NET:INSPECTOR_UPDATE'
'NET:INSTANCE_DESELECT'
'NET:INSTANCE_SELECT'
'NET:INSTANCE_UPDATE_POSITION'
'NET:INSTANCE_UPDATE'
'NET:INSTANCES_UPDATE'
'NET:INSTANCESLIST_UPDATE'
'NET:LOG_ENABLE'
'NET:LOG_EVENT'
'NET:PROJECTS_UPDATE'
'NET:REQ_PROJDATA'
'NET:SCRIPT_EDITOR_CLOSE'
'NET:SCRIPT_UPDATE'
'NET:SCRIPT_UPDATED'
'NET:SET_CHARCONTROL_BPIDLIST'
'NET:SIM_RESET'
'NET:SIM_WAS_RESET'
'NET:TRANSFORM_REQ'
'NET:UPDATE_MODEL'
Hello,
Would it be possible to subscribe to the game state and other events communicated to the front end via WebSockets externally?
Our UseCase
We want to collect the game state and other events in the game externally for a data collection effort in an ongoing study. Currently, we capture the game state info via log files dumped in the filesystem and listen to external filesystem events. We want to improve upon this if it's possible to listen to the server events directly.