search

ZeroK-RTS / Zero-K

Open source RTS game running on the Spring/Recoil engine
https://zero-k.info
GNU General Public License v2.0
676 stars 204 forks source link

Look into loading times #4381

Open GoogleFrog opened 3 years ago

GoogleFrog commented 3 years ago

I think I talked about this earlier but failed to make a ticket. A lot of lazy caching, generation and UnitDefs access is done in luaUI, luaRules and unitdefs_posts. The first step is to figure out how to profile these things, since synced luaRules seem likely to be a major culprit.

GoogleFrog commented 3 years ago

Profiling synced luarules loading: https://github.com/ZeroK-RTS/Zero-K/commit/190d4a18da40bc5a77ac4a9de2fbc7c69f74944a

Here are the offenders.

[f=-000001] Loaded LuaRules/Gadgets/wake_fx.lua 2.15899992
[f=-000001] Loaded LuaRules/Gadgets/unit_target_priority.lua 1.61299992
[f=-000001] Loaded LuaRules/Gadgets/unit_target_features.lua 0.98399997
[f=-000001] Loaded LuaRules/Gadgets/unit_show_shooter.lua 0.91399997
[f=-000001] Loaded gadget LUS 0.103
[f=-000001] Loaded LuaRules/Gadgets/unit_ward_fire.lua 0.098
[f=-000001] Loaded LuaRules/Gadgets/weapon_reaim_time.lua 0.08899999
[f=-000001] Loaded LuaRules/Gadgets/unit_tactical_ai.lua 0.086
[f=-000001] Loaded gadget Metalspot Finder Gadget 0.084
[f=-000001] Loaded LuaRules/Gadgets/unit_target_on_the_move.lua 0.07799999
[f=-000001] Loaded LuaRules/Gadgets/cmd_onoff.lua 0.063
[f=-000001] Loaded LuaRules/Gadgets/unit_control_gunship_stafe_range.lua 0.055
[f=-000001] Loaded LuaRules/Gadgets/unit_boolean_disable.lua 0.054
[f=-000001] Loaded LuaRules/Gadgets/ai_cai2.lua 0.047
[f=-000001] Loaded LuaRules/Gadgets/unit_thrower.lua 0.028
[f=-000001] Loaded LuaRules/Gadgets/gui_dualfog_gadget.lua 0.024
[f=-000001] Loaded LuaRules/Gadgets/api_modularcomms.lua 0.023
[f=-000001] Loaded LuaRules/Gadgets/unit_mex_overdrive.lua 0.022
[f=-000001] Loaded LuaRules/Gadgets/ai_cai.lua 0.02
[f=-000001] Loaded LuaRules/Gadgets/unit_carrier_drones.lua 0.017
[f=-000001] Loaded LuaRules/Gadgets/unit_decloak_damaged.lua 0.017
[f=-000001] Loaded LuaRules/Gadgets/unit_capture.lua 0.015
[f=-000001] Loaded LuaRules/Gadgets/unit_terraform.lua 0.015
[f=-000001] Loaded LuaRules/Gadgets/unit_centeroffset.lua 0.014
[f=-000001] Loaded LuaRules/Gadgets/endgame_graphs.lua 0.013
[f=-000001] Loaded LuaRules/Gadgets/unit_refuel_pad_handler.lua 0.012
[f=-000001] Loaded LuaRules/Gadgets/unit_stockpile.lua 0.012
[f=-000001] Loaded LuaRules/Gadgets/unit_fall_damage.lua 0.011
[f=-000001] Loaded LuaRules/Gadgets/unit_paralysis_damage.lua 0.011
[f=-000001] Loaded gadget Terraformers 0.011
[f=-000001] Loaded LuaRules/Gadgets/cmd_remove_wait.lua 0.01
[f=-000001] Loaded LuaRules/Gadgets/unit_planetwars.lua 0.01

The test was done on FolsomDamDeluxeV4, which is at least relevant for mex spot finder. The map Oktogon v3 is large and has a metal config, testing it drops Metalspot Finder Gadget to 0.006. There is something to be gained by caching all the metal spots for featured maps, but it would not be much.

The effect of synced-only Spring callouts was not measured. The raw data file has an echo every time one of these callouts is attempted.

rawLoadSyncedLuarules.txt

GoogleFrog commented 3 years ago

I dealt with the top four https://github.com/ZeroK-RTS/Zero-K/commit/319c12670757595a9a4c3ecd00c4388572ed6ebc

GoogleFrog commented 3 years ago

A nonstandard method of checking for synced luarules potentially hid some offenders.

Fixed https://github.com/ZeroK-RTS/Zero-K/commit/dca1c58ce6b14d00bd226736db2393923f43079c

GoogleFrog commented 3 years ago

Here are the new offenders.

[f=-000001] Loaded LuaRules/Gadgets/cmd_fire_once.lua 0.99699998
[f=-000001] Loaded LuaRules/Gadgets/unit_target_priority.lua 0.40599999
[f=-000001] Loaded gadget LUS 0.10399999
[f=-000001] Loaded gadget Metalspot Finder Gadget 0.103
[f=-000001] Loaded LuaRules/Gadgets/unit_tactical_ai.lua 0.09
[f=-000001] Loaded LuaRules/Gadgets/unit_target_on_the_move.lua 0.08199999
[f=-000001] Loaded LuaRules/Gadgets/unit_ward_fire.lua 0.075
[f=-000001] Loaded LuaRules/Gadgets/unit_target_features.lua 0.074
[f=-000001] Loaded LuaRules/Gadgets/weapon_reaim_time.lua 0.07099999
[f=-000001] Loaded LuaRules/Gadgets/cmd_onoff.lua 0.063
[f=-000001] Loaded LuaRules/Gadgets/unit_control_gunship_stafe_range.lua 0.063
[f=-000001] Loaded LuaRules/Gadgets/unit_boolean_disable.lua 0.049
[f=-000001] Loaded LuaRules/Gadgets/unit_show_shooter.lua 0.048
[f=-000001] Loaded LuaRules/Gadgets/ai_cai2.lua 0.045
[f=-000001] Loaded LuaRules/Gadgets/weapon_shield_merge.lua 0.039
[f=-000001] Loaded LuaRules/Gadgets/unit_fall_damage.lua 0.034
[f=-000001] Loaded LuaRules/Gadgets/gui_dualfog_gadget.lua 0.024
[f=-000001] Loaded LuaRules/Gadgets/api_modularcomms.lua 0.022
[f=-000001] Loaded LuaRules/Gadgets/unit_mex_overdrive.lua 0.02
[f=-000001] Loaded LuaRules/Gadgets/unit_jumpjets.lua 0.017
[f=-000001] Loaded LuaRules/Gadgets/ai_cai.lua 0.016
[f=-000001] Loaded LuaRules/Gadgets/unit_decloak_damaged.lua 0.015
[f=-000001] Loaded LuaRules/Gadgets/cmd_remove_wait.lua 0.014
[f=-000001] Loaded LuaRules/Gadgets/unit_centeroffset.lua 0.014
[f=-000001] Loaded LuaRules/Gadgets/unit_paralysis_damage.lua 0.014
[f=-000001] Loaded LuaRules/Gadgets/lups_napalm.lua 0.013
[f=-000001] Loaded LuaRules/Gadgets/unit_capture.lua 0.013
[f=-000001] Loaded LuaRules/Gadgets/unit_jugglenaut_juggle.lua 0.012
[f=-000001] Loaded LuaRules/Gadgets/unit_terraform.lua 0.012
[f=-000001] Loaded gadget Terraformers 0.012
[f=-000001] Loaded LuaRules/Gadgets/endgame_graphs.lua 0.011
[f=-000001] Loaded LuaRules/Gadgets/unit_planetwars.lua 0.011
[f=-000001] Loaded LuaRules/Gadgets/unit_is_on_fire.lua 0.01
GoogleFrog commented 3 years ago

Fixed https://github.com/ZeroK-RTS/Zero-K/commit/95e1ceda18b1a9af1d6ab65f1f2fd528feb5fc15 cmd_fire_once and unit_target_priority

Load time for synced luarules is 7.15 -> 1.95 seconds in total.

New offenders list.

[f=-000001] Loaded LuaRules/Gadgets/unit_target_priority.lua 0.32499999
[f=-000001] Loaded gadget LUS 0.133
[f=-000001] Loaded LuaRules/Gadgets/unit_tactical_ai.lua 0.085
[f=-000001] Loaded LuaRules/Gadgets/cmd_fire_once.lua 0.083
[f=-000001] Loaded gadget Metalspot Finder Gadget 0.077
[f=-000001] Loaded LuaRules/Gadgets/unit_ward_fire.lua 0.076
[f=-000001] Loaded LuaRules/Gadgets/unit_target_on_the_move.lua 0.072
[f=-000001] Loaded LuaRules/Gadgets/cmd_onoff.lua 0.069
[f=-000001] Loaded LuaRules/Gadgets/unit_control_gunship_stafe_range.lua 0.066
[f=-000001] Loaded LuaRules/Gadgets/unit_boolean_disable.lua 0.061
[f=-000001] Loaded LuaRules/Gadgets/unit_target_features.lua 0.058
[f=-000001] Loaded LuaRules/Gadgets/weapon_reaim_time.lua 0.056
[f=-000001] Loaded LuaRules/Gadgets/ai_cai2.lua 0.053
[f=-000001] Loaded LuaRules/Gadgets/unit_show_shooter.lua 0.044
[f=-000001] Loaded LuaRules/Gadgets/gui_dualfog_gadget.lua 0.027
[f=-000001] Loaded LuaRules/Gadgets/api_modularcomms.lua 0.022
[f=-000001] Loaded LuaRules/Gadgets/unit_fall_damage.lua 0.018
[f=-000001] Loaded LuaRules/Gadgets/unit_thrower.lua 0.018
[f=-000001] Loaded LuaRules/Gadgets/ai_cai.lua 0.017
[f=-000001] Loaded LuaRules/Gadgets/unit_paralysis_damage.lua 0.017
[f=-000001] Loaded LuaRules/Gadgets/cmd_remove_wait.lua 0.016
[f=-000001] Loaded LuaRules/Gadgets/unit_mex_overdrive.lua 0.016
[f=-000001] Loaded gadget Terraformers 0.016
[f=-000001] Loaded LuaRules/Gadgets/unit_terraform.lua 0.015
[f=-000001] Loaded LuaRules/Gadgets/unit_centeroffset.lua 0.014
[f=-000001] Loaded LuaRules/Gadgets/lups_napalm.lua 0.013
[f=-000001] Loaded LuaRules/Gadgets/unit_jugglenaut_juggle.lua 0.013
[f=-000001] Loaded LuaRules/Gadgets/unit_jumpjets.lua 0.011
[f=-000001] Loaded LuaRules/Gadgets/endgame_graphs.lua 0.01
[f=-000001] Loaded LuaRules/Gadgets/unit_bait_prevention.lua 0.01
[f=-000001] Loaded LuaRules/Gadgets/unit_command_adder.lua 0.01
[f=-000001] Loaded LuaRules/Gadgets/unit_decloak_damaged.lua 0.01
GoogleFrog commented 3 years ago

Here are the offenders in Unsynced Luarules. Note that Loaded gadget means gadget:Initialize as opposed to reading in the file. I don't really want to touch CUS and there isn't any problem elsewhere.

[f=-000001] Loaded gadget CustomUnitShaders 1.86300004
[f=-000001] Loaded gadget Lups 0.064
[f=-000001] Loaded LuaRules/Gadgets/unit_target_features.lua 0.056
[f=-000001] Loaded LuaRules/Gadgets/ai_cai2.lua 0.045
[f=-000001] Loaded LuaRules/Gadgets/gui_dualfog_gadget.lua 0.024
[f=-000001] Loaded LuaRules/Gadgets/api_modularcomms.lua 0.022
[f=-000001] Loaded LuaRules/Gadgets/lups_wrapper.lua 0.022
[f=-000001] Loaded LuaRules/Gadgets/ai_cai.lua 0.02
[f=-000001] Loaded gadget Lups Shield 0.015
[f=-000001] Loaded LuaRules/Gadgets/unit_thrower.lua 0.008
[f=-000001] Loaded LuaRules/Gadgets/weapon_hide_sniper.lua 0.008
[f=-000001] Loaded gadget Modular Comm Info 0.008
[f=-000001] Loaded gadget Target on the move 0.008
[f=-000001] Loaded LuaRules/Gadgets/unit_bomber_command.lua 0.005
[f=-000001] Loaded LuaRules/Gadgets/unit_terraform.lua 0.005
[f=-000001] Loaded gadget Mex Placement 0.005
[f=-000001] Loaded gadget Aircraft Command 0.005
GoogleFrog commented 3 years ago

Unitdef caching did not do much https://github.com/ZeroK-RTS/Zero-K/commit/14b6c8cd7a0ceceebc25727fa829d9c466486272

GoogleFrog commented 3 years ago

Here are the offenders in luaUI. The total is 4.01 seconds.

[f=-000001] LoadWidget LuaUI/Widgets/gui_chili_selections_and_cursortip.lua 0.833
[f=-000001] Add widget HealthBars 0.759
[f=-000001] Add widget Chili Framework 0.56400001
[f=-000001] LoadWidget LuaUI/Widgets/api_i18n.lua 0.19299999
[f=-000001] LoadWidget LuaUI/Widgets/unit_start_state.lua 0.16
[f=-000001] Add widget Attack AoE 0.101
[f=-000001] Add widget Map Edge Extension 0.072
[f=-000001] Add widget Lups 0.054
[f=-000001] Add widget Chili Pro Console 0.04
[f=-000001] LoadWidget LuaUI/Widgets/unit_show_selected_range.lua 0.038
[f=-000001] Add widget Chili Selections & CursorTip v2 0.036
[f=-000001] Add widget Chili Minimap 0.031
[f=-000001] Add widget Chili EndGame Window 0.028
[f=-000001] LoadWidget LuaUI/Widgets/gui_contextmenu.lua 0.026
[f=-000001] LoadWidget LuaUI/Widgets/gfx_lups_manager.lua 0.025
[f=-000001] LoadWidget LuaUI/Widgets/lups_wrapper.lua 0.024
[f=-000001] LoadWidget LuaUI/Widgets/api_modularcomms.lua 0.023
[f=-000001] Add widget EPIC Menu 0.022
[f=-000001] LoadWidget LuaUI/Widgets/gui_chili_keyboardmenu.lua 0.02
[f=-000001] Add widget Unit Start State 0.02
[f=-000001] LoadWidget LuaUI/Widgets/api_shared_functions.lua 0.019
[f=-000001] Add widget Chili Economy Panel Default 0.019
[f=-000001] LoadWidget LuaUI/Widgets/gui_epicmenu.lua 0.017
[f=-000001] Add widget Projectile Lights 0.016
[f=-000001] LoadWidget LuaUI/Widgets/api_gadget_icons.lua 0.014
[f=-000001] LoadWidget LuaUI/Widgets/gui_economic_victory_announcer.lua 0.014
[f=-000001] Add widget UnitShapes 0.014
[f=-000001] LoadWidget LuaUI/Widgets/gui_chili_core_selector.lua 0.013
[f=-000001] LoadWidget LuaUI/Widgets/gui_chili_integral_menu.lua 0.013
[f=-000001] LoadWidget LuaUI\Widgets\jugglerai.lua 0.013
[f=-000001] LoadWidget LuaUI/Widgets/gui_chili_cheats.lua 0.011
[f=-000001] Add widget Startup Info and Selector 0.011
[f=-000001] Add widget Chili Integral Menu 0.011
[f=-000001] LoadWidget LuaUI/Widgets/gui_chili_gesture_menu.lua 0.01
[f=-000001] LoadWidget LuaUI/Widgets/gui_healthbars.lua 0.01
[f=-000001] LoadWidget LuaUI/Widgets/gui_selection_hierarchy.lua 0.01
[f=-000001] Add widget Chili Global Commands 0.01
GoogleFrog commented 3 years ago

Solved the top two offending widgets with better usage of ud.weapons. https://github.com/ZeroK-RTS/Zero-K/commit/53023b553906f5683a85f20cbe867e5de0ad31ec

There is probably some profit to be made in _posts, but I don't know how to profile that so am going to call the lua speedup good enough for now.