godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
90.76k stars 21.13k forks source link

directory.copy is 2x slower in windows #51573

Open qdwang opened 3 years ago

qdwang commented 3 years ago

Godot version

3.3.2 stable

System information

windows 10, ubuntu 16.04

Issue description

directory.copy is very slow in windows, much slower than native windows copy, and 2x slower than directory.copy in linux

diff_copy_100m

Steps to reproduce

extends Node2D

func _ready():
  var dir = Directory.new()
  var t1 = OS.get_system_time_msecs()
  dir.copy("res://dummy.file", "res://dummy.file.copy")
  printt("Time", OS.get_system_time_msecs() - t1)

Minimal reproduction project

CopySlowIssue.zip

Pls notice there is a 100MB dummy.file inside

mhilbrunner commented 3 years ago

directory.copy is very slow in windows, much slower than native windows copy, and 2x slower than directory.copy in linux

Could you provide how long it approximately takes with your test on each OS? (Edit: Saw your screenshot shows time)

The model of your disk may help.

Yeah, the current copy() implementation seems like it always copies the whole file over in small chunks, which maybe could be improved, especially if everything is on the same disk/partition/fs. Need to investigate.

Zylann commented 3 years ago

Is filesystem access even comparable between Linux and Windows? I know file access (the open/close part) is definitely slower on Windows in general, not just with Godot (matters when dealing with lots of files), or has it become worse since a previous version?

akien-mga commented 3 years ago

Yeah Linux being twice as fast as Windows for I/O doesn't strike me as a bug, slow I/O is a well-known Windows issue which is not specific to Godot.

More interesting is the claim that Godot copy on Windows would be much slower than the native copy in explorer.exe, in which case there might be room for improvements indeed.

Calinou commented 3 years ago

Microsoft generally advises using its proprietary RoboCopy for fast file copies, but this is most likely not something we can do (even using OS.execute()).

qdwang commented 3 years ago

Microsoft generally advises using its proprietary RoboCopy for fast file copies, but this is most likely not something we can do (even using OS.execute()).

This reminds me that I can use OS.execute("copy", .... in windows and OS.execute("cp", ... in *nix. I've tested the performance, it's very fast. Thanks. Pls feel free to close the issue.