search

eclipse-che / che

Kubernetes based Cloud Development Environments for Enterprise Teams
http://eclipse.org/che
Eclipse Public License 2.0
6.96k stars 1.19k forks source link

Performance - Copying Multiple Files #3891

Closed JamesDrummond closed 7 years ago

JamesDrummond commented 7 years ago

Copying multiple image files (~1MB each) to another folder takes a considerable amount of time using project explorer in a workspace.

Reproduction Steps:

  1. Import che-docs project
  2. Copy images from src/main/artik/imgs/* to src/main/_docs/assets/imgs/
  3. Takes about 2-3 minutes. I didn't time it I just know it was longer than it should be.

Che version: codenvy nightly

OS and version: debian linode 4gb workspace

Docker version:

Containers: 20 Running: 13 Paused: 0 Stopped: 7 Images: 37 Server Version: 1.12.2 Storage Driver: devicemapper Pool Name: docker-8:0-122557-pool Pool Blocksize: 65.54 kB Base Device Size: 10.74 GB Backing Filesystem: ext4 Data file: /dev/loop0 Metadata file: /dev/loop1 Data Space Used: 9.83 GB Data Space Total: 107.4 GB Data Space Available: 41.47 GB Metadata Space Used: 9.331 MB Metadata Space Total: 2.147 GB Metadata Space Available: 2.138 GB Thin Pool Minimum Free Space: 10.74 GB Udev Sync Supported: true Deferred Removal Enabled: false Deferred Deletion Enabled: false Deferred Deleted Device Count: 0 Data loop file: /var/lib/docker/devicemapper/devicemapper/data Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata Library Version: 1.02.90 (2014-09-01) Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: overlay bridge host null Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: containerd version: runc version: init version: Kernel Version: 4.8.6-x86_64-linode78 Operating System: Debian GNU/Linux 8 (jessie) OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 7.793 GiB Name: debian ID: FYOW:4NQ7:A2P2:35QG:LM74:IJBF:3WDW:54RM:ZL6Q:XY2O:V6FC:KWCB Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

skabashnyuk commented 7 years ago

What OS on host?

JamesDrummond commented 7 years ago

@skabashnyuk From above it's Operating System: Debian GNU/Linux 8 (jessie) . The workspace is ready-to-go java stack.

skabashnyuk commented 7 years ago

Is this you host OS? or this is workspace os?

JamesDrummond commented 7 years ago

Sorry. I updated the description above. It's copying files with project explorer in a workspace.

skabashnyuk commented 7 years ago

How do you run che? on Mac, Linux or Windows?

JamesDrummond commented 7 years ago

I ran codenvy on Operating System: Debian GNU/Linux 8 (jessie) with Total Memory: 7.793 GiB. Gave workspace 4gb of ram.

skabashnyuk commented 7 years ago

How much does it take the same operation on host?

JamesDrummond commented 7 years ago

@skabashnyuk Way less on host or when I copy in workspace via terminal(.1secs versus 3minutes). I performed the test myself as I changed the repo for the test case I gave above. Copy all images in eclipse/che-docs repo from /projects/che-docs/src/main/_docs/assets/imgs/ into /projects/temp/imgs/ via project explorer took 3 minutes and 2 seconds. Copy all images in eclipse/che-docs repo from /projects/che-docs/src/main/_docs/assets/imgs/ into /projects/temp/imgs2/ via terminal cp CLI took .1 seconds.

Project Explorer:

user@bb447a5c5b99:/projects$ cd temp/imgs/                                                                                                                                                                                                                                                                                       
user@bb447a5c5b99:/projects/temp/imgs$ find -type f -printf '%T+ %p\n' | sort | head -n 1                                                                                                                                                                                                                                        
2017-01-26+14:15:19.9994728450 ./Clipboard.jpg                                                                                                                                                                                                                                                                                   
user@bb447a5c5b99:/projects/temp/imgs$ find -type f -printf '%T+ %p\n' | sort | tail -n 1                                                                                                                                                                                                                                        
2017-01-26+14:18:17.6057121330 ./zdebug-settings.png

CLI copy

user@bb447a5c5b99:/projects$ cp che-docs/src/main/_docs/assets/imgs/* temp/imgs2/                                                                                                                                                                                                                                                
user@bb447a5c5b99:/projects$ cd temp/imgs2/                                                                                                                                                                                                                                                                                      
user@bb447a5c5b99:/projects/temp/imgs2$ find -type f -printf '%T+ %p\n' | sort | head -n 1                                                                                                                                                                                                                                       
2017-01-26+14:22:29.7603848990 ./01436220-a2ea-11e6-895f-be29fd448965.png                                                                                                                                                                                                                                                        
user@bb447a5c5b99:/projects/temp/imgs2$ find -type f -printf '%T+ %p\n' | sort | tail -n 1                                                                                                                                                                                                                                       
2017-01-26+14:22:29.8503830020 ./z-ray-debug.png

issue

dkuleshov commented 7 years ago

I see too possible problems here: project tree explorer move operation is buggy or non-optimized and this one https://github.com/eclipse/che/issues/3467. Or both.

TylerJewell commented 7 years ago

We are testing fixes that will resolve this and some other issues. We will issue a bug fix as soon as this branch is merged - probably within a week.

https://github.com/eclipse/che/tree/che%233891