havardgulldahl / jottalib

A library to access files stored at jottacloud.com.
GNU General Public License v3.0
83 stars 19 forks source link

jotta-scanner enters excluded subdirectories #121

Open awerner opened 8 years ago

awerner commented 8 years ago
$ pip freeze | grep jotta
jottalib==0.5.0
$ cat /etc/debian_version
8.5

I try to do a full backup of one of my vservers with the command: jotta-scanner / /Jotta/Archive/backup/marty --exclude /proc/ --exclude /dev/ --exclude /sys/ --exclude /run/ --exclude /var/run/ --exclude /var/lib/libvirt/images/.

What happens is that jotta enters each and every subdirectory before it gets excluded:

Entering dir: /proc/110/fdinfo
Entering dir: /proc/110/ns
Entering dir: /proc/110/net
Entering dir: /proc/110/net/xt_recent
Entering dir: /proc/110/net/dev_snmp6
Entering dir: /proc/110/net/netfilter
Entering dir: /proc/110/net/stat
Entering dir: /proc/110/attr
Entering dir: /proc/120
Entering dir: /proc/120/task
Entering dir: /proc/120/task/120
Entering dir: /proc/120/task/120/fd
Entering dir: /proc/120/task/120/fdinfo
Entering dir: /proc/120/task/120/ns
Entering dir: /proc/120/task/120/attr
Entering dir: /proc/120/fd
Entering dir: /proc/120/map_files
Entering dir: /proc/120/fdinfo
Entering dir: /proc/120/ns
Entering dir: /proc/120/net
Entering dir: /proc/120/net/xt_recent
Entering dir: /proc/120/net/dev_snmp6
Entering dir: /proc/120/net/netfilter
Entering dir: /proc/120/net/stat

This process is very slow, so the expected behaviour would be to just jump to the next directory.

havardgulldahl commented 8 years ago

@awerner Hey, thanks for the report.

Well, I agree, that sounds like an area that could use some more smartness. I guess we could even keep a default exclusion list. Who would ever need to backup /proc and /sys, anyway?

I guess this wouldn't be too much work, tweaking the directory traversing code. Do you want to take a look at it yourself?

awerner commented 8 years ago

Might take a little while as I'm currently moving but yeah, will have a look.

havardgulldahl commented 8 years ago

@awerner Great!