search

jotta / jotta-cli-issues

46 stars 1 forks source link

jotta-cli ignores set <filename> failes with out of memory error #62

Closed pamunthe closed 3 years ago

pamunthe commented 5 years ago

Make sure you are running the latest version of jotta-cli before reporting an issue.

jotta-cli release (jotta-cli version):

jotta-cli version 0.4.4986

jottad version 0.4.4986 jottad host ubuntu jottad path /usr/bin/jottad jottad appdata /var/lib/jottad

Description of problem: peram@ubuntu:~$ jotta-cli ignores set /nasmedia/backup/old_home_hackingtosh_050616.tar.gz fatal error: runtime: out of memory

Expected:

jotta-cli status (jotta-cli status):

peram@ubuntu:~$ jotta-cli status
------------------------------------------------------------------------------
 Account   : pamunthe
 Usage     : 4.88TiB / ( Unlimited )
 Device    : NASBOX
 Backups   :
------------------------------------------------------------------------------
   Path      : /nasmedia/backup
   Files     : 1021693 files / 3.18TiB
   Status    : Transferring 1021668 files / 2.41TiB remaining

------------------------------------------------------------------------------
   Path      : /nasmedia/movies
   Files     : 123 files / 4.30TiB
   Status    : Up to date!

------------------------------------------------------------------------------
OK

Relevant logs for the issue (cat ~/.jottad/jottabackup.log)

peram@ubuntu:~$ tail -40 /var/lib/jottad/jottabackup.log
pid:2145 2019/01/07 08:06:49 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:08:59 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:11:10 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:13:19 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:15:29 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:17:39 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:19:39 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:19:51 Deferring scan of /nasmedia/movies while waiting for previous queue to finish
pid:2145 2019/01/07 08:21:59 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:24:09 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:26:19 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:28:29 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:30:39 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:32:50 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:34:59 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:36:20 Updating account info
pid:2145 2019/01/07 08:36:59 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:39:09 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:41:19 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:43:29 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:45:39 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:47:49 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:49:59 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:52:09 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:54:10 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:56:19 Running scan of /nasmedia/movies
pid:2145 2019/01/07 08:58:30 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:00:39 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:02:49 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:04:49 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:06:21 Updating account info
pid:2145 2019/01/07 09:06:59 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:09:09 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:11:19 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:13:29 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:15:40 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:17:49 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:19:59 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:22:09 Running scan of /nasmedia/movies
pid:2145 2019/01/07 09:24:20 Running scan of /nasmedia/movies

Traceback

runtime stack:
runtime.throw(0x8fe1d3, 0x16)
        /usr/local/go/src/runtime/panic.go:605 +0x95
runtime.sysMap(0xc620300000, 0x200000000, 0x3be00, 0xbc9b18)
        /usr/local/go/src/runtime/mem_linux.go:216 +0x1d0
runtime.(*mheap).sysAlloc(0xbb0700, 0x200000000, 0xffffffff000af8f9)
        /usr/local/go/src/runtime/malloc.go:470 +0xd7
runtime.(*mheap).grow(0xbb0700, 0x100000, 0x0)
        /usr/local/go/src/runtime/mheap.go:887 +0x60
runtime.(*mheap).allocSpanLocked(0xbb0700, 0x100000, 0xbc9b28, 0x0)
        /usr/local/go/src/runtime/mheap.go:800 +0x334
runtime.(*mheap).alloc_m(0xbb0700, 0x100000, 0xffffffffffff0101, 0xc4201b2a80)
        /usr/local/go/src/runtime/mheap.go:666 +0x118
runtime.(*mheap).alloc.func1()
        /usr/local/go/src/runtime/mheap.go:733 +0x4d
runtime.systemstack(0x7f09afffee10)
        /usr/local/go/src/runtime/asm_amd64.s:360 +0xab
runtime.(*mheap).alloc(0xbb0700, 0x100000, 0xc420010101, 0x7f09afffee80)
        /usr/local/go/src/runtime/mheap.go:732 +0xa1
runtime.largeAlloc(0x1fffffe00, 0xc420020101, 0xc420000180)
        /usr/local/go/src/runtime/malloc.go:827 +0x98
runtime.mallocgc.func1()
        /usr/local/go/src/runtime/malloc.go:722 +0x46
runtime.systemstack(0xc4201b2ab8)
        /usr/local/go/src/runtime/asm_amd64.s:344 +0x79
runtime.mstart()
        /usr/local/go/src/runtime/proc.go:1135

goroutine 1 [running]:
runtime.systemstack_switch()
        /usr/local/go/src/runtime/asm_amd64.s:298 fp=0xc420051818 sp=0xc420051810 pc=0x457810
runtime.mallocgc(0x1fffffe00, 0x83a6a0, 0xc420051801, 0x40000000)
        /usr/local/go/src/runtime/malloc.go:721 +0x7ae fp=0xc4200518c0 sp=0xc420051818 pc=0x4126fe
runtime.makeslice(0x83a6a0, 0x1fffffe00, 0x1fffffe00, 0xc52028c000, 0xfffffe00, 0xfffffe00)
        /usr/local/go/src/runtime/slice.go:54 +0x77 fp=0xc4200518f0 sp=0xc4200518c0 pc=0x442f87
bytes.makeSlice(0x1fffffe00, 0x0, 0x0, 0x0)
        /usr/local/go/src/bytes/buffer.go:231 +0x77 fp=0xc420051930 sp=0xc4200518f0 pc=0x46e687
bytes.(*Buffer).ReadFrom(0xc4200519d8, 0xb56480, 0xc420250008, 0xc42025c000, 0x0, 0x200)
        /usr/local/go/src/bytes/buffer.go:203 +0x291 fp=0xc4200519a8 sp=0xc420051930 pc=0x46e5c1
io/ioutil.readAll(0xb56480, 0xc420250008, 0x200, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/io/ioutil/ioutil.go:33 +0x12c fp=0xc420051a50 sp=0xc4200519a8 pc=0x53209c
io/ioutil.ReadAll(0xb56480, 0xc420250008, 0xc420250008, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/io/ioutil/ioutil.go:42 +0x3e fp=0xc420051aa0 sp=0xc420051a50 pc=0x53217e
main.(*Cli).SetExcludes(0xc420143760, 0x7ffe38cef7bc, 0x33, 0x0, 0x0)
        /go/src/git.jotta.us/dev/jotta-nix-backup/jotta-cli/apicli.go:90 +0x20a fp=0xc420051b78 sp=0xc420051aa0 pc=0x7e079a
main.main()
        /go/src/git.jotta.us/dev/jotta-nix-backup/jotta-cli/main.go:396 +0x2492 fp=0xc420051f80 sp=0xc420051b78 pc=0x7e7b82
runtime.main()
        /usr/local/go/src/runtime/proc.go:195 +0x226 fp=0xc420051fe0 sp=0xc420051f80 pc=0x42d766
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc420051fe8 sp=0xc420051fe0 pc=0x45a421

goroutine 5 [syscall, 38 minutes]:
os/signal.signal_recv(0x0)
        /usr/local/go/src/runtime/sigqueue.go:131 +0xa6
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
        /usr/local/go/src/os/signal/signal_unix.go:28 +0x41

goroutine 8 [select, 38 minutes]:
git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc.(*addrConn).transportMonitor(0xc420096400)
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/clientconn.go:1002 +0x166
git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc.(*addrConn).connect.func1(0xc420096400)
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/clientconn.go:703 +0x1ce
created by git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc.(*addrConn).connect
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/clientconn.go:694 +0x40d

goroutine 11 [IO wait, 38 minutes]:
internal/poll.runtime_pollWait(0x7f09c05b8f70, 0x72, 0x0)
        /usr/local/go/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc42015a398, 0x72, 0xffffffffffffff00, 0xb580c0, 0xb53668)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
internal/poll.(*pollDesc).waitRead(0xc42015a398, 0xc4201d2000, 0x800, 0x800)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc42015a380, 0xc4201d2000, 0x800, 0x800, 0x0, 0x0, 0x0)
        /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
net.(*netFD).Read(0xc42015a380, 0xc4201d2000, 0x800, 0x800, 0x26, 0x0, 0x0)
        /usr/local/go/src/net/fd_unix.go:202 +0x52
net.(*conn).Read(0xc42000e120, 0xc4201d2000, 0x800, 0x800, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/net.go:176 +0x6d
crypto/tls.(*block).readFromUntil(0xc4201c0000, 0x7f09c05bd0b0, 0xc42000e120, 0x5, 0xc42000e120, 0x0)
        /usr/local/go/src/crypto/tls/conn.go:488 +0x95
crypto/tls.(*Conn).readRecord(0xc420158700, 0x915417, 0xc420158820, 0x0)
        /usr/local/go/src/crypto/tls/conn.go:590 +0xe0
crypto/tls.(*Conn).Read(0xc420158700, 0xc420224000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
        /usr/local/go/src/crypto/tls/conn.go:1134 +0x110
bufio.(*Reader).Read(0xc42005ec00, 0xc4201a62d8, 0x9, 0x9, 0x9, 0x0, 0x0)
        /usr/local/go/src/bufio/bufio.go:213 +0x30b
io.ReadAtLeast(0xb54ec0, 0xc42005ec00, 0xc4201a62d8, 0x9, 0x9, 0x9, 0xc420040de8, 0x401c6e, 0xc420040e97)
        /usr/local/go/src/io/io.go:309 +0x86
io.ReadFull(0xb54ec0, 0xc42005ec00, 0xc4201a62d8, 0x9, 0x9, 0x44b868, 0xa4c0dda227396, 0xc4138ef53e)
        /usr/local/go/src/io/io.go:327 +0x58
git.jotta.us/dev/jotta-nix-backup/vendor/golang.org/x/net/http2.readFrameHeader(0xc4201a62d8, 0x9, 0x9, 0xb54ec0, 0xc42005ec00, 0x0, 0x0, 0xba8560, 0xbf04dd3a92a288c1)
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/golang.org/x/net/http2/frame.go:237 +0x7b
git.jotta.us/dev/jotta-nix-backup/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc4201a62a0, 0xc42023a020, 0xc42023a020, 0x0, 0x0)
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/golang.org/x/net/http2/frame.go:492 +0xa4
git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport.(*http2Client).reader(0xc4201be480)
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport/http2_client.go:1118 +0xbe
created by git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport.newHTTP2Client
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport/http2_client.go:266 +0xc80

goroutine 12 [select, 38 minutes]:
git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport.loopyWriter(0xb5cb40, 0xc420010940, 0xc4201649f0, 0xc420041fb8)
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport/transport.go:750 +0x2e4
git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport.newHTTP2Client.func3(0xc4201be480)
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport/http2_client.go:298 +0x5e
created by git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport.newHTTP2Client
        /go/src/git.jotta.us/dev/jotta-nix-backup/vendor/google.golang.org/grpc/transport/http2_client.go:297 +0xdd1
peram@ubuntu:~$

Additional info:

peram@ubuntu:~$ ls -lh /nasmedia/backup/old_home_hackingtosh_050616.tar.gz -rwxrwxrwx 1 peram staff 437G Jun 5 2016 /nasmedia/backup/old_home_hackingtosh_050616.tar.gz

Kimbsen commented 5 years ago

I think you misunderstood what the ignore command does.

https://docs.jottacloud.com/jottacloud-command-line-tool/using-the-command-line-tool/ignoring-files-and-folders-from-backup-with-jottacloud-cli

But i do think we should support something like what you attempted.

Perhaps on the form of jotta-cli ignores add --path=/path/to/file/or/folder or something like that. Added to our internal todo.

coopersimon commented 3 years ago

Closing this as it is stale. If this is still a problem, please open a new issue :)