DataDog / kvexpress

## Auto-archived due to inactivity. ## Go program to move data in and out of Consul's KV store.
Apache License 2.0
129 stars 13 forks source link

Add `chown` command #49

Closed darron closed 8 years ago

darron commented 9 years ago

So that files can land with different owners / groups.

darron commented 9 years ago

Due to the research I have found - there isn't any specific ways to do this within the Golang stdlib. Because it wasn't able to be cross platform it never got included and keeps getting pushed back.

May just go down to unix chown to do it for now.

darron commented 9 years ago

You can always chown by UID, GID, but I want to do it by name.

Looks like this might be a clue on how to do it:

https://github.com/lomik/go-carbon/blob/e0c1eb3205ffe1b0cb39e830d88001e62b2c650e/logging/logger.go https://golang.org/pkg/os/user/#Current

Will give it a shot.

darron commented 9 years ago

Turns out there's some sort of error here:

[staging]root@i-86402c2e:~# ./kvexpress out -k hosts -f darron2 -o dog
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x46f895]

goroutine 1 [running]:
_/Users/DarronFroeseDD/Desktop/kvexpress/kvexpress.GetOwnerId(0x7fff2cfe390f, 0x3, 0x8c6980, 0x3, 0x4500)
    /Users/DarronFroeseDD/Desktop/kvexpress/kvexpress/util.go:50 +0x415
_/Users/DarronFroeseDD/Desktop/kvexpress/kvexpress.WriteFile(0xc82015c500, 0x41d2, 0x7fff2cfe3904, 0x7, 0x1a0, 0x7fff2cfe390f, 0x3, 0x0, 0x0, 0x8c6980, ...)
    /Users/DarronFroeseDD/Desktop/kvexpress/kvexpress/files.go:43 +0xf6
_/Users/DarronFroeseDD/Desktop/kvexpress/commands.outRun(0xafe420, 0xc820018ae0, 0x0, 0x6)
    /Users/DarronFroeseDD/Desktop/kvexpress/commands/out.go:68 +0xb5a
github.com/spf13/cobra.(*Command).execute(0xafe420, 0xc820018900, 0x6, 0x6, 0x0, 0x0)
    /Users/DarronFroeseDD/Dropbox/src/gopath/src/github.com/spf13/cobra/command.go:551 +0x869
github.com/spf13/cobra.(*Command).Execute(0xafe600, 0x0, 0x0)
    /Users/DarronFroeseDD/Dropbox/src/gopath/src/github.com/spf13/cobra/command.go:634 +0x46a
main.main()
    /Users/DarronFroeseDD/Desktop/kvexpress/main.go:33 +0x5be

goroutine 17 [IO wait]:
net.runtime_pollWait(0x7f33a1bb97d8, 0x72, 0xc82000e1c0)
    /usr/local/Cellar/go/1.5.1/libexec/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8200dab50, 0x72, 0x0, 0x0)
    /usr/local/Cellar/go/1.5.1/libexec/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8200dab50, 0x0, 0x0)
    /usr/local/Cellar/go/1.5.1/libexec/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8200daaf0, 0xc8200f4000, 0x1000, 0x1000, 0x0, 0x7f33a1bb4050, 0xc82000e1c0)
    /usr/local/Cellar/go/1.5.1/libexec/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8200ee000, 0xc8200f4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.5.1/libexec/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f33a1b74000, 0xc8200ee000, 0xc8200c3f48, 0xc8200f4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.5.1/libexec/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8200ea040, 0xc8200f4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8200f2000)
    /usr/local/Cellar/go/1.5.1/libexec/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8200f2000, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/Cellar/go/1.5.1/libexec/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8200c3ef0)
    /usr/local/Cellar/go/1.5.1/libexec/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/Cellar/go/1.5.1/libexec/src/net/http/transport.go:685 +0xc78

goroutine 18 [select]:
net/http.(*persistConn).writeLoop(0xc8200c3ef0)
    /usr/local/Cellar/go/1.5.1/libexec/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
    /usr/local/Cellar/go/1.5.1/libexec/src/net/http/transport.go:686 +0xc9d

Looking into it tomorrow.

darron commented 8 years ago

It works just as expected in Vagrant:

  1. Try and chown the file to the owner/group that's passed after it's written.
  2. If there's an error - just keep on and log it:

The command bin/kvexpress out -k sorting -f /home/vagrant/wonder32 -o root:

 kvexpress:  2015/11/16 17:56:38 main: kvexpress version:1.3-dev git:45f14f812e90e87841c5e10c4c876c21b20ce832 date:20151116.165851
 kvexpress:  2015/11/16 17:56:38 out: Checking cli flags.
 kvexpress:  2015/11/16 17:56:38 out: Required cli flags present.
 kvexpress:  2015/11/16 17:56:38 out: path='data' full_path='kvexpress/sorting/data'
 kvexpress:  2015/11/16 17:56:38 out: path='checksum' full_path='kvexpress/sorting/checksum'
 kvexpress:  2015/11/16 17:56:38 out: path='stop' full_path='kvexpress/sorting/stop'
 kvexpress:  2015/11/16 17:56:38 out: server='localhost:8500' token=''
 kvexpress:  2015/11/16 17:56:38 out: action='get' key='kvexpress/sorting/stop'
 kvexpress:  2015/11/16 17:56:38 out: Stop Key is NOT present - continuing.
 kvexpress:  2015/11/16 17:56:38 out: action='get' key='kvexpress/sorting/data'
 kvexpress:  2015/11/16 17:56:38 out: action='get' key='kvexpress/sorting/checksum'
 kvexpress:  2015/11/16 17:56:38 out: length='11' min_length='10'
 kvexpress:  2015/11/16 17:56:38 out: longEnough='true'
 kvexpress:  2015/11/16 17:56:38 out: computed_checksum='0ab71a1c8fef24ade8d650e2cc248aac1e499a45a0e9456ba0b47901f99176d8#012'
 kvexpress:  2015/11/16 17:56:38 out: checksum='0ab71a1c8fef24ade8d650e2cc248aac1e499a45a0e9456ba0b47901f99176d8#012' computed_checksum='0ab71a1c8fef24ade8d650e2cc248aac1e499a45a0e9456ba0b47901f99176d8#012'
 kvexpress:  2015/11/16 17:56:38 out: checksumMatch='true'
 kvexpress:  2015/11/16 17:56:38 out: there is NO file at /home/vagrant/wonder32
 kvexpress:  2015/11/16 17:56:38 out: owner='root' status='found' uid='0'
 kvexpress:  2015/11/16 17:56:38 out: group='' status='not_found' gid='0'
 kvexpress:  2015/11/16 17:56:38 out: file_wrote='true' location='/home/vagrant/wonder32' permissions='416'
 kvexpress:  2015/11/16 17:56:38 out: file_chown='true' location='/home/vagrant/wonder32' owner='0' group='0'
 kvexpress:  2015/11/16 17:56:38 out: location='complete', elapsed='48.894976ms'
darron commented 8 years ago

Testing again on staging. It works fine.

I may have uploaded the wrong binary for testing.