pshima / consul-snapshot

consul-snapshot is a backup and restore utility for Consul (https://www.consul.io). This is slightly different than some other utilities out there as this runs as a daemon for backups and ships them to S3. Also has integrated monitoring and backup health checks.
Apache License 2.0
116 stars 35 forks source link

Upload to S3 with SSE #9

Closed jason-riddle closed 8 years ago

jason-riddle commented 8 years ago

In my organization, uploading using SSE (server side encryption) is enforced through bucket policies. It looks like in writeBackupRemote there is no option to specify SSE.

https://github.com/pshima/consul-snapshot/blob/master/backup/backup.go#L300-L327.

func (b *Backup) writeBackupRemote() {
    s3Conn := session.New(&aws.Config{Region: aws.String(string(b.Config.S3Region))})

    t := time.Unix(b.StartTime, 0)
    remotePath := fmt.Sprintf("backups/%v/%d/%v/%v", t.Year(), t.Month(), t.Day(), filepath.Base(b.FullFilename))

    b.RemoteFilePath = remotePath

    // re-read the compressed file.  There is probably a better way to do this
    localFileContents, err := ioutil.ReadFile(b.FullFilename)
    if err != nil {
        log.Fatalf("[ERR] Could not read compressed file!: %v", err)
    }

    // Create the params to pass into the actual uploader
    params := &s3manager.UploadInput{
        Bucket: &b.Config.S3Bucket,
        Key:    &b.RemoteFilePath,
        Body:   bytes.NewReader(localFileContents),
    }

    log.Printf("[INFO] Uploading %v/%v to S3 in %v", string(b.Config.S3Bucket), b.RemoteFilePath, string(b.Config.S3Region))
    uploader := s3manager.NewUploader(s3Conn)
    _, err = uploader.Upload(params)
    if err != nil {
        log.Fatalf("[ERR] Could not upload to S3!: %v", err)
    }
}

What would be the cleanest way to add this option? Would it just be easiest to add another environment variable and check to see if it's set?

pshima commented 8 years ago

Hi @jason-riddle, that seems like a good option to add! I think adding an environment variable such as S3SSEENABLED or something similar would do the trick! If you want to do a PR I would have a look or can add this to the feature list.

jason-riddle commented 8 years ago

So I thought I would have time to work on this sometime this week, but now I'm not sure anymore. Can this at least be added to the feature list?