tmthrgd / go-bindata

A small utility which generates Go code from any file. Useful for embedding binary data in a Go program. [Obsolete].
https://golang.org/issue/35950
Other
72 stars 4 forks source link

Documentation is out of date. #2

Open dpastoor opened 7 years ago

dpastoor commented 7 years ago

Would it be possible to note the biggest changes or provide some examples with the new API for how to use this version? I'd be happy to flesh it out and put in a PR if you can give some starting information

tmthrgd commented 7 years ago

@dpastoor I'm glad you've come across my former fork. The command line API should be entirely backward compatible but doesn't have access to any of the newer features I've added and I don't guarantee it will stay long-term.

The biggest change to the project itself was using text/template to produce all the generated code. That's an implementation detail though and should not be visible to any users.

The package API has changed quite a lot from lestrrat/go-bindata. I unexported everything that should have been internal to the implementation (like StringWriter).

Where before you called Translate which read each file from disk and generated the go code, you now call FindFiles once for each directory you want to read and then Files.Generate to generate the code. Files is just a typed slice, so you can append to it to merge multiple directories together. All of the options around walking the directory tree (prefix, recursive and ignore ! ) are provided to FindFiles as a FindFilesOptions and all of the options that effect code generation are passed to Files.Generate as a GenerateOptions.

I added support for hash.Hash based name mangling which I'm using to make static assets content-addressable (somewhat like in this article by Facebook).

Edit: I should also add, the generated code is now correctly formatted and this is enforced with a test in format_test.go. I've also added support for a test corpus in corpus_test.go which I can compare against between commits to check for any unintended changes.

Edit: Files.Generate operates on a File interface now, so it's now possible to generate embedded code from something other than the filesystem. You can see an example of that in testdata_test.go which generates pseudo-random files for the test suite.

I'll attach an excerpt of a go:generate file I'm using in another comment.

tmthrgd commented 7 years ago
// Copyright 2017 Tom Thorogood. All rights reserved.
// Use of this source code is governed by a Modified
// BSD License that can be found in the LICENSE file.

// +build ignore

package main

// ...

func createBindata(output, input string, genOpts *bindata.GenerateOptions, ignore []*regexp.Regexp) error {
    if err := os.MkdirAll(filepath.Dir(output), 0744); err != nil {
        return err
    }

    files, err := bindata.FindFiles(input, &bindata.FindFilesOptions{
        Prefix:    input,
        Recursive: true,
        Ignore:    ignore,
    })
    if err != nil {
        return err
    }

    f, err := os.OpenFile(output, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0666)
    if err != nil {
        return err
    }
    defer f.Close()

    return files.Generate(f, genOpts)
}

func main() {
    // walkSass compiles each .scss file in assets into .asset-cache
    if err := walkSass(".asset-cache", "assets"); err != nil {
        panic(err)
    }

    // walkCompress compress each file in .asset-cache with gzip (zopfli)
    // and brotli into subdirectories of .compress-cache
    if err := walkCompress(".compress-cache", ".asset-cache"); err != nil {
        panic(err)
    }

    h, _ := blake2b.New512(nil)
    if err := createBindata("internal/assets/bindata.go", "assets", &bindata.GenerateOptions{
        Package:  "assets",
        Metadata: true,
        Mode:     0444,
        Hash:     h,
    }, []*regexp.Regexp{
        regexp.MustCompile(`\.scss$`),
    }); err != nil {
        panic(err)
    }

    if err := createBindata("internal/brotli/bindata.go", ".compress-cache/br", &bindata.GenerateOptions{
        Package: "brotli",
    }, nil); err != nil {
        panic(err)
    }

    if err := createBindata("internal/gzip/bindata.go", ".compress-cache/gz", &bindata.GenerateOptions{
        Package: "gzip",
    }, nil); err != nil {
        panic(err)
    }

    h, _ = blake2b.New512(nil)
    if err := createBindata("internal/hashassets/bindata.go", ".asset-cache", &bindata.GenerateOptions{
        Package:      "hashassets",
        ModTime:      978310800, // "Mon, 01 Jan 2001 01:00:00 GMT"
        Hash:         h,
        HashFormat:   bindata.DirHash,
        HashLength:   12,
        HashEncoding: bindata.Base64Hash,
    }, nil); err != nil {
        panic(err)
    }

    if err := createBindata("internal/views/bindata.go", "views", &bindata.GenerateOptions{
        Package:  "views",
        Metadata: true,
        Mode:     0444,
    }, nil); err != nil {
        panic(err)
    }
}

This file is invoked with //go:generate go run generate.go.

Then I use the views template with MustAsset for html/template. assets, gzip, brotli and hashassets are then used for a http router like the following:

func getAssetRouter(h http.Handler) http.Handler {
    router := httprouter.New()
    router.NotFound = h
    router.HandleMethodNotAllowed = false

    assetsHandler := httpasset.NewWithETag(assets.AssetAndInfo, 32)
    assetsHandler = handlers.SetHeader(assetsHandler, "Cache-Control", "public, max-age=604800")
    router.GetAndHead("/favicon.ico", assetsHandler)
    router.GetAndHead("/robots.txt", assetsHandler)

    hashAssets := httprouter.PathHandler(&httpasset.FileServer{
        Asset:  hashassets.AssetAndInfo,
        Brotli: brotli.AssetAndInfo,
        Gzip:   gzip.AssetAndInfo,
    })
    hashAssets = handlers.NeverModified(hashAssets)
    hashAssets = handlers.SetHeader(hashAssets, "Cache-Control", "public, max-age=31536000, immutable")
    router.GetAndHead("/assets/*filepath", hashAssets)

    return router
}

httpasset provides methods to serve the files over http. Before I forked, I was using go-bindata-assetfs for that purpose. (I'm using my fork of julienschmidt/httprouter in that example and methods from tmthrgd/httphandlers).

This is only one particular way of using this package. It's the one that's working best for me but YMMV. I hope that helps and gives you a solid understanding of this package. If you feel like contributing better documentation, that would be most appreciated and valued.

steigr commented 6 years ago

Is there a working example of httpasset?

tmthrgd commented 6 years ago

@steigr The example in my last comment should work without issue. What are you having trouble with?

steigr commented 6 years ago

Checking tomorrow. :-)