roscopecoltran
commented
5 years ago Hi Dave,
Hope you are all well again !
Parse package AST and repack codebases based on custom templates
Context
I have forked the amalgomate program which is go program, that programmatically combines the source code of separate stand-alone Go programs into a single combined program. It also provides a library and invocation mechanism for invoking the original programs from the combined program. Amalgomate can be used to create a single executable that combines the functionality of several stand-alone executables. This provides advantages such as reducing the over-all size, locking in versions, creating a single point of execution and allowing the program to implement logic that makes use of its component programs.
Goals
Parsing go packages with the forky/golib/dst combo and be able to match some user-defined declarations in order to repack/rebundle a unique package or some amalgomated packages.
Playground
I created some public playgrounds for making everybody's life easier but it won't work as play.jsgo.io's filesystem is not writable but you can test it locally. Link: https://play.jsgo.io/df0b88edb687b16597f41a33d6fd4a70b8799d23
Real-World Use cases
Repacking flags from native flags or most popular cli libs declarations
- sflags, parse flags for commands and sub-commands by and match some ast declaration and convert/unify original flags commands into parsable structures with sflags.
- sflag examples: https://github.com/octago/sflags/tree/master/examples
- amalgomate
- native flag repacking: https://github.com/palantir/amalgomate/blob/master/amalgomate/astfuncs.go#L122
- amalgomated main package, actually looks like the code excerpt below, but it doesn't unify or expand available sub-commands (default values or help messages)
// generated by amalgomate; DO NOT EDIT package main
import ( "fmt" "os" "sort"
flogogen "cranepp/internal/github.com/TIBCOSoftware/flogo-cli/cmd/flogogen"
swaggable "cranepp/internal/github.com/artheus/swaggable"
wuzz "cranepp/internal/github.com/asciimoo/wuzz"
gotodo "cranepp/internal/github.com/asticode/go-astitodo/astitodo"
jargon "cranepp/internal/github.com/clipperhouse/jargon/cmd/jargon"
forky "cranepp/internal/github.com/dave/forky/forkgo"
gocmplx "cranepp/internal/github.com/drathier/gocmplx"
zoekt "cranepp/internal/github.com/google/zoekt/cmd/zoekt"
zoekt_git_index "cranepp/internal/github.com/google/zoekt/cmd/zoekt-git-index"
zoekt_index "cranepp/internal/github.com/google/zoekt/cmd/zoekt-index"
htcat "cranepp/internal/github.com/htcat/htcat/cmd/htcat"
autoreadme "cranepp/internal/github.com/jimmyfrasche/autoreadme"
doc2json "cranepp/internal/github.com/jimmyfrasche/doc2json"
mage "cranepp/internal/github.com/magefile/mage"
crane "cranepp/internal/github.com/michaelsauter/crane"
interfacer "cranepp/internal/github.com/orisano/impast/cmd/interfacer"
mocker "cranepp/internal/github.com/orisano/impast/cmd/mocker"
stuber "cranepp/internal/github.com/orisano/impast/cmd/stuber"
fgen "cranepp/internal/github.com/retgits/fgen"
fweb "cranepp/internal/github.com/retgits/flogowebhelper"
src_cli "cranepp/internal/github.com/sourcegraph/src-cli/cmd/src"
src_scan "cranepp/internal/github.com/sourcegraph/srcscan/cmd/srcscan"
zoekt_sg_indexer "cranepp/internal/github.com/sourcegraph/zoekt/cmd/zoekt-sourcegraph-indexserver"
jsonschema2go "cranepp/internal/github.com/taskcluster/jsonschema2go/jsonschema2go"
gomutate "cranepp/internal/github.com/zabawaba99/gomutate/cmd/gomutate")
var programs = map[string]func(){"autoreadme": func() { autoreadme.RepackedMain() }, "crane": func() { crane.RepackedMain() }, "doc2json": func() { doc2json.RepackedMain() }, "fgen": func() { fgen.RepackedMain() }, "flogogen": func() { flogogen.RepackedMain() }, "forky": func() { forky.RepackedMain() }, "fweb": func() { fweb.RepackedMain() }, "gocmplx": func() { gocmplx.RepackedMain() }, "gomutate": func() { gomutate.RepackedMain() }, "gotodo": func() { gotodo.RepackedMain() }, "htcat": func() { htcat.RepackedMain() }, "interfacer": func() { interfacer.RepackedMain() }, "jargon": func() { jargon.RepackedMain() }, "jsonschema2go": func() { jsonschema2go.RepackedMain() }, "mage": func() { mage.RepackedMain() }, "mocker": func() { mocker.RepackedMain() }, "src-cli": func() { src_cli.RepackedMain() }, "src-scan": func() { src_scan.RepackedMain() }, "stuber": func() { stuber.RepackedMain() }, "swaggable": func() { swaggable.RepackedMain() }, "wuzz": func() { wuzz.RepackedMain() }, "zoekt": func() { zoekt.RepackedMain() }, "zoekt-git-index": func() { zoekt_git_index.RepackedMain() }, "zoekt-index": func() { zoekt_index.RepackedMain() }, "zoekt-sg-indexer": func() { zoekt_sg_indexer.RepackedMain() }, }
func main() { if len(os.Args) < 2 { fmt.Println("Missing program argument. Valid values:", cmds()) os.Exit(1) }
programName := os.Args[1]
if _, ok := programs[programName]; !ok {
fmt.Printf("Unknown program: \"%v\". Valid values: %v\n", programName, cmds())
os.Exit(1)
}
os.Args = append([]string{os.Args[0]}, os.Args[2:]...)
programs[programName]()}
func cmds() []string { var cmds []string for key := range programs { cmds = append(cmds, key) } sort.Strings(cmds) return cmds }
#### **Go Plugins**
Another use cases could be for go plugins; parse/match a package ast tree and unmarshal go plugins into/from structs. eg .https://github.com/tcard/pluginunmarshal,
#### **Flogo Components/Contribs from a go package**
- generate flogo contribs on the fly based on templates, erf. https://github.com/retgits/fgen/blob/master/gen/activity.go#L60
- Good example, would be to create alternative activity components like https://github.com/mmussett/flogo-components/tree/master/activity/transform but from https://github.com/agrison/go-tablib. We could match exported functions with their arguments and generate the component by casting types of inputs (ref. https://github.com/mmussett/flogo-components/blob/master/activity/transform/activity.go#L37-L38)
## Epilogue :-)
Hope that it sounds clear to you as it is only to assist in repacking librairies or programs by mixing amalgomate features with forky/dst features.... The real plus for all things mentioned above, it is that you have a UI for quick playgrounds that could be generated/repacked with a common/unique codebase.
Cheers,
Rosco
dave
Same as https://github.com/dave/forky/issues/5 but for the
flagpackage.