Identify generated files

awalterschulze commented 7 years ago

We have added a comment above in each file saying that it is generated and that helps alot. It would be even better if we could add a file extension. protobuf has .pb.go Maybe we could have .gocc.go This will also help when wanting to clean up after gocc, because its easier to identify the generated files this way.

mewmew commented 7 years ago

To my knowledge, only protobuf uses this behaviour. Most generated files of the Go standard library still uses a .go extension.

That being said, I can't say I would be against in proposal as such.

List (loong list) of Go source files from the standard library containing the word "generated":

Only one Go file has two extensions, namely cmd/internal/obj/arm64/a.out.go.

archive/zip/reader_test.go
bufio/scan.go
bufio/scan_test.go
cmd/asm/internal/asm/endtoend_test.go
cmd/cgo/gcc.go
cmd/cgo/main.go
cmd/cgo/out.go
cmd/cgo/zdefaultcc.go
cmd/compile/internal/amd64/ggen.go
cmd/compile/internal/amd64/gsubr.go
cmd/compile/internal/amd64/prog.go
cmd/compile/internal/amd64/ssa.go
cmd/compile/internal/arm/ggen.go
cmd/compile/internal/arm/prog.go
cmd/compile/internal/arm64/ggen.go
cmd/compile/internal/arm64/prog.go
cmd/compile/internal/big/accuracy_string.go
cmd/compile/internal/big/float.go
cmd/compile/internal/big/ftoa.go
cmd/compile/internal/big/intconv.go
cmd/compile/internal/big/natconv.go
cmd/compile/internal/big/roundingmode_string.go
cmd/compile/internal/gc/asm_test.go
cmd/compile/internal/gc/bexport.go
cmd/compile/internal/gc/bimport.go
cmd/compile/internal/gc/builtin.go
cmd/compile/internal/gc/cgen.go
cmd/compile/internal/gc/closure.go
cmd/compile/internal/gc/esc.go
cmd/compile/internal/gc/export.go
cmd/compile/internal/gc/lex.go
cmd/compile/internal/gc/main.go
cmd/compile/internal/gc/mkbuiltin.go
cmd/compile/internal/gc/opnames.go
cmd/compile/internal/gc/order.go
cmd/compile/internal/gc/pgen.go
cmd/compile/internal/gc/popt.go
cmd/compile/internal/gc/reflect.go
cmd/compile/internal/gc/reg.go
cmd/compile/internal/gc/sinit.go
cmd/compile/internal/gc/ssa.go
cmd/compile/internal/gc/swt.go
cmd/compile/internal/gc/testdata/copy_ssa.go
cmd/compile/internal/gc/testdata/gen/arithBoundaryGen.go
cmd/compile/internal/gc/testdata/gen/arithConstGen.go
cmd/compile/internal/gc/testdata/gen/constFoldGen.go
cmd/compile/internal/gc/testdata/gen/copyGen.go
cmd/compile/internal/gc/testdata/gen/zeroGen.go
cmd/compile/internal/gc/testdata/zero_ssa.go
cmd/compile/internal/gc/typecheck.go
cmd/compile/internal/gc/walk.go
cmd/compile/internal/mips64/ggen.go
cmd/compile/internal/mips64/prog.go
cmd/compile/internal/ppc64/ggen.go
cmd/compile/internal/ppc64/prog.go
cmd/compile/internal/s390x/ggen.go
cmd/compile/internal/s390x/prog.go
cmd/compile/internal/ssa/compile.go
cmd/compile/internal/ssa/dom_test.go
cmd/compile/internal/ssa/gen/AMD64Ops.go
cmd/compile/internal/ssa/gen/main.go
cmd/compile/internal/ssa/gen/rulegen.go
cmd/compile/internal/ssa/nilcheck.go
cmd/compile/internal/ssa/nilcheck_test.go
cmd/compile/internal/ssa/opGen.go
cmd/compile/internal/ssa/prove.go
cmd/compile/internal/ssa/regalloc.go
cmd/compile/internal/ssa/rewrite.go
cmd/compile/internal/ssa/rewriteAMD64.go
cmd/compile/internal/ssa/rewriteARM.go
cmd/compile/internal/ssa/rewritedec.go
cmd/compile/internal/ssa/rewritegeneric.go
cmd/compile/internal/x86/ggen.go
cmd/compile/internal/x86/gsubr.go
cmd/compile/internal/x86/prog.go
cmd/cover/profile.go
cmd/dist/build.go
cmd/dist/buildgo.go
cmd/dist/buildruntime.go
cmd/dist/buildtool.go
cmd/dist/deps.go
cmd/doc/dirs.go
cmd/fix/main.go
cmd/go/alldocs.go
cmd/go/build.go
cmd/go/main.go
cmd/go/pkg.go
cmd/go/test.go
cmd/go/zdefaultcc.go
cmd/go/zosarch.go
cmd/internal/obj/arm/anames.go
cmd/internal/obj/arm/asm5.go
cmd/internal/obj/arm64/a.out.go
cmd/internal/obj/arm64/anames.go
cmd/internal/obj/arm64/asm7.go
cmd/internal/obj/funcdata.go
cmd/internal/obj/mips/anames.go
cmd/internal/obj/obj.go
cmd/internal/obj/ppc64/anames.go
cmd/internal/obj/ppc64/obj9.go
cmd/internal/obj/s390x/anames.go
cmd/internal/obj/stringer.go
cmd/internal/obj/x86/anames.go
cmd/internal/obj/x86/asm6.go
cmd/internal/obj/zbootstrap.go
cmd/internal/pprof/commands/commands.go
cmd/link/internal/ld/data.go
cmd/link/internal/ld/go.go
cmd/link/internal/ld/lib.go
cmd/link/internal/ld/macho_combine_dwarf.go
cmd/yacc/yacc.go
compress/flate/huffman_bit_writer.go
compress/flate/huffman_code.go
compress/zlib/reader_test.go
crypto/dsa/dsa.go
crypto/elliptic/elliptic.go
crypto/md5/gen.go
crypto/rand/rand_unix.go
crypto/rsa/pkcs1v15_test.go
crypto/rsa/pss_test.go
crypto/rsa/rsa.go
crypto/rsa/rsa_test.go
crypto/tls/common.go
crypto/tls/handshake_server_test.go
crypto/tls/prf_test.go
crypto/x509/pem_decrypt_test.go
crypto/x509/pkcs8_test.go
crypto/x509/sec1_test.go
crypto/x509/x509_test.go
database/sql/driver/driver.go
database/sql/sql.go
debug/dwarf/class_string.go
debug/dwarf/line.go
debug/dwarf/line_test.go
debug/elf/symbols_test.go
debug/gosym/symtab.go
encoding/encoding.go
encoding/gob/error.go
encoding/json/encode.go
encoding/xml/marshal.go
encoding/xml/xml.go
fmt/scan.go
go/ast/ast.go
go/build/zcgo.go
go/doc/testdata/testing.go
go/internal/gccgoimporter/importer.go
go/internal/gcimporter/gcimporter.go
go/printer/nodes.go
go/types/exprstring.go
go/types/hilbert_test.go
go/types/typestring.go
image/color/palette/gen.go
image/color/palette/palette.go
image/color/ycbcr_test.go
image/internal/imageutil/gen.go
image/internal/imageutil/impl.go
image/jpeg/dct_test.go
image/jpeg/reader_test.go
internal/syscall/windows/registry/zsyscall_windows.go
internal/syscall/windows/sysdll/sysdll.go
internal/syscall/windows/zsyscall_windows.go
log/log.go
math/big/accuracy_string.go
math/big/float.go
math/big/ftoa.go
math/big/intconv.go
math/big/natconv.go
math/big/roundingmode_string.go
math/rand/rand.go
math/rand/regress_test.go
mime/multipart/writer.go
net/dnsclient_unix_test.go
net/http/h2_bundle.go
net/http/httptest/recorder.go
net/http/internal/chunked.go
net/http/internal/testcert.go
net/http/serve_test.go
net/http/server.go
net/smtp/smtp_test.go
reflect/all_test.go
reflect/type.go
reflect/value.go
regexp/exec_test.go
regexp/syntax/doc.go
regexp/syntax/perl_groups.go
runtime/cgo.go
runtime/cgocall.go
runtime/defs_darwin_arm.go
runtime/error.go
runtime/fastlog2table.go
runtime/internal/sys/gengoos.go
runtime/internal/sys/zgoarch_386.go
runtime/internal/sys/zgoarch_amd64.go
runtime/internal/sys/zgoarch_amd64p32.go
runtime/internal/sys/zgoarch_arm.go
runtime/internal/sys/zgoarch_arm64.go
runtime/internal/sys/zgoarch_mips64.go
runtime/internal/sys/zgoarch_mips64le.go
runtime/internal/sys/zgoarch_ppc64.go
runtime/internal/sys/zgoarch_ppc64le.go
runtime/internal/sys/zgoarch_s390x.go
runtime/internal/sys/zgoos_android.go
runtime/internal/sys/zgoos_darwin.go
runtime/internal/sys/zgoos_dragonfly.go
runtime/internal/sys/zgoos_freebsd.go
runtime/internal/sys/zgoos_linux.go
runtime/internal/sys/zgoos_nacl.go
runtime/internal/sys/zgoos_netbsd.go
runtime/internal/sys/zgoos_openbsd.go
runtime/internal/sys/zgoos_plan9.go
runtime/internal/sys/zgoos_solaris.go
runtime/internal/sys/zgoos_windows.go
runtime/internal/sys/zversion.go
runtime/mbitmap.go
runtime/mkduff.go
runtime/mkfastlog2table.go
runtime/pprof/pprof_test.go
runtime/rune.go
runtime/signal_amd64x.go
runtime/time.go
runtime/wincallback.go
runtime/zcallback_windows.go
strconv/ftoa.go
strconv/isprint.go
strconv/itoa.go
strconv/makeisprint.go
strconv/quote.go
strings/replace_test.go
syscall/exec_solaris.go
syscall/mkpost.go
syscall/mksyscall_windows.go
syscall/zerrors_darwin_386.go
syscall/zerrors_darwin_amd64.go
syscall/zerrors_darwin_arm.go
syscall/zerrors_darwin_arm64.go
syscall/zerrors_dragonfly_amd64.go
syscall/zerrors_freebsd_386.go
syscall/zerrors_freebsd_amd64.go
syscall/zerrors_freebsd_arm.go
syscall/zerrors_linux_386.go
syscall/zerrors_linux_amd64.go
syscall/zerrors_linux_arm.go
syscall/zerrors_linux_arm64.go
syscall/zerrors_linux_mips64.go
syscall/zerrors_linux_mips64le.go
syscall/zerrors_linux_ppc64.go
syscall/zerrors_linux_ppc64le.go
syscall/zerrors_linux_s390x.go
syscall/zerrors_netbsd_386.go
syscall/zerrors_netbsd_amd64.go
syscall/zerrors_netbsd_arm.go
syscall/zerrors_openbsd_386.go
syscall/zerrors_openbsd_amd64.go
syscall/zerrors_openbsd_arm.go
syscall/zerrors_solaris_amd64.go
syscall/zerrors_windows.go
syscall/zsyscall_darwin_386.go
syscall/zsyscall_darwin_amd64.go
syscall/zsyscall_darwin_arm.go
syscall/zsyscall_darwin_arm64.go
syscall/zsyscall_dragonfly_amd64.go
syscall/zsyscall_freebsd_386.go
syscall/zsyscall_freebsd_amd64.go
syscall/zsyscall_freebsd_arm.go
syscall/zsyscall_linux_386.go
syscall/zsyscall_linux_amd64.go
syscall/zsyscall_linux_arm.go
syscall/zsyscall_linux_arm64.go
syscall/zsyscall_linux_mips64.go
syscall/zsyscall_linux_mips64le.go
syscall/zsyscall_linux_ppc64.go
syscall/zsyscall_linux_ppc64le.go
syscall/zsyscall_linux_s390x.go
syscall/zsyscall_nacl_386.go
syscall/zsyscall_nacl_amd64p32.go
syscall/zsyscall_nacl_arm.go
syscall/zsyscall_netbsd_386.go
syscall/zsyscall_netbsd_amd64.go
syscall/zsyscall_netbsd_arm.go
syscall/zsyscall_openbsd_386.go
syscall/zsyscall_openbsd_amd64.go
syscall/zsyscall_openbsd_arm.go
syscall/zsyscall_plan9_386.go
syscall/zsyscall_plan9_amd64.go
syscall/zsyscall_plan9_arm.go
syscall/zsyscall_solaris_amd64.go
syscall/zsyscall_windows.go
syscall/zsysctl_openbsd.go
syscall/zsysnum_darwin_386.go
syscall/zsysnum_darwin_amd64.go
syscall/zsysnum_darwin_arm.go
syscall/zsysnum_darwin_arm64.go
syscall/zsysnum_dragonfly_amd64.go
syscall/zsysnum_freebsd_386.go
syscall/zsysnum_freebsd_amd64.go
syscall/zsysnum_freebsd_arm.go
syscall/zsysnum_linux_386.go
syscall/zsysnum_linux_amd64.go
syscall/zsysnum_linux_arm.go
syscall/zsysnum_linux_arm64.go
syscall/zsysnum_linux_mips64.go
syscall/zsysnum_linux_mips64le.go
syscall/zsysnum_linux_ppc64.go
syscall/zsysnum_linux_ppc64le.go
syscall/zsysnum_linux_s390x.go
syscall/zsysnum_netbsd_386.go
syscall/zsysnum_netbsd_amd64.go
syscall/zsysnum_netbsd_arm.go
syscall/zsysnum_openbsd_386.go
syscall/zsysnum_openbsd_amd64.go
syscall/zsysnum_openbsd_arm.go
syscall/zsysnum_plan9.go
testing/cover.go
testing/testing.go
time/format.go
time/genzabbrs.go
time/time_test.go
time/zoneinfo_abbrs_windows.go
unicode/casetables.go
unicode/letter.go
unicode/maketables.go
unicode/tables.go

awalterschulze commented 7 years ago

Hmmm very very interesting.

awalterschulze commented 7 years ago

It would be nice not to have to write this code though

nuke: clean
    rm -rf ./errors/errors.go
    rm -rf ./lexer/acttab.go
    rm -rf ./lexer/lexer.go
    rm -rf ./lexer/transitiontable.go
    rm -rf ./parser/action.go
    rm -rf ./parser/actiontable.go
    rm -rf ./parser/gototable.go
    rm -rf ./parser/parser.go
    rm -rf ./parser/productionstable.go
    rm -rf ./token/token.go
    rm -rf ./util

https://github.com/katydid/katydid/blob/master/relapse/Makefile#L33

mewmew commented 7 years ago

How about introducing gocc clean as a subcommand?

awalterschulze commented 7 years ago

That sounds really good, but extensions are useful for other things as well. For instance adding *.gocc.go binary to .gitattributes can be used to eliminate generated code from the git diff for reviews. Its not always what you want, but its a nice option to have in a project that has nothing to do with gocc, but is simply a user.

awalterschulze commented 7 years ago

So I am on the fence now. Maybe someone else can make a compelling argument either way.

mewmew commented 7 years ago

*.gocc.go binary to .gitattributes can be used to eliminate generated code from the git diff for reviews.

Makes sense. Also, this would make it easier to ignore Gocc generated files from golint, at least until issues such as #36 are resolved.

So, if you which to use this now in one of your projects, you have a +1 from me. As long as its not just an imagined feature, but one that you would use today. I think I might use it in the future, and extensions make it more flexible.

awalterschulze commented 7 years ago

I think I should add the gocc clean command with this change, to show that the file extensions really makes it easier.

sangisos commented 7 years ago

Is there a way in go to remove directories only if empty?

sangisos commented 7 years ago

Sorry for the noise.

os.Remove will do it fine I guess, returning an error if directory not empty I understood it as.

awalterschulze commented 7 years ago

@sangisos So we could just remove the folders containing the generated code, but I typically add more code to the parser folder at least. And I sometimes like to add a doc.go to the other folders.

sangisos commented 7 years ago

@awalterschulze Yes, sorry if I was unclear. You first remove all induvidual generated files, then you try to remove the directories. As I understand the os.Remove command, it will only remove a directory if it is empty, so it should be safe to use even when you have your own files inside the directories. It will then not remove the directory leaving your non-generated files in the directory.

dgryski commented 6 years ago

awalterschulze commented 6 years ago

We have already updated the code generator to add the suggested comment to the files. But we have not implemented any tooling like gocc clean yet. Does anyone know whether go is thinking of creating a go generate clean command, so that we don't have to do double work here?

On Fri, 29 Dec 2017, 07:30 Damian Gryski, notifications@github.com wrote:

Related, golang/go#13560 https://github.com/golang/go/issues/13560

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/goccmack/gocc/issues/40#issuecomment-354404588, or mute the thread https://github.com/notifications/unsubscribe-auth/ABvsLZR5x_KRzKedGA1zUwDn7lcyzoT3ks5tFIbtgaJpZM4LKz4j .

goccmack / gocc

Identify generated files #40