gobuffalo / buffalo

Rapid Web Development w/ Go
http://gobuffalo.io
MIT License
8.08k stars 578 forks source link

buffalo build uses stale output in ./public/assets #1561

Closed hdm closed 5 years ago

hdm commented 5 years ago

Description

The buffalo build command will include all files in ./public/assets in the binary at build-time. This includes all of the previously-versioned asset folders, older assets, etc. In one particularly ugly case, this resulted in 20Gb of memory usage and 450Mb binary files (with 30 minute build times).

Steps to Reproduce the Problem

  1. Place lots of random files in ./public/assets
  2. Run buffalo build
  3. Notice the increased file size

Expected Behavior

I expected buffalo build to regenerate the entire assets directory on each build, removing old files. On a system with 6 months of development work, this directory had grown to over 3Gb, and the resulting build times, memory usage, and binary files were all huge as a result.

Actual Behavior

The buffalo build command would purge the contents of ./public/assets on each build. Doing this manually currently fails due to #1560

Info

Please run buffalo info and paste the information below where it says "PASTE_HERE".

``` ### Buffalo Version v0.14.0-beta.3 ### App Information Pwd=C:\Users\Developer\go\src\github.com\myuser\myapp-console Root=C:\Users\Developer\go\src\github.com\myuser\myapp-console GoPath=C:\Users\Developer\go PackagePkg=github.com/myuser/myapp-console ActionsPkg=github.com/myuser/myapp-console/actions ModelsPkg=github.com/myuser/myapp-console/models GriftsPkg=github.com/myuser/myapp-console/grifts WithModules=false Name=myapp-console Bin=bin\myapp-console.exe VCS=git WithPop=true WithSQLite=false WithDep=true WithWebpack=true WithNodeJs=true WithYarn=true WithDocker=false WithGrifts=true AsWeb=true AsAPI=false ### Go Version go version go1.11.4 windows/amd64 ### Go Env set GOARCH=amd64 set GOBIN= set GOCACHE=C:\Users\Developer\AppData\Local\go-build set GOEXE=.exe set GOFLAGS= set GOHOSTARCH=amd64 set GOHOSTOS=windows set GOOS=windows set GOPATH=C:\Users\Developer\go set GOPROXY= set GORACE= set GOROOT=C:\Go set GOTMPDIR= set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64 set GCCGO=gccgo set CC=gcc set CXX=g++ set CGO_ENABLED=1 set GOMOD= set CGO_CFLAGS=-g -O2 set CGO_CPPFLAGS= set CGO_CXXFLAGS=-g -O2 set CGO_FFLAGS=-g -O2 set CGO_LDFLAGS=-g -O2 set PKG_CONFIG=pkg-config set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\DEVELO~1\AppData\Local\Temp\go-build989579850=/tmp/go-build -gno-record-gcc-switches ### Node Version v10.15.0 ### NPM Version 6.4.1 ### Yarn Version 1.13.0 ### PostgreSQL Version PostgreSQL Not Found ### MySQL Version MySQL Not Found ### SQLite Version 3.7.17 2013-05-20 00:56:22 118a3b35693b134d56ebd780123b7fd6f1497668 ### Dep Version dep: version : devel build date : git hash : go version : go1.11.1 go compiler : gc platform : windows/amd64 features : ImportDuringSolve=false ### Dep Status Gopkg.lock is out of sync with imports and/or Gopkg.toml. Run `dep check` for details. PROJECT MISSING PACKAGES input-digest mismatch ```
markbates commented 5 years ago

I'm not an expert on webpack stuff, so I don't have any answers, just more questions. :)

We can add a flag, --clean-assets, to delete the folder before we start the building process. I don't want to auto delete everything, as people put other things in that folder, that aren't webpack, and I don't want to destroy them.

My real question is, and one I don't have any answer to is, shouldn't webpack be cleaning up the old files when it generates new ones? I feel like it should.

@paganotoni might be able to shed more light on this issue than I can (which is none, none more light). :)