frou / yt2pod

Daemon that monitors YouTube channels and publishes audio podcasts of them
GNU Affero General Public License v3.0
112 stars 12 forks source link

fix(docker): docker image was no longer building #41

Closed sabberworm closed 2 years ago

sabberworm commented 2 years ago

I had the following errors when building the previous docker image:

#6 21.93     running build_ext
#6 21.93     building 'Cryptodome.Hash._MD2' extension
#6 21.93     creating build/temp.linux-x86_64-3.9
#6 21.93     creating build/temp.linux-x86_64-3.9/src
#6 21.93     gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fomit-frame-pointer -g -fno-semantic-interposition -fomit-frame-pointer -g -fno-semantic-interposition -fomit-frame-pointer -g -fno-semantic-interposition -DTHREAD_STACK_SIZE=0x100000 -fPIC -DPYCRYPTO_LITTLE_ENDIAN -DSYS_BITS=64 -DLTC_NO_ASM -Isrc/ -I/usr/include/python3.9 -c src/MD2.c -o build/temp.linux-x86_64-3.9/src/MD2.o
#6 21.93     error: command 'gcc' failed: No such file or directory
#6 21.93     ----------------------------------------
#6 21.93 ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ns_w5dpx/pycryptodomex_83e19353a64a47d6bffa73ff32039b5c/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ns_w5dpx/pycryptodomex_83e19353a64a47d6bffa73ff32039b5c/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-il14ja9p/install-record.txt --single-version-externally-managed --compile --install-headers /usr/include/python3.9/pycryptodomex Check the logs for full command output.

and

 > [stage-0 5/5] RUN go get -d ./...  && go install:                                                                                                                                                                                             
#13 2.328 go: downloading github.com/frou/stdext v0.0.0-20190909174947-20d00569c4d9                                                                                                                                                              
#13 2.329 go: downloading github.com/frou/poor-mans-generics v0.0.0-20181024095521-2ccba2ef9f15                                                                                                                                                  
#13 2.330 go: downloading github.com/go-playground/validator/v10 v10.9.0                                                                                                                                                                         
#13 2.334 go: downloading github.com/jbub/podcasts v0.1.0                                                                                                                                                                                        
#13 2.343 go: downloading github.com/magefile/mage v1.11.0                                                                                                                                                                                       
#13 2.423 go: downloading github.com/snapas/resize v1.0.0                                                                                                                                                                                        
#13 2.448 go: downloading github.com/tzdybal/go-disk-usage v1.0.0                                                                                                                                                                                
#13 2.546 go: downloading google.golang.org/api v0.56.0                                                                                                                                                                                          
#13 3.722 go: downloading github.com/go-playground/universal-translator v0.18.0                                                                                                                                                                  
#13 3.722 go: downloading github.com/leodido/go-urn v1.2.1                                                                                                                                                                                       
#13 3.847 go: downloading golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97
#13 4.217 go: downloading golang.org/x/text v0.3.6
#13 5.304 go: downloading golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf
#13 6.822 go: downloading github.com/go-playground/locales v0.14.0
#13 7.553 go: downloading golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f
#13 8.634 go: downloading google.golang.org/grpc v1.40.0
#13 8.638 go: downloading cloud.google.com/go v0.93.3
#13 8.694 go: downloading google.golang.org/appengine v1.6.7
#13 9.792 go: downloading github.com/googleapis/gax-go/v2 v2.1.0
#13 10.46 go: downloading go.opencensus.io v0.23.0
#13 11.09 go: downloading golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420
#13 11.26 go: downloading github.com/golang/protobuf v1.5.2
#13 11.67 go: downloading google.golang.org/protobuf v1.27.1
#13 12.08 go: downloading google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71
#13 12.08 go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
#13 15.90 error obtaining VCS status: exit status 128
#13 15.90   Use -buildvcs=false to disable VCS stamping.
------
executor failed running [/bin/sh -c go get -d ./...  && go install]: exit code: 1
frou commented 2 years ago

I actually don't use Docker personally, so I don't have much opinion on how to write the Dockerfile. If you want to make a decision on how to fix/modernise it then I'll trust that.

frou commented 2 years ago

Regarding https://github.com/frou/yt2pod/issues/42#issuecomment-1134778535 - Sorry about that, I will make that more robust. In addition, I think an alternative to being forced to use -buildvcs=false is that the .git directory could be added to the Docker image such that it is present at build time.

Besides that, I suspect that the approach taken by the Dockerfile has become outdated since this project transitioned from a traditional GOPATH-oriented project to a Go Module project.

The part where it creates the deeply nested directory structure (/go/src/github.com/frou/yt2pod/) that mimics the project's Import Path, and then runs go get -d ./... && go install looks suspicious. In a Go Module world, I think those parts should instead be replaced with approximately this:

RUN mkdir /build_dir

ADD .      /build_dir
ADD ./.git /build_dir/.git

WORKDIR /build_dir
RUN go build
# copy ./yt2pod executable to somewhere
frou commented 2 years ago

introspectOwnVersion function fixed in https://github.com/frou/yt2pod/commit/13702ab86b6acddfa96a52ef86ba75f2fa79395e

sabberworm commented 2 years ago
RUN mkdir /build_dir

ADD .      /build_dir
ADD ./.git /build_dir/.git

WORKDIR /build_dir
RUN go build
# copy ./yt2pod executable to somewhere

This seems to work very well, thanks.

frou commented 2 years ago

Cool - and the version shows as non-unknown in the log? If so, we can probably merge this?

sabberworm commented 2 years ago

Ha, I forgot to check that. Turns out git does need to be installed during build.

frou commented 2 years ago

Thanks for doing this.

BTW I'm always curious about how yt2pod users are handling the requirement to have a YouTube Data API key. Do you have your own key that you originally got from Google for another purpose? Or did you get one from Google to be able to use this? I know a bunch of people previously used a "shared" key that I placed in the example config file, but IIRC that got deactivated a while ago.

sabberworm commented 2 years ago

I did get a YouTube key directly from Google some time ago. So far, it’s been working.

frou commented 2 years ago

FYI, I tweaked this a bit after remembering why pip was being used in the first place. Commit message for 828d6bf4a9121b87339296c678e24f617db620af explains. Also used some standard paths from here.