search

googleforgames / agones

Dedicated Game Server Hosting and Scaling for Multiplayer Games on Kubernetes
https://agones.dev
Apache License 2.0
5.98k stars 789 forks source link

Issue running `make site-server` from build directory #3885

Closed aallbrig closed 1 month ago

aallbrig commented 2 months ago

What happened: I tried setting up a local instance of the agones site locally by following instructions found on agones website to make site-server from the build directory. I see that it's ultimately running a hugo server that will bind to port 1313 but from inside a container.

I encountered some issues with set up. I tried resolving the issue myself (read more in "what I tried" section) but it was late at night.

Below is log output central to the issue. To highlight a couple things before the output wall:

  1. docker run --privileged --rm tonistiigi/binfmt:qemu-v6.2.0 --install arm64 followed up by installing: arm64 cannot register "/usr/bin/qemu-aarch64" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: no such file or directory
  2. ERROR Failed to read Git log: fatal: detected dubious ownership in repository at '/go/src/agones.dev/agones'
λ ~/src/agones/build/ make site-server
mkdir -p /Users/$USER/.kube/
mkdir -p /Users/$USER/src/agones/build//.gocache
mkdir -p /Users/$USER/src/agones/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
docker run --privileged --rm tonistiigi/binfmt:qemu-v6.2.0 --install arm64
installing: arm64 cannot register "/usr/bin/qemu-aarch64" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: no such file or directory
{
  "supported": [
    "linux/arm64",
    "linux/amd64",
    "linux/riscv64",
    "linux/ppc64le",
    "linux/s390x",
    "linux/386",
    "linux/mips64le",
    "linux/mips64",
    "linux/arm/v7",
    "linux/arm/v6"
  ],
  "emulators": [
    "arm",
    "i386",
    "mips64",
    "mips64le",
    "ppc64le",
    "riscv64",
    "rosetta",
    "rosetta-wrapper",
    "s390x",
    "x86_64"
  ]
}
/Applications/Xcode.app/Contents/Developer/usr/bin/make ensure-image IMAGE_TAG=agones-build:5bab735929 BUILD_TARGET=build-build-image
docker run --user 501:20 --rm -v /Users/$USER/src/agones/build//.config/gcloud:/root/.config/gcloud -v /Users/$USER/.kube/:/root/.kube -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm -v /Users/$USER/src/agones:/go/src/agones.dev/agones -v /Users/$USER/src/agones/build//.gomod:/go/pkg/mod -v /Users/$USER/src/agones/build//.gocache:/root/.cache/go-build --workdir=/go/src/agones.dev/agones/site --platform linux/x86_64 -p 1313:1313 agones-build:5bab735929 bash -c \
        "RELEASE_VERSION="1.42.0" RELEASE_BRANCH=main hugo server --watch --baseURL=http://localhost:1313/ --bind=0.0.0.0 -F"
hugo: downloading modules …
hugo: collected modules in 27202 ms
Watching for changes in /go/src/agones.dev/agones/site/{assets,content,layouts,static}
Watching for config changes in /go/src/agones.dev/agones/site/config.toml, /go/src/agones.dev/agones/site/go.mod
Start building sites …
hugo v0.118.2-da7983ac4b94d97d776d7c2405040de97e95c03d+extended linux/amd64 BuildDate=2023-08-31T11:23:51Z VendorInfo=gohugoio

ERROR Failed to read Git log: fatal: detected dubious ownership in repository at '/go/src/agones.dev/agones'
To add an exception for this directory, call:

        git config --global --add safe.directory /go/src/agones.dev/agones
WARN  Shortcode "tabpane": duplicate language "yaml" detected, disabling persistance of language to avoid multiple tab display. Position: "/go/src/agones.dev/agones/site/content/en/docs/Guides/fleet-updates.md:95:1"
WARN  Shortcode "tabpane": duplicate language "yaml" detected, disabling persistance of language to avoid multiple tab display. Position: "/go/src/agones.dev/agones/site/content/en/docs/Reference/gameserverallocation.md:19:1"
Built in 4059 ms
Error: error building site: logged 1 error(s)
make: *** [site-server] Error 1

What you expected to happen: I am able to navigate to http://localhost:1313 and see a development version of the agones site to validate website changes.

How to reproduce it (as minimally and precisely as possible):

cd build
make site-server

It seems likely this might be an arm issue, so running arm hardware may also be a prerequisite?

Anything else we need to know?:

Some relevant ARM parts of agones codebase are found here:

I was able to move passed this issue using the following steps.

I set WITH_ARM64=0 to (air quotes) "resolve" the arm64 cannot register "/usr/bin/qemu-aarch64" to /proc/sys/fs/binfmt_misc/register issue.

I see git config --global --add safe.directory /go/src/agones.dev/agones in output so I tried prepending $(git_safe) && to build/includes/website.mk#L34 because that makefile variable exactly to that command. I get a Permission denied error.

λ ~/src/agones/build/ add-gamenetcode-resource-to-agones-documentation* make site-server
mkdir -p /Users/$USER/.kube/
mkdir -p /Users/$USER/src/agones/build//.gocache
mkdir -p /Users/$USER/src/agones/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
/Applications/Xcode.app/Contents/Developer/usr/bin/make ensure-image IMAGE_TAG=agones-build:5bab735929 BUILD_TARGET=build-build-image
docker run --user 501:20 --rm -v /Users/$USER/src/agones/build//.config/gcloud:/root/.config/gcloud -v /Users/$USER/.kube/:/root/.kube -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm -v /Users/$USER/src/agones:/go/src/agones.dev/agones -v /Users/$USER/src/agones/build//.gomod:/go/pkg/mod -v /Users/$USER/src/agones/build//.gocache:/root/.cache/go-build --workdir=/go/src/agones.dev/agones/site --platform linux/x86_64 -p 1313:1313 agones-build:5bab735929 bash -c \
    "git config --global --add safe.directory /go/src/agones.dev/agones && RELEASE_VERSION="1.42.0" RELEASE_BRANCH=main hugo server --watch --baseURL=http://localhost:1313/ --bind=0.0.0.0 -F"
error: could not lock config file //.gitconfig: Permission denied
make: *** [site-server] Error 255

Permission issue, eh? Next thing is to remove --user $(UID):$(GID) from the docker run command. This moves me passed the issue and I'm able to see the development website.

Environment:

igooch commented 1 month ago

It might be because the container is running as root, and the git repo is running as your local user. Does this work if you run the command sudo chown -R $(whoami):$(id -g -n $(whoami)) .? Also, did just removing the --user $(UID):$(GID) work for you, or did you need to remove --user $(UID):$(GID) and add $(git_safe)?

Orza commented 1 month ago

It might be because the container is running as root, and the git repo is running as your local user. Does this work if you run the command sudo chown -R $(whoami):$(id -g -n $(whoami)) .? Also, did just removing the --user $(UID):$(GID) work for you, or did you need to remove --user $(UID):$(GID) and add $(git_safe)?

Same issue, I run the repo as a local user and make the site with sudo make site-server. You need to add $(git_safe) and remove --user $(UID):$(GID) The whole solution worked for me:

site-server: ensure-build-image
        docker run --rm $(common_mounts) --workdir=$(mount_path)/site $(DOCKER_RUN_ARGS) -p 1313:1313 $(build_tag) bash -c \
        "$(git_safe) && $(ENV) hugo server --watch --baseURL=http://localhost:1313/ --bind=0.0.0.0 $(ARGS)"
aallbrig commented 1 month ago

I hacked in a workaround. I'll post it here soon. Actually it's the exact same as what @Orza posted (see screenshot). Great minds etc etc :) Link to build/includes/website.mk#L32

cc @igooch @markmandel Are you two having issues when running cd build && make site-server?

Screenshot 2024-07-18 at 1 21 22 PM
igooch commented 1 month ago

cd build && make site-server works perfectly fine for me as-is. That being said I don't see any issues with adding this change. @markmandel is there any reason to keep things as-is?

aallbrig commented 1 month ago

Oh cool I wonder what the issue is with @Orza and my dev machines. What dev machine system are you working on? AMD or ARM?

I'll try it as is on my intel based MBP later. I'm on my ARM MBP, so I wonder if there's just a difference there...

Getting ahead, I wonder if we'll want to modify the generated docker command based on what type of dev machine is running it.

Orza commented 1 month ago

Oh cool I wonder what the issue is with @Orza and my dev machines. What dev machine system are you working on? AMD or ARM?

I'm on x86-64, but the issue is not related to ARM. It is specifically a permissions issue because I'm running Docker with sudo

markmandel commented 1 month ago

if you are on ARM (like an M2) you'll likely run into issues -- I expect a bunch of stuff will break since ARM was not commonplace when this was written.

@Orza I never thought of what would happen if people used Docker under sudo... there's an assumption there that you can run docker as the same user $(whoami).

I do wonder if the --user $(UID):$(GID) only will work for linux machines. We might want to make that OS specific.

In theory the git config --global --add safe.directory /go/src/agones.dev/agones workaround should work - but maybe not with the uid/guid id in combination?

Yay, supporting different OS's with different architectures and commands 🤦🏻

aallbrig commented 1 month ago

Yay, supporting different OS's with different architectures and commands 🤦🏻

It's fun & an honor to be a part of the bleeding edge.

I have an M2 MBP, intel MBP, and Ubuntu thinkpad (amd64). I'll try on all three devices; since this issue represents my M2 MBP I just need to run on other 2 devices. Will report back soon.

aallbrig commented 1 month ago

Just confirmed on my x86_64 ubuntu machine that cd build && make site-server works as is (tested on commit 03a0a4c34999c840b7db38bbcb1dac93aed7d19d). For fun here's the output of lscpu.

$ lscpu
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Byte Order:                           Little Endian
Address sizes:                        39 bits physical, 48 bits virtual
CPU(s):                               8
On-line CPU(s) list:                  0-7
Thread(s) per core:                   2
Core(s) per socket:                   4
Socket(s):                            1
NUMA node(s):                         1
Vendor ID:                            GenuineIntel
CPU family:                           6
Model:                                142
Model name:                           Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
Stepping:                             12
CPU MHz:                              2300.000
CPU max MHz:                          4900.0000
CPU min MHz:                          400.0000
BogoMIPS:                             4599.93
Virtualization:                       VT-x
L1d cache:                            128 KiB
L1i cache:                            128 KiB
L2 cache:                             1 MiB
L3 cache:                             8 MiB
NUMA node0 CPU(s):                    0-7
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d arch_capabilities
aallbrig commented 1 month ago

Tested on my x86_64 MBP and got the "...detected dubious ownership..." issue.

$ make site-server 
mkdir -p /Users/$USER/.kube/
mkdir -p /Users/$USER/src/agones/build//.gocache
mkdir -p /Users/$USER/src/agones/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
docker run --privileged --rm tonistiigi/binfmt:qemu-v6.2.0 --install arm64
installing: arm64 qemu-aarch64 already registered
{
  "supported": [
    "linux/amd64",
    "linux/arm64",
    "linux/riscv64",
    "linux/ppc64le",
    "linux/s390x",
    "linux/386",
    "linux/mips64le",
    "linux/mips64",
    "linux/arm/v7",
    "linux/arm/v6"
  ],
  "emulators": [
    "aarch64",
    "arm",
    "mips64",
    "mips64le",
    "ppc64le",
    "qemu-aarch64",
    "riscv64",
    "s390x"
  ]
}
/Library/Developer/CommandLineTools/usr/bin/make ensure-image IMAGE_TAG=agones-build:a69193b6bf BUILD_TARGET=build-build-image
docker run --user 501:20 --rm -v /Users/$USER/src/agones/build//.config/gcloud:/root/.config/gcloud -v /Users/$USER/.kube/:/root/.kube -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm -v /Users/$USER/src/agones:/go/src/agones.dev/agones -v /Users/$USER/src/agones/build//.gomod:/go/pkg/mod -v /Users/$USER/src/agones/build//.gocache:/root/.cache/go-build --workdir=/go/src/agones.dev/agones/site  -p 1313:1313 agones-build:a69193b6bf bash -c \
        "RELEASE_VERSION="1.43.0" RELEASE_BRANCH=main hugo server --watch --baseURL=http://localhost:1313/ --bind=0.0.0.0 -F"
hugo: downloading modules …
hugo: collected modules in 17610 ms
Watching for changes in /go/src/agones.dev/agones/site/{assets,content,layouts,static}
Watching for config changes in /go/src/agones.dev/agones/site/config.toml, /go/src/agones.dev/agones/site/go.mod
Start building sites …
hugo v0.118.2-da7983ac4b94d97d776d7c2405040de97e95c03d+extended linux/amd64 BuildDate=2023-08-31T11:23:51Z VendorInfo=gohugoio

ERROR Failed to read Git log: fatal: detected dubious ownership in repository at '/go/src/agones.dev/agones'
To add an exception for this directory, call:

        git config --global --add safe.directory /go/src/agones.dev/agones
WARN  Shortcode "tabpane": duplicate tab-persistence key "yaml" detected, disabling persistance to avoid multiple tab display. Position: "/go/src/agones.dev/agones/site/content/en/docs/Guides/fleet-updates.md:95:1"
WARN  Shortcode "tabpane": duplicate tab-persistence key "yaml" detected, disabling persistance to avoid multiple tab display. Position: "/go/src/agones.dev/agones/site/content/en/docs/Reference/gameserverallocation.md:19:1"
Built in 1734 ms
Error: error building site: logged 1 error(s)
make: *** [site-server] Error 1

Just tried the sudo chown -R $(whoami):$(id -g -n $(whoami)) . and re-ran make site-server -- did not work (same output as above).

Just tried only removing --user $(UID):$(GID) from build/includes/website.mkL33 and it works!!

Screenshot 2024-07-19 at 7 40 55 AM Screenshot 2024-07-19 at 7 47 38 AM
aallbrig commented 1 month ago

I tried removing just --user $(UID):$(GID) line on x86_64 ubuntu and ran into the "...detected dubious ownership..."

mkdir -p /home/$USER/.kube/
mkdir -p /home/$USER/src/agones/build//.gocache
mkdir -p /home/$USER/src/agones/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
docker run --privileged --rm tonistiigi/binfmt:qemu-v6.2.0 --install arm64
installing: arm64 qemu-aarch64 already registered
{
  "supported": [
    "linux/amd64",
    "linux/arm64",
    "linux/386"
  ],
  "emulators": [
    "qemu-aarch64"
  ]
}
make ensure-image IMAGE_TAG=agones-build:a69193b6bf BUILD_TARGET=build-build-image
make[1]: Entering directory '/home/$USER/src/agones/build'
make[1]: Leaving directory '/home/$USER/src/agones/build'
docker run --rm -v /home/$USER/src/agones/build//.config/gcloud:/root/.config/gcloud -v /home/$USER/.kube/:/root/.kube -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm -v /home/$USER/src/agones:/go/src/agones.dev/agones -v /home/$USER/src/agones/build//.gomod:/go/pkg/mod -v /home/$USER/src/agones/build//.gocache:/root/.cache/go-build --workdir=/go/src/agones.dev/agones/site  -p 1313:1313 agones-build:a69193b6bf bash -c \
"RELEASE_VERSION="1.43.0" RELEASE_BRANCH=main hugo server --watch --baseURL=http://localhost:1313/ --bind=0.0.0.0 -F"
hugo: downloading modules …
hugo: collected modules in 18357 ms
Watching for changes in /go/src/agones.dev/agones/site/{assets,content,layouts,static}
Watching for config changes in /go/src/agones.dev/agones/site/config.toml, /go/src/agones.dev/agones/site/go.mod
Start building sites … 
hugo v0.118.2-da7983ac4b94d97d776d7c2405040de97e95c03d+extended linux/amd64 BuildDate=2023-08-31T11:23:51Z VendorInfo=gohugoio

ERROR Failed to read Git log: fatal: detected dubious ownership in repository at '/go/src/agones.dev/agones'
To add an exception for this directory, call:

        git config --global --add safe.directory /go/src/agones.dev/agones
WARN  Shortcode "tabpane": duplicate tab-persistence key "yaml" detected, disabling persistance to avoid multiple tab display. Position: "/go/src/agones.dev/agones/site/content/en/docs/Guides/fleet-updates.md:95:1"
WARN  Shortcode "tabpane": duplicate tab-persistence key "yaml" detected, disabling persistance to avoid multiple tab display. Position: "/go/src/agones.dev/agones/site/content/en/docs/Reference/gameserverallocation.md:19:1"
Built in 3570 ms
Error: error building site: logged 1 error(s)
make: *** [includes/website.mk:33: site-server] Error 1

I both removed --user $(UID):$(GID) and added $(git_safe) && and see that it works!

Screenshot from 2024-07-19 11-28-32

Screenshot from 2024-07-19 11-47-15

Looks like making both these changes would help.

The PR that introduces --user $(UID):$(GID) is this one: https://github.com/googleforgames/agones/pull/3335

aallbrig commented 1 month ago

Created PR with the fix described in above comments here: https://github.com/googleforgames/agones/pull/3914

aallbrig commented 1 month ago

@igooch what do you think?