stapelberg
commented
1 year ago Thanks for the report. This should be fixed with https://github.com/gokrazy/tools/commit/522f5b0a23b520a12a23b5eaf54fa892d7ae1081 now
Closed daenney closed 1 year ago
stapelberg
commented
1 year ago Thanks for the report. This should be fixed with https://github.com/gokrazy/tools/commit/522f5b0a23b520a12a23b5eaf54fa892d7ae1081 now
daenney
commented
1 year ago This definitely fixed the panic, thanks for that!
However, I'm running into a corrupt image now:
unsquashfs -l root.squashfs
squashfs-root
squashfs-root/dev
squashfs-root/etc
squashfs-root/etc/gokr-pw.txt
squashfs-root/etc/gokrazy
[..]
squashfs-root/user/fbstatus
squashfs-root/user/gotosocial
squashfs-root/user/mkfs
squashfs-root/user/node_exporter
squashfs-root/user/serial-busybox
squashfs-root/user/wifi
squashfs-root/usr
File system corrupted: directory has duplicate names or is unsorted
FATAL ERROR: dir_scan: failed to read directory squashfs-root/usr/shareI'm guessing I'm still doing something wrong with how I have ExtraFilePaths set up?
stapelberg
commented
1 year ago Might just be another edge case you’re hitting. Can you provide your full config please, and anything else I need to reproduce?
daenney
commented
1 year ago I tried to create as small a case as possible, by starting fresh from the base template:
$ gok new -i testtar
$ cd /home/daenney/gokrazy/testtarThat results in the following config.json:
{
"Hostname": "testtar",
"Update": {
"HTTPPassword": "3aeYDfmqTfRItieG4roN"
},
"Packages": [
"github.com/gokrazy/fbstatus",
"github.com/gokrazy/hello",
"github.com/gokrazy/serial-busybox",
"github.com/gokrazy/breakglass"
],
"PackageConfig": {
"github.com/gokrazy/breakglass": {
"CommandLineFlags": [
"-authorized_keys=/etc/breakglass.authorized_keys"
],
"ExtraFilePaths": {
"/etc/breakglass.authorized_keys": "/home/daenney/gokrazy/testtar/breakglass.authorized_keys"
}
},
"github.com/gokrazy/gokrazy/cmd/randomd": {
"ExtraFileContents": {
"/etc/machine-id": "b353b4b659034dacaa3cbaa484036dab\n"
}
}
},
"SerialConsole": "disabled"
}Tried to build that, and that works:
$ gok overwrite -i testtar --root root.squashfs
$ gokrazy gok g522f5b on GOARCH=amd64 GOOS=linux
Build target: CGO_ENABLED=0 GOARCH=arm64 GOOS=linux
Build timestamp: 2023-06-28T19:19:49+02:00
Loading system CA certificates from /etc/ssl/certs/ca-certificates.crt
Building 4 Go packages:
github.com/gokrazy/fbstatus
github.com/gokrazy/hello
github.com/gokrazy/serial-busybox
github.com/gokrazy/breakglass
will be started with command-line flags
from /home/daenney/gokrazy/testtar/config.json
last modified: 2023-06-28T19:19:07+02:00 (42s ago)
[building (go compiler)]go: finding module for package github.com/gokrazy/kernel
go: downloading github.com/gokrazy/kernel v0.0.0-20230627041833-fcef5bf84cf8
go: finding module for package github.com/gokrazy/firmware
go: finding module for package github.com/gokrazy/rpi-eeprom
go: finding module for package github.com/gokrazy/gokrazy/cmd/dhcp
go: finding module for package github.com/gokrazy/gokrazy/cmd/ntp
go: finding module for package github.com/gokrazy/gokrazy/cmd/randomd
go: finding module for package github.com/gokrazy/gokrazy/cmd/heartbeat
go: finding module for package github.com/gokrazy/fbstatus
go: finding module for package github.com/gokrazy/hello
go: finding module for package github.com/gokrazy/serial-busybox
go: finding module for package github.com/gokrazy/breakglass
go: finding module for package github.com/gokrazy/gokrazy
[done] in 34.79s
Including extra files for Go packages:
github.com/gokrazy/breakglass
will include extra files in the root file system
from /home/daenney/gokrazy/testtar/breakglass.authorized_keys
last modified: 2023-06-28T19:19:07+02:00 (1m18s ago)
go: finding module for package github.com/gokrazy/gokrazy
Including loadable kernel modules from:
/home/daenney/.cache/go/pkg/mod/github.com/gokrazy/kernel@v0.0.0-20230627041833-fcef5bf84cf8/lib/modules
Feature summary:
use GPT: true
use PARTUUID: true
use GPT PARTUUID: true
Creating root file system
[done] in 0.54s
Build complete!
To interact with the device, gokrazy provides a web interface reachable at:
http://gokrazy:3aeYDfmqTfRItieG4roN@testtar/
In addition, the following Linux consoles are set up:
1. foreground Linux framebuffer console on HDMIThe squashfs image is valid:
$ unsquashfs -l root.squashfs
squashfs-root
squashfs-root/dev
squashfs-root/etc
squashfs-root/etc/breakglass.authorized_keys
[..]
squashfs-root/varUpdated config.json to look like this:
{
"Hostname": "testtar",
"Update": {
"HTTPPassword": "3aeYDfmqTfRItieG4roN"
},
"Packages": [
"github.com/gokrazy/fbstatus",
"github.com/gokrazy/hello",
"github.com/gokrazy/serial-busybox",
"github.com/gokrazy/breakglass"
],
"PackageConfig": {
"github.com/gokrazy/breakglass": {
"CommandLineFlags": [
"-authorized_keys=/etc/breakglass.authorized_keys"
],
"ExtraFilePaths": {
"/etc/breakglass.authorized_keys": "/home/daenney/gokrazy/testtar/breakglass.authorized_keys",
"/usr/share/gotosocial": "assets/gotosocial_0.9.0_web-assets"
}
},
"github.com/gokrazy/gokrazy/cmd/randomd": {
"ExtraFileContents": {
"/etc/machine-id": "b353b4b659034dacaa3cbaa484036dab\n"
}
}
},
"SerialConsole": "disabled"
}Created the assets/ folder next to config.json, and within it I put https://github.com/superseriousbusiness/gotosocial/releases/download/v0.9.0/gotosocial_0.9.0_web-assets.tar.gz which I ran a gunzip on afterwards to remove the gzip compression.
$ tree
.
├── assets
│ └── gotosocial_0.9.0_web-assets.tar
├── breakglass.authorized_keys
├── builddir
│ ├── github.com
│ │ └── gokrazy
│ │ ├── breakglass
│ │ │ ├── go.mod
│ │ │ └── go.sum
│ │ ├── fbstatus
│ │ │ ├── go.mod
│ │ │ └── go.sum
│ │ ├── firmware
│ │ │ ├── go.mod
│ │ │ └── go.sum
│ │ ├── gokrazy
│ │ │ ├── cmd
│ │ │ │ ├── dhcp
│ │ │ │ │ ├── go.mod
│ │ │ │ │ └── go.sum
│ │ │ │ ├── heartbeat
│ │ │ │ │ ├── go.mod
│ │ │ │ │ └── go.sum
│ │ │ │ ├── ntp
│ │ │ │ │ ├── go.mod
│ │ │ │ │ └── go.sum
│ │ │ │ └── randomd
│ │ │ │ ├── go.mod
│ │ │ │ └── go.sum
│ │ │ ├── go.mod
│ │ │ └── go.sum
│ │ ├── hello
│ │ │ ├── go.mod
│ │ │ └── go.sum
│ │ ├── kernel
│ │ │ ├── go.mod
│ │ │ └── go.sum
│ │ ├── rpi-eeprom
│ │ │ ├── go.mod
│ │ │ └── go.sum
│ │ └── serial-busybox
│ │ ├── go.mod
│ │ └── go.sum
│ └── init
│ ├── go.mod
│ └── go.sum
├── config.json
└── root.squashfsBuilding still works:
$ gok overwrite -i testtar --root root.squashfs
gokrazy gok g522f5b on GOARCH=amd64 GOOS=linux
Build target: CGO_ENABLED=0 GOARCH=arm64 GOOS=linux
Build timestamp: 2023-06-28T19:21:45+02:00
Loading system CA certificates from /etc/ssl/certs/ca-certificates.crt
Building 4 Go packages:
github.com/gokrazy/fbstatus
github.com/gokrazy/hello
github.com/gokrazy/serial-busybox
github.com/gokrazy/breakglass
will be started with command-line flags
from /home/daenney/gokrazy/testtar/config.json
last modified: 2023-06-28T19:21:10+02:00 (35s ago)
[done] in 1.31s
Including extra files for Go packages:
github.com/gokrazy/breakglass
will include extra files in the root file system
from assets/gotosocial_0.9.0_web-assets.tar
last modified: 2023-05-24T16:54:54+02:00 (842h26m53s ago)
will include extra files in the root file system
from /home/daenney/gokrazy/testtar/breakglass.authorized_keys
last modified: 2023-06-28T19:19:07+02:00 (2m40s ago)
Including loadable kernel modules from:
/home/daenney/.cache/go/pkg/mod/github.com/gokrazy/kernel@v0.0.0-20230627041833-fcef5bf84cf8/lib/modules
Feature summary:
use GPT: true
use PARTUUID: true
use GPT PARTUUID: true
Creating root file system
[done] in 0.58s
Build complete!
To interact with the device, gokrazy provides a web interface reachable at:
http://gokrazy:3aeYDfmqTfRItieG4roN@testtar/
In addition, the following Linux consoles are set up:
1. foreground Linux framebuffer console on HDMIBut the root.squashfs image now has the error:
$ unsquashfs -l root.squashfs
squashfs-root
squashfs-root/dev
squashfs-root/etc
squashfs-root/etc/breakglass.authorized_keys
squashfs-root/etc/gokr-pw.txt
[..]
squashfs-root/user/serial-busybox
squashfs-root/usr
squashfs-root/usr/share
File system corrupted: directory has duplicate names or is unsorted
FATAL ERROR: dir_scan: failed to read directory squashfs-root/usr/share/gotosocialThanks, I can reproduce the issue with these instructions. Will try to find a fix soon.
stapelberg
commented
1 year ago Found the bug! Should be fixed with https://github.com/gokrazy/tools/commit/56839a456adf4b36657ee052b296adcb47ac0b56
daenney
commented
1 year ago Worked like a charm. Thanks so much for fixing these!
I'm trying to use the
ExtraFilePathsto get a bunch of things bundled in a.tar.gzadded to the base image. I removed the gzip compression withgzipfirst. Between the docs and the code I got the impression it's supposed to be used as follows:From https://github.com/gokrazy/tools/blob/cd4073ee8575d968d5684adedc3ff4d2f8710685/internal/packer/packer.go#L472-L491
I got the impression it seems I'm not supposed to append
.tarto the path? When I do this it seems to ignore the file entirely. In my case I'm using a path relative toconfig.json, so it looks like"/usr/share/directory": "assets/tarbal-without-the-extension".full config.json
``` { "Hostname": "gotosocial", "Update": { "HTTPPort": "8080", "HTTPSPort": "8443", "HTTPPassword": "redacted" }, "Packages": [ "github.com/gokrazy/fbstatus", "github.com/gokrazy/mkfs", "github.com/gokrazy/serial-busybox", "github.com/gokrazy/wifi", "github.com/superseriousbusiness/gotosocial/cmd/gotosocial" ], "PackageConfig": { "github.com/superseriousbusiness/gotosocial/cmd/gotosocial":{ "WaitForClock": true, "GoBuildTags": [ "netgo", "osusergo", "static_build", "kvformat", "notracing" ], "GoBuildFlags": [ "-trimpath", "-ldflags=-s -w -extldflags '-static' -X 'main.Version=0.9.0'" ], "CommandLineFlags": [ "server", "start", "--config=/perm/gotosocial/config.yaml" ], "ExtraFilePaths": { "/usr/share/gotosocial": "assets/gotosocial_0.9.0_web-assets" } } }, "SerialConsole": "disabled", "InternalCompatibilityFlags": {} } ```But this triggers a panic:
panic: runtime error: invalid memory address or nil pointer dereference
``` gokrazy gok gcd4073 on GOARCH=amd64 GOOS=linux Build target: CGO_ENABLED=0 GOARCH=arm64 GOOS=linux Build timestamp: 2023-06-23T18:18:15+02:00 Loading system CA certificates from /etc/ssl/certs/ca-certificates.crt Building 5 Go packages: github.com/gokrazy/fbstatus github.com/gokrazy/mkfs github.com/gokrazy/serial-busybox github.com/gokrazy/wifi github.com/superseriousbusiness/gotosocial/cmd/gotosocial will be compiled with build flags from /home/daenney/Development/github.com/gotosocial-deployments/raspberrypi/config.json last modified: 2023-06-23T18:18:13+02:00 (2s ago) will be compiled with build tags from /home/daenney/Development/github.com/gotosocial-deployments/raspberrypi/config.json last modified: 2023-06-23T18:18:13+02:00 (2s ago) will be started with command-line flags from /home/daenney/Development/github.com/gotosocial-deployments/raspberrypi/config.json last modified: 2023-06-23T18:18:13+02:00 (2s ago) will wait for clock synchronization before start from /home/daenney/Development/github.com/gotosocial-deployments/raspberrypi/config.json last modified: 2023-06-23T18:18:13+02:00 (2s ago) [done] in 3.10s panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x7fafab] goroutine 1 [running]: github.com/gokrazy/tools/internal/packer.(*archiveExtraction).extractArchive(0xc00017e6d8, {0xc000486210?, 0xa01c98?}) /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/internal/packer/packer.go:449 +0x42b github.com/gokrazy/tools/internal/packer.findExtraFilesInDir({0xc00014e280, 0x39}, {0xc000148330, 0x22}, 0xc000491320) /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/internal/packer/packer.go:487 +0x1c7 github.com/gokrazy/tools/internal/packer.FindExtraFiles(0xc00019c000) /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/internal/packer/packer.go:554 +0xa7e github.com/gokrazy/tools/internal/packer.(*Pack).logic(0xc00017fbf0, {0x8e1a60, 0xb}) /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/internal/packer/packer.go:1130 +0x129f github.com/gokrazy/tools/internal/packer.(*Pack).Main(0xc00013e0f0?, {0x8e1a60?, 0x0?}) /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/internal/packer/packer.go:1875 +0x1e github.com/gokrazy/tools/internal/gok.(*overwriteImplConfig).run(0xcb0aa0, {0x0?, 0x0?}, {0x0?, 0x0?, 0x0?}, {0x0?, 0x0?}, {0xa05fa0, 0xc000120010}) /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/internal/gok/overwrite.go:128 +0x3ed github.com/gokrazy/tools/internal/gok.glob..func6(0xca9c20, {0xc000122900, 0x0, 0x6}) /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/internal/gok/overwrite.go:39 +0x14e github.com/spf13/cobra.(*Command).execute(0xca9c20, {0xc000124500, 0x6, 0x8}) /home/daenney/.cache/go/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:916 +0x862 github.com/spf13/cobra.(*Command).ExecuteC(0xcaa1e0) /home/daenney/.cache/go/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:1044 +0x3bd github.com/spf13/cobra.(*Command).Execute(0xc0000061a0?) /home/daenney/.cache/go/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:968 +0x19 github.com/gokrazy/tools/gok.Context.Execute({{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0, 0x0}}, {0xa0a2c8, ...}) /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/gok/gok.go:34 +0xe5 main.main() /home/daenney/.cache/go/pkg/mod/github.com/gokrazy/tools@v0.0.0-20230601205805-cd4073ee8575/cmd/gok/gok.go:15 +0x49 ```This is what the tar-archive contains:
tar -tvf assets/gotosocial_0.9.0_web-assets.tar
``` -rw-r--r-- root/root 34520 2023-05-24 16:53 LICENSE -rw-r--r-- root/root 43405 2023-05-24 16:53 web/assets/Fork-Awesome/CONTRIBUTORS.md -rw-r--r-- root/root 25608 2023-05-24 16:53 web/assets/Fork-Awesome/LICENSES -rw-r--r-- root/root 44305 2023-05-24 16:53 web/assets/Fork-Awesome/css/fork-awesome.css -rw-r--r-- root/root 36910 2023-05-24 16:53 web/assets/Fork-Awesome/css/fork-awesome.min.css -rw-r--r-- root/root 15269 2023-05-24 16:53 web/assets/Fork-Awesome/css/fork-awesome.min.css.map -rw-r--r-- root/root 8537 2023-05-24 16:53 web/assets/Fork-Awesome/css/v5-compat.css -rw-r--r-- root/root 7422 2023-05-24 16:53 web/assets/Fork-Awesome/css/v5-compat.min.css -rw-r--r-- root/root 2616 2023-05-24 16:53 web/assets/Fork-Awesome/css/v5-compat.min.css.map -rw-r--r-- root/root 219194 2023-05-24 16:53 web/assets/Fork-Awesome/fonts/forkawesome-webfont.eot -rw-r--r-- root/root 560551 2023-05-24 16:53 web/assets/Fork-Awesome/fonts/forkawesome-webfont.svg -rw-r--r-- root/root 219004 2023-05-24 16:53 web/assets/Fork-Awesome/fonts/forkawesome-webfont.ttf -rw-r--r-- root/root 136120 2023-05-24 16:53 web/assets/Fork-Awesome/fonts/forkawesome-webfont.woff -rw-r--r-- root/root 109916 2023-05-24 16:53 web/assets/Fork-Awesome/fonts/forkawesome-webfont.woff2 -rw-r--r-- root/root 495 2023-05-24 16:53 web/assets/LICENSE -rw-r--r-- root/root 8348 2023-05-24 16:53 web/assets/default_avatars/GoToSocial_icon1.png -rw-r--r-- root/root 8815 2023-05-24 16:53 web/assets/default_avatars/GoToSocial_icon2.png -rw-r--r-- root/root 16525 2023-05-24 16:53 web/assets/default_avatars/GoToSocial_icon3.png -rw-r--r-- root/root 17204 2023-05-24 16:53 web/assets/default_avatars/GoToSocial_icon4.png -rw-r--r-- root/root 16717 2023-05-24 16:53 web/assets/default_avatars/GoToSocial_icon5.png -rw-r--r-- root/root 14846 2023-05-24 16:53 web/assets/default_avatars/GoToSocial_icon6.png -rw-r--r-- root/root 67 2023-05-24 16:53 web/assets/default_header.png -rw-r--r-- root/root 4444 2023-05-24 16:54 web/assets/dist/_colors.css -rw-r--r-- root/root 14685 2023-05-24 16:54 web/assets/dist/base.css -rw-r--r-- root/root 210344 2023-05-24 16:54 web/assets/dist/frontend.js -rw-r--r-- root/root 1027 2023-05-24 16:54 web/assets/dist/index.css -rw-r--r-- root/root 6251 2023-05-24 16:54 web/assets/dist/profile.css -rw-r--r-- root/root 21000 2023-05-24 16:54 web/assets/dist/settings-style.css -rw-r--r-- root/root 1153338 2023-05-24 16:54 web/assets/dist/settings.js -rw-r--r-- root/root 52456 2023-05-24 16:54 web/assets/dist/status.css -rw-r--r-- root/root 4350 2023-05-24 16:53 web/assets/fonts/OFL.txt -rw-r--r-- root/root 16312 2023-05-24 16:53 web/assets/fonts/noto-sans-v27-latin-700.woff -rw-r--r-- root/root 12684 2023-05-24 16:53 web/assets/fonts/noto-sans-v27-latin-700.woff2 -rw-r--r-- root/root 16592 2023-05-24 16:53 web/assets/fonts/noto-sans-v27-latin-regular.woff -rw-r--r-- root/root 12860 2023-05-24 16:53 web/assets/fonts/noto-sans-v27-latin-regular.woff2 -rw-r--r-- root/root 27204 2023-05-24 16:53 web/assets/logo.png -rw-r--r-- root/root 22267 2023-05-24 16:53 web/assets/logo.svg -rw-r--r-- root/root 1499 2023-05-24 16:53 web/assets/mastodon.svg -rw-r--r-- root/root 5785 2023-05-24 16:53 web/assets/plyr.svg -rw-r--r-- root/root 153798 2023-05-24 16:54 web/assets/swagger.yaml -rw-r--r-- root/root 16784 2023-05-24 16:53 web/assets/tusky.svg -rw-r--r-- root/root 1390 2023-05-24 16:53 web/template/404.tmpl -rw-r--r-- root/root 3022 2023-05-24 16:53 web/template/about.tmpl -rw-r--r-- root/root 1550 2023-05-24 16:53 web/template/authorize.tmpl -rw-r--r-- root/root 1022 2023-05-24 16:53 web/template/confirmed.tmpl -rw-r--r-- root/root 1742 2023-05-24 16:53 web/template/domain-blocklist.tmpl -rw-r--r-- root/root 1186 2023-05-24 16:53 web/template/email_confirm.tmpl -rw-r--r-- root/root 1247 2023-05-24 16:53 web/template/email_new_report.tmpl -rw-r--r-- root/root 1262 2023-05-24 16:53 web/template/email_report_closed.tmpl -rw-r--r-- root/root 1154 2023-05-24 16:53 web/template/email_reset.tmpl -rw-r--r-- root/root 1004 2023-05-24 16:53 web/template/email_test.tmpl -rw-r--r-- root/root 1069 2023-05-24 16:53 web/template/error.tmpl -rw-r--r-- root/root 2277 2023-05-24 16:53 web/template/finalize.tmpl -rw-r--r-- root/root 1552 2023-05-24 16:53 web/template/footer.tmpl -rw-r--r-- root/root 919 2023-05-24 16:53 web/template/frontend.tmpl -rw-r--r-- root/root 3620 2023-05-24 16:53 web/template/header.tmpl -rw-r--r-- root/root 3190 2023-05-24 16:53 web/template/index.tmpl -rw-r--r-- root/root 1063 2023-05-24 16:53 web/template/oob.tmpl -rw-r--r-- root/root 4516 2023-05-24 16:53 web/template/profile.tmpl -rw-r--r-- root/root 1556 2023-05-24 16:53 web/template/sign-in.tmpl -rw-r--r-- root/root 4465 2023-05-24 16:53 web/template/status.tmpl -rw-r--r-- root/root 1287 2023-05-24 16:53 web/template/thread.tmpl ```Based on the panic I'm sort of assuming I've misunderstood the documentation on how to use
ExtraFilePathsbut I'm struggling to find a repository that uses gokrazy andExtraFilePathswith a tar bundle to see where I'm going wrong.