Panic when initializing tag database

[DangerOnTheRanger]

I've been using captchouli for a while with the following Dockerfile, and it's worked great:

FROM debian:buster-slim
# install prereqs
RUN apt-get update
RUN apt-get install -y golang libopencv-dev
RUN apt-get install -y git
# grab utils for bootstrapping watcher
RUN apt-get install -y netcat-openbsd psmisc
# make a new user so we don't run as root
RUN useradd -r captchouli
WORKDIR /home/captchouli
RUN chown -R captchouli /home/captchouli
# preemptively make the supporting directory so docker won't
# attempt to make a new directory owned by root (aids when
# attempting to persist .captchouli)
RUN mkdir -p .captchouli/images
RUN chown -R captchouli .captchouli
# install captchouli
USER captchouli
RUN go get github.com/bakape/captchouli/cmd/captchouli
# setup entrypoint and port
COPY entrypoint.sh .
COPY captchouli.sh .
EXPOSE 8512
# entrypoint switches back to the captchouli user but needs to
# start with root privileges
USER root
ENTRYPOINT ["sh", "./entrypoint.sh"]

(There's also captchouli.sh and entrypoint.sh, but they're small shell scripts and don't really do much beyond handle startup.)

Lately, I tried to rebuild my captchouli container and got the following message when initializing the tag database (with me exiting the container on the last two lines, and the first line stemming from captchouli.sh):

Waiting for captchouli to finish bootstrapping...
2020/05/05 12:35:04 captchouli: upgrading database to version 1
2020/05/05 12:35:04 captchouli: upgrading database to version 2
2020/05/05 12:35:04 captchouli: upgrading database to version 3
2020/05/05 12:35:04 captchouli: upgrading database to version 4
2020/05/05 12:35:04 captchouli: initializing tag=izayoi_sakuya explicitness=[safe, questionable, explicit]

fetch attempt: 1    panic: captchouli: error initializing image pool for tag `izayoi_sakuya`: &{%!w(string=number) %!w(*reflect.rtype=&{1 0 335480517 7 1 1 129 0xbe4ec0 0x93b6c1 9448 169216}) %!w(int64=306) %!w(string=gelbooruDecoder) %!w(string=Sample)}

goroutine 1 [running]:
main.main()
    /home/captchouli/go/src/github.com/bakape/captchouli/cmd/captchouli/main.go:52 +0x307
^C
Session terminated, killing shell... ...killed.

Now, I realized I had the prerequisites for captchouli still directly installed on my Debian Sid machine, and so I re-installed captchouli on my laptop without any containers to see what would happen, and got the following message:

2020/05/05 07:33:59 captchouli: upgrading database to version 1
2020/05/05 07:33:59 captchouli: upgrading database to version 2
2020/05/05 07:33:59 captchouli: upgrading database to version 3
2020/05/05 07:33:59 captchouli: upgrading database to version 4
2020/05/05 07:33:59 captchouli: initializing tag=senjougahara_hitagi explicitness=[safe]

fetch attempt: 1    panic: captchouli: error initializing image pool for tag `senjougahara_hitagi`: json: cannot unmarshal number into Go struct field gelbooruDecoder.Sample of type bool

goroutine 1 [running]:
main.main()
    /home/kermit/projects/go/src/github.com/bakape/captchouli/cmd/captchouli/main.go:52 +0x3ee

Any idea as to what the problem might be?

[bakape]

EIther Gelbooru changed the API or some schema-less bullsit in their DB. Does this always happen on start? If yes, they likely changed the API.

On Tue, 5 May 2020 at 15:49, Kermit Alexander II notifications@github.com wrote:

I've been using captchouli for a while with the following Dockerfile, and it's worked great:

FROM debian:buster-slim

install prereqs

RUN apt-get update RUN apt-get install -y golang libopencv-dev RUN apt-get install -y git

grab utils for bootstrapping watcher

RUN apt-get install -y netcat-openbsd psmisc

make a new user so we don't run as root

RUN useradd -r captchouli WORKDIR /home/captchouli RUN chown -R captchouli /home/captchouli

preemptively make the supporting directory so docker won't

attempt to make a new directory owned by root (aids when

attempting to persist .captchouli)

RUN mkdir -p .captchouli/images RUN chown -R captchouli .captchouli

install captchouli

USER captchouli RUN go get github.com/bakape/captchouli/cmd/captchouli

setup entrypoint and port

COPY entrypoint.sh . COPY captchouli.sh . EXPOSE 8512

entrypoint switches back to the captchouli user but needs to

start with root privileges

USER root ENTRYPOINT ["sh", "./entrypoint.sh"]

(There's also captchouli.sh https://github.com/DangerOnTheRanger/maniwani/blob/ecdef4b198ada5363216b8cb847d57781f1c77ca/build-helpers/captchouli/captchouli.sh and entrypoint.sh https://github.com/DangerOnTheRanger/maniwani/blob/ecdef4b198ada5363216b8cb847d57781f1c77ca/build-helpers/captchouli/entrypoint.sh, but they're small shell scripts and don't really do much beyond handle startup.)

Lately, I tried to rebuild my captchouli container and got the following message when initializing the tag database (with me exiting the container on the last two lines, and the first line stemming from captchouli.sh):

Waiting for captchouli to finish bootstrapping... 2020/05/05 12:35:04 captchouli: upgrading database to version 1 2020/05/05 12:35:04 captchouli: upgrading database to version 2 2020/05/05 12:35:04 captchouli: upgrading database to version 3 2020/05/05 12:35:04 captchouli: upgrading database to version 4 2020/05/05 12:35:04 captchouli: initializing tag=izayoi_sakuya explicitness=[safe, questionable, explicit]

fetch attempt: 1 panic: captchouli: error initializing image pool for tag izayoi_sakuya: &{%!w(string=number) %!w(*reflect.rtype=&{1 0 335480517 7 1 1 129 0xbe4ec0 0x93b6c1 9448 169216}) %!w(int64=306) %!w(string=gelbooruDecoder) %!w(string=Sample)}

goroutine 1 [running]: main.main() /home/captchouli/go/src/github.com/bakape/captchouli/cmd/captchouli/main.go:52 +0x307 ^C Session terminated, killing shell... ...killed.

Now, I realized I had the prerequisites for captchouli still directly installed on my Debian Sid machine, and so I re-installed captchouli on my laptop without any containers to see what would happen, and got the following message:

2020/05/05 07:33:59 captchouli: upgrading database to version 1 2020/05/05 07:33:59 captchouli: upgrading database to version 2 2020/05/05 07:33:59 captchouli: upgrading database to version 3 2020/05/05 07:33:59 captchouli: upgrading database to version 4 2020/05/05 07:33:59 captchouli: initializing tag=senjougahara_hitagi explicitness=[safe]

fetch attempt: 1 panic: captchouli: error initializing image pool for tag senjougahara_hitagi: json: cannot unmarshal number into Go struct field gelbooruDecoder.Sample of type bool

goroutine 1 [running]: main.main() /home/kermit/projects/go/src/github.com/bakape/captchouli/cmd/captchouli/main.go:52 +0x3ee

Any idea as to what the problem might be?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bakape/captchouli/issues/37, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB347MDWY3EIZ6UPBQYLTBLRQADOVANCNFSM4MZRNLCQ .

[DangerOnTheRanger]

Yes, it always happens on start - as far as I can tell no images are ever written, so it always tries to load new images for each tag, and then the error occurs.

@ECHibiki I noticed you just committed to your fork of captchouli a couple days ago and seem to be looking for particular error messages inside the gelbooru-specific parts - is this the same issue you ran into?

[ECHibiki]

I believe I was getting this panic notification as a result of my own modifications. Changes I made were done months ago though so I don't recall. Whatever fixes I did might be hacky, pointless or irrelevant since I'm 2 commits behind.

fetch go
56,6 +58,10 @@ func fetch(req common.FetchRequest) (err error) {
    }
    f, img, err := fn(req)
    if f == nil || err != nil {
        log.Printf("..")
        err =  errors.New("no file/req")
        fmt.Println(f)
        fmt.Println(err)
        return
    }

I undid a change // api has max of 200 as of date My own problem since I set it to 1000 and page limit is 200

[bakape]

Yes, the faggots broke their API.

[bakape]

I submitted an issue. Will make boorufetch work with both later today.