Panic on proxy argument when running `generate --watch`

grindlemire commented 8 months ago

Hi! I am running into a panic when trying to hotreload templ files using the templ generate --watch command. It looks like a failure when attempting to roundtrip the proxy request.

Panic

2023/12/31 23:03:16 http: panic serving 127.0.0.1:46666: runtime error: invalid memory address or nil pointer dereference
goroutine 50 [running]:
net/http.(*conn).serve.func1()
        /usr/local/go/src/net/http/server.go:1868 +0xb9
panic({0x886300?, 0xc810e0?})
        /usr/local/go/src/runtime/panic.go:920 +0x270
github.com/a-h/templ/cmd/templ/generatecmd/proxy.(*roundTripper).RoundTrip(0xc0000be000, 0xc000228300)
        /home/grindlemire/go/pkg/mod/github.com/a-h/templ@v0.2.501/cmd/templ/generatecmd/proxy/proxy.go:97 +0xc1
net/http/httputil.(*ReverseProxy).ServeHTTP(0xc0000ba000, {0x9d0f68, 0xc00025e000}, 0xc0001be000)
        /usr/local/go/src/net/http/httputil/reverseproxy.go:473 +0xe9a
github.com/a-h/templ/cmd/templ/generatecmd/proxy.(*Handler).ServeHTTP(0xc0000bc060, {0x9d0f68?, 0xc00025e000}, 0xc0001be000)
        /home/grindlemire/go/pkg/mod/github.com/a-h/templ@v0.2.501/cmd/templ/generatecmd/proxy/proxy.go:78 +0xbe
net/http.serverHandler.ServeHTTP({0xc0001982d0?}, {0x9d0f68?, 0xc00025e000?}, 0x6?)
        /usr/local/go/src/net/http/server.go:2938 +0x8e
net/http.(*conn).serve(0xc0001b6000, {0x9d2078, 0xc0001981e0})
        /usr/local/go/src/net/http/server.go:2009 +0x5f4
created by net/http.(*Server).Serve in goroutine 21
        /usr/local/go/src/net/http/server.go:3086 +0x5cb
Server not ready. Retrying in 590.892911ms...

Reproducible Setup

`main.go`

package main

import (
    "fmt"
    "net/http"

    "github.com/a-h/templ"
)

func main() {
    component := hello("foobar")

    http.Handle("/", templ.Handler(component))

    fmt.Println("Listening on :3000")
    http.ListenAndServe(":3000", nil)
}

`base.templ`

package main

templ hello(name string) {
    <div>Hello, { name }</div>
}

command

templ generate --watch --proxy="http://localhost:8081"

muradbu commented 8 months ago

I'm having the same issue even after updating to include the fix for #358.

templ generate --watch --proxy="http://localhost:3000"
Processing path: /Users/murad/Developer/seedbox
(✓) Generated code for "/Users/murad/Developer/seedbox/web/dashboard/index.templ" in 524.125µs
(✓) Generated code for "/Users/murad/Developer/seedbox/web/index.templ" in 497.875µs
(✓) Generated code for "/Users/murad/Developer/seedbox/internal/templates/layout.templ" in 757.125µs
(✓) Generated code for 3 templates with 0 errors in 1.102875ms
Proxying from http://127.0.0.1:7331 to target: http://localhost:3000
Opening URL: http://localhost:3000
(!) templ version check failed: failed to parse go.mod file: /Users/murad/Developer/seedbox/go.mod:3: invalid go version '1.21.5': must match format 1.23
(!) templ version check failed: failed to parse go.mod file: /Users/murad/Developer/seedbox/go.mod:3: invalid go version '1.21.5': must match format 1.23
2024/01/01 14:49:15 http: panic serving 127.0.0.1:64797: runtime error: invalid memory address or nil pointer dereference
goroutine 9 [running]:
net/http.(*conn).serve.func1()
    /opt/homebrew/Cellar/go/1.21.3/libexec/src/net/http/server.go:1868 +0xb0
panic({0x100a2e5a0?, 0x100d5b290?})
    /opt/homebrew/Cellar/go/1.21.3/libexec/src/runtime/panic.go:920 +0x26c
github.com/a-h/templ/cmd/templ/generatecmd/proxy.(*roundTripper).RoundTrip(0x14000126000, 0x14000132200)
    /Users/murad/go/pkg/mod/github.com/a-h/templ@v0.2.501/cmd/templ/generatecmd/proxy/proxy.go:97 +0xb4
net/http/httputil.(*ReverseProxy).ServeHTTP(0x14000122000, {0x100abd2e8, 0x14000150000}, 0x140000f6400)
    /opt/homebrew/Cellar/go/1.21.3/libexec/src/net/http/httputil/reverseproxy.go:473 +0xbe0
github.com/a-h/templ/cmd/templ/generatecmd/proxy.(*Handler).ServeHTTP(0x14000124060, {0x100abd2e8?, 0x14000150000}, 0x140000f6400)
    /Users/murad/go/pkg/mod/github.com/a-h/templ@v0.2.501/cmd/templ/generatecmd/proxy/proxy.go:78 +0xa4
net/http.serverHandler.ServeHTTP({0x140000fadb0?}, {0x100abd2e8?, 0x14000150000?}, 0x6?)
    /opt/homebrew/Cellar/go/1.21.3/libexec/src/net/http/server.go:2938 +0xbc
net/http.(*conn).serve(0x140000a65a0, {0x100abe328, 0x14000222d80})
    /opt/homebrew/Cellar/go/1.21.3/libexec/src/net/http/server.go:2009 +0x518
created by net/http.(*Server).Serve in goroutine 34
    /opt/homebrew/Cellar/go/1.21.3/libexec/src/net/http/server.go:3086 +0x4cc
Server not ready. Retrying in 668.571296ms...

go version
go version go1.21.5 darwin/arm64

templ version
v0.2.501

eussam commented 8 months ago

Hi, are you sure you're using the fix #358 ?, I believe it has been merged with version 0.2.508 Maybe you want to install the templ CLI currently from the main branch:

go install github.com/a-h/templ/cmd/templ@main

Skylli202 commented 8 months ago

Hi,

I can report the same issue. With just the commands:

go install github.com/a-h/templ/cmd/templ
go get github.com/a-h/templ

➜ go version
go version go1.21.4 linux/amd64
➜ templ version
v0.2.501

After, doing what @eussam said (go get -u github.com/a-h/templ), I have the following message: (!) templ version check failed: generator v0.2.508 is newer than templ version v0.2.502-0.20231231113737-d5d2dec17d9e found in go.mod file, consider running 'go get -u github.com/a-h/templ' to upgrade.

I confirmed that the update of templ command with a templ version that returned v0.2.508.

After running the go get -u github.com/a-h/templ, the warning does not go away. I then tried go get -u github.com/a-h/templ@main. Same the warning is still here.

This is annoying because, the hot-reload command from the doc does not work due to this issue.

Edit 1: I manually updated my go.mod file to have the following line: require github.com/a-h/templ v0.2.508 // indirect and the generation pass w/o warnings/errors.

Edit 2: The LSP is now not happy about the new version of templ. WindowsTerminal_NktNUcTpTZ

Edit 3: I finally run the go install and go get command with the @d5d2dec17d9eec2a95f6728eb5c073bd2243adf2 (which is the latest commit on main at the time). Did not solve the warning, but did solve LSP issues. Also the templ generate --watch --proxy="http://localhost:8080" --cmd="runtest" does not work, the proxy fails to connect. When I curl the proxy URL displayed in console, I go a 502 Bad Gateway.

drornir commented 8 months ago

@eussam How can I help prepare a new production release?

eussam commented 8 months ago

@drornir I'm not the right person to ask :), I just participated recently to #358

regarding the templ built in hot reload feature, I am still experiencing some problems personaly like:

reload of the page staying in the pending state.
quite important delay between update file detection and actual templ generation compared to the wgo utility (cf: #386)

but I have to spend some time on it.

a-h commented 8 months ago

I think that what we need here is a decent integration test for the watch feature. The test needs to:

Create a temp directory and initialise a new templ project in it (main.go and template.templ). The main.go runs a web server on a port that's passed as an argument.
Run templ generate from the cmd directory, as the other unit tests do, to generate code.
Run the templ watch operation, proxying through to the templ project.
Assert that...
- HTML returned from the proxy port contains the hot reload script, i.e. do a GET request to the proxy port and check the results.
- The HTML returned from the proxy port is what we expected.
Next, the test should connect to the SSE endpoint of the proxy and listen for updates.
Then, the test needs to modify the template.templ file, and assert that...
- The template_templ.go file was updated.
- HTML returned from the proxy port contains the new changes (i.e. do another GET request to the proxy port).
- The SSE endpoint received a trigger event (i.e. the browser was notified of changes and instructed to reload).

A more advanced version would test both POST and GET requests, and test that delays in starting up the main.go (i.e. having a time.Sleep in there) don't cause issues with the proxy, plus do other tests, like make lots of changes to files in sequence.

With this integration style test in place, we should be able to reproduce issues more easily, and solve any weirdness.

a-h commented 7 months ago

I added a test to the project that completes the assertions above in https://github.com/a-h/templ/pull/441

Those changes have hit the main branch today.

On the main issue, the --watch behaviour has been completely changed, there's a really cool new dev mode that only reloads the web server if the Go code within templates has changed: https://github.com/a-h/templ/pull/366

So I think it's unlikely we'll see the the same behaviour. If folks can re-test with the latest version on main, that would be great.

If you run go install github.com/a-h/templ/cmd/templ it installs the latest tagged version, i.e. the latest formal release.

If you run go install github.com/a-h/templ/cmd/templ@cac13b5f96806286cc934a58745bd100f656872c (change the commit ID to install whatever you want), you can install a specific commit from main.

I'll close this issue in a week or so if there's no more to go on.

a-h commented 7 months ago

Fix released in https://github.com/a-h/templ/releases/tag/v0.2.543 - upgrade with go install github.com/a-h/templ/cmd/templ@v0.2.543

haydenrou commented 7 months ago

Hi @a-h - I seem to still be seeing a similar issue here on the following versions:

templ version
v0.2.543
go version go1.21.0 darwin/arm64

This is what I'm seeing:

Proxying from http://127.0.0.1:7331 to target: http://localhost:1323
Server not ready. Retrying in 701.315134ms...

Then it actually runs successfully, however I get this immediately after

⇨ http server started on [::]:1323
Proxy to target error: http: proxy error: max retries reached
Proxy to target error: http: proxy error: max retries reached

The command I'm using is: (my server is running on 1323)

templ generate --watch --proxy="http://localhost:1323" --cmd="go run ."

Apologies if this is not related, should I create a separate issue?

a-h commented 7 months ago

I've completely rewritten the watch feature since the release you're using, but I haven't released it yet.

If you could test by building the latest commit of the main branch from source, that would be great.

I would like to have a GitHub workflow process that builds a pre-release version, but haven't set it up yet. I think it would help people to try out the upcoming versions.

haydenrou commented 7 months ago

Thanks for the fast response @a-h - I upgraded using go install github.com/a-h/templ/cmd/templ@a5c60767ef03cfb52db78cb666df5a8df9f7a7a5 and ran into the same issue - see screenshot below.

For the record, it works fine for everything else, and just manually reloading the page, but it doesn't auto reload.

haydenrou commented 6 months ago

@a-h apologies if this isn't the correct forum for the question but I had another question regarding the proxy. If I run templ generate I get a different .templ.go file generated (containing the HTML) than if I run it via templ generate --watch --proxy="http://localhost:1323" --cmd="go run .".

Is it expected that they differ, or am I expected to re-run templ generate after running my proxy and then commit the changes made.

a-h commented 6 months ago

It's not really the forum, no. This is a closed issue, and your issue isn't related, but... 😁 it's expected.

If you run templ generate --watch it generates developer mode files which behave differently. In --watch mode, HTML and text is loaded from files on the filesystem instead of being compiled into the binary.

This is great locally, because you don't need to recompile your app to see changes to the HTML and text, but isn't at all what you want for an optimised production build because you don't want HTTP requests reading from the file system (terrible performance).

Since https://github.com/a-h/templ/commit/5821706bae5ad461ec011a1e3b4f6a663fbcaf3d --watch mode runs a standard production build afterwards, however, it's not part of a release yet. I want to write some automated tests for the LSP behaviour before I cut the next release, because I've made some huge refactors in the parser, and want a bit of extra safety.

haydenrou commented 6 months ago

Appreciate the response, sorry for the off-topic question!

a-h / templ