search

Allenxuxu / gev

🚀Gev is a lightweight, fast non-blocking TCP network library / websocket server based on Reactor mode. Support custom protocols to quickly and easily build high-performance servers.
MIT License
1.73k stars 194 forks source link

centos6.5 32位系统下 RussellLuo/timingwheel 运行后 panic #8

Closed defsky closed 5 years ago

defsky commented 5 years ago

panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a6bc]

goroutine 1 [running]: runtime/internal/atomic.Xchg64(0xa1201ac, 0xc9fda7b8, 0x16d, 0x0, 0x38b) /usr/lib/golang/src/runtime/internal/atomic/asm_386.s:151 +0xc github.com/Allenxuxu/timingwheel%2eV2.(bucket).SetExpiration(...) /home/gohome/pkg/mod/github.com/!allenxuxu/timingwheel.!v2@v0.0.0-20190920124930-519aff1ed880/bucket.go:74 github.com/Allenxuxu/timingwheel%2eV2.(TimingWheel).add(0xa0ae0a0, 0xa0ade80, 0xa0ae0a0) /home/gohome/pkg/mod/github.com/!allenxuxu/timingwheel.!v2@v0.0.0-20190920124930-519aff1ed880/timingwheel.go:79 +0x167 github.com/Allenxuxu/timingwheel%2eV2.(TimingWheel).add(0xa0ae000, 0xa0ade80, 0xa0adea0) /home/gohome/pkg/mod/github.com/!allenxuxu/timingwheel.!v2@v0.0.0-20190920124930-519aff1ed880/timingwheel.go:106 +0x1f2 github.com/Allenxuxu/timingwheel%2eV2.(TimingWheel).addOrRun(0xa0ae000, 0xa0ade80) /home/gohome/pkg/mod/github.com/!allenxuxu/timingwheel.!v2@v0.0.0-20190920124930-519aff1ed880/timingwheel.go:113 +0x29 github.com/Allenxuxu/timingwheel%2eV2.(TimingWheel).EveryFunc(0xa0ae000, 0x3b9aca00, 0x0, 0xa0782c0, 0xa07c3a0) /home/gohome/pkg/mod/github.com/!allenxuxu/timingwheel.!v2@v0.0.0-20190920124930-519aff1ed880/timingwheel.go:185 +0x190 github.com/Allenxuxu/gev.(Server).RunEvery(...) /home/gohome/pkg/mod/github.com/!allenxuxu/gev@v0.0.2/server.go:84 main.(*Server).Start(0xa07c3a0) /home/gohome/pkg/mod/github.com/!allenxuxu/gev@v0.0.2/example/pushmessage/main.go:33 +0x72 main.main() /home/gohome/pkg/mod/github.com/!allenxuxu/gev@v0.0.2/example/pushmessage/main.go:90 +0x85 exit status 2

系统信息： Linux develop 2.6.32-358.el6.i686 #1 SMP Thu Feb 21 21:50:49 UTC 2013 i686 i686 i386 GNU/Linux

Allenxuxu commented 5 years ago

从报错信息来看，是 timingwheel 库的 bug，我会尽快修复。

Allenxuxu commented 5 years ago

timingwheel 库现在已经支持 定时任务，所以我将自己临时修改的 timingwheel 库替换。 https://github.com/RussellLuo/timingwheel

老哥，你再测试看看。

我本地测试是没有问题的，你方便的话，发下你的测试脚本。

defsky commented 5 years ago

测试脚本就是这个项目里面的 example/pushmessage/main.go

还是这个报错，是不是不支持centos6.5呢？ centos7里面跑就对的

panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a6bc]

goroutine 1 [running]: runtime/internal/atomic.Xchg64(0x8d1778c, 0xcd350110, 0x16d, 0x0, 0x362) /usr/lib/golang/src/runtime/internal/atomic/asm_386.s:151 +0xc github.com/RussellLuo/timingwheel.(bucket).SetExpiration(...) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/bucket.go:74 github.com/RussellLuo/timingwheel.(TimingWheel).add(0x8c9a0a0, 0x8c99ea0, 0x8c9a0a0) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:79 +0x167 github.com/RussellLuo/timingwheel.(TimingWheel).add(0x8c9a000, 0x8c99ea0, 0x8c99ec0) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:106 +0x1f2 github.com/RussellLuo/timingwheel.(TimingWheel).addOrRun(0x8c9a000, 0x8c99ea0) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:113 +0x29 github.com/RussellLuo/timingwheel.(TimingWheel).ScheduleFunc(0x8c9a000, 0x817bd20, 0x8c14358, 0x8c102e8, 0x2) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:222 +0x2b5 github.com/defsky/gev.(Server).RunEvery(...) /home/gohome/pkg/mod/github.com/defsky/gev@v0.0.3/server.go:86 main.(*Server).Start(0x8c6a060) /home/repo/myserver/main.go:40 +0x136 main.main() /home/repo/myserver/main.go:97 +0x85 exit status 2

Allenxuxu commented 5 years ago

你把你的执行命令 和 回显 信息都贴上来我看看

defsky commented 5 years ago

这样？ 这个main.go就是复制的 pushmessage/main.go [root@develop myserver]# go run main.go 2019/10/15 10:26:20 Starting server at :5555 panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a6bc]

goroutine 1 [running]: runtime/internal/atomic.Xchg64(0x9123e2c, 0xcd3cdcc8, 0x16d, 0x0, 0x17d) /usr/lib/golang/src/runtime/internal/atomic/asm_386.s:151 +0xc github.com/RussellLuo/timingwheel.(bucket).SetExpiration(...) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/bucket.go:74 github.com/RussellLuo/timingwheel.(TimingWheel).add(0x90ac0a0, 0x90abe80, 0x90ac0a0) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:79 +0x167 github.com/RussellLuo/timingwheel.(TimingWheel).add(0x90ac000, 0x90abe80, 0x90abea0) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:106 +0x1f2 github.com/RussellLuo/timingwheel.(TimingWheel).addOrRun(0x90ac000, 0x90abe80) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:113 +0x29 github.com/RussellLuo/timingwheel.(TimingWheel).ScheduleFunc(0x90ac000, 0x817bd20, 0x908a2b0, 0x90762d0, 0x2) /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:222 +0x2b5 github.com/defsky/gev.(Server).RunEvery(...) /home/gohome/pkg/mod/github.com/defsky/gev@v0.0.3/server.go:86 main.(*Server).Start(0x909a030) /home/repo/myserver/main.go:40 +0x136 main.main() /home/repo/myserver/main.go:97 +0x85 exit status 2

Allenxuxu commented 5 years ago

你把 main.go 也贴出来把

defsky commented 5 years ago

package main

import ( "container/list" "github.com/defsky/gev" "github.com/Allenxuxu/gev/connection" "github.com/Allenxuxu/ringbuffer" "log" "sync" "time" )

type Server struct { ip string port string conn list.List mu sync.RWMutex server gev.Server }

func New(ip, port string) (*Server, error) { var err error s := new(Server) s.conn = list.New() s.server, err = gev.NewServer(s, gev.Address(ip+":"+port)) if err != nil { return nil, err }

s.ip = ip
s.port = port

return s, nil

}

func (s *Server) Start() { log.Printf("Starting server at %s:%s\n", s.ip, s.port)

s.server.RunEvery(1*time.Second, s.RunPush)
s.server.Start()

}

func (s *Server) Stop() { s.server.Stop() }

func (s Server) RunPush() { var next list.Element

s.mu.RLock()
defer s.mu.RUnlock()

for e := s.conn.Front(); e != nil; e = next {
    next = e.Next()

    c := e.Value.(*connection.Connection)
    _ = c.Send([]byte("hello\n"))
}

}

func (s Server) OnConnect(c connection.Connection) { log.Println(" OnConnect ： ", c.PeerAddr())

s.mu.Lock()
e := s.conn.PushBack(c)
s.mu.Unlock()
c.SetContext(e)

} func (s Server) OnMessage(c connection.Connection, buffer *ringbuffer.RingBuffer) (out []byte) { log.Println("OnMessage") first, end := buffer.PeekAll() out = first if len(end) > 0 { out = append(out, end...) } buffer.RetrieveAll() return }

func (s Server) OnClose(c connection.Connection) { log.Println("OnClose") e := c.Context().(*list.Element)

s.mu.Lock()
s.conn.Remove(e)
s.mu.Unlock()

}

func main() { s, err := New("", "5555") if err != nil { panic(err) } defer s.Stop()

s.Start()

}

Allenxuxu commented 5 years ago 
package main

import (
    "container/list"
    "github.com/Allenxuxu/gev"
    "github.com/Allenxuxu/gev/connection"
    "github.com/Allenxuxu/ringbuffer"
    "log"
    "sync"
    "time"
)

type Server struct {
    ip     string
    port   string
    conn   *list.List
    mu     sync.RWMutex
    server *gev.Server
}

func New(ip, port string) (*Server, error) {
    var err error
    s := new(Server)
    s.conn = list.New()
    s.server, err = gev.NewServer(s,
        gev.Address(ip+":"+port))
    if err != nil {
        return nil, err
    }

    s.ip = ip
    s.port = port

    return s, nil
}

func (s *Server) Start() {
    log.Printf("Starting server at %s:%s\n", s.ip, s.port)

    s.server.RunEvery(1*time.Second, s.RunPush)
    s.server.Start()
}

func (s *Server) Stop() {
    s.server.Stop()
}

func (s *Server) RunPush() {
    var next *list.Element

    s.mu.RLock()
    defer s.mu.RUnlock()

    for e := s.conn.Front(); e != nil; e = next {
        next = e.Next()

        c := e.Value.(*connection.Connection)
        _ = c.Send([]byte("hello\n"))
    }
}

func (s *Server) OnConnect(c *connection.Connection) {
    log.Println(" OnConnect ： ", c.PeerAddr())

    s.mu.Lock()
    e := s.conn.PushBack(c)
    s.mu.Unlock()
    c.SetContext(e)
}
func (s *Server) OnMessage(c *connection.Connection, buffer *ringbuffer.RingBuffer) (out []byte) {
    log.Println("OnMessage")
    first, end := buffer.PeekAll()
    out = first
    if len(end) > 0 {
        out = append(out, end...)
    }
    buffer.RetrieveAll()
    return
}

func (s *Server) OnClose(c *connection.Connection) {
    log.Println("OnClose")
    e := c.Context().(*list.Element)

    s.mu.Lock()
    s.conn.Remove(e)
    s.mu.Unlock()
}

func main() {
    s, err := New("", "5555")
    if err != nil {
        panic(err)
    }
    defer s.Stop()

    s.Start()
}
go run main.go 
2019/10/15 10:49:28 Starting server at :5555
2019/10/15 10:49:50  OnConnect ：  127.0.0.1:63521
2019/10/15 10:49:55 OnClose

我这样启动没问题，区别就在于 导入包 "github.com/defsky/gev" 。

你复制我这个代码，再试试。

ps：建议学一下 markdown 的语法，这样贴代码比较容易阅读。

defsky commented 5 years ago

就算直接运行 gev/example/pushmessage/main.go，都是这样的

[root@develop pushmessage]# echo $GOPATH
/home/gohome
[root@develop pushmessage]# pwd
/home/gohome/src/gev/example/pushmessage
[root@develop pushmessage]# go run main.go
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a6bc]

goroutine 1 [running]:
runtime/internal/atomic.Xchg64(0x9108e4c, 0xcd623748, 0x16d, 0x0, 0x33d)
        /usr/lib/golang/src/runtime/internal/atomic/asm_386.s:151 +0xc
github.com/RussellLuo/timingwheel.(*bucket).SetExpiration(...)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/bucket.go:74
github.com/RussellLuo/timingwheel.(*TimingWheel).add(0x909a0a0, 0x9099ea0, 0x909a0a0)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:79 +0x167
github.com/RussellLuo/timingwheel.(*TimingWheel).add(0x909a000, 0x9099ea0, 0x9099ec0)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:106 +0x1f2
github.com/RussellLuo/timingwheel.(*TimingWheel).addOrRun(0x909a000, 0x9099ea0)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:113 +0x29
github.com/RussellLuo/timingwheel.(*TimingWheel).ScheduleFunc(0x909a000, 0x817bbb0, 0x9014320, 0x90102d8, 0x806ff75)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:222 +0x2b5
github.com/Allenxuxu/gev.(*Server).RunEvery(...)
        /home/gohome/src/gev/server.go:86
main.(*Server).Start(0x90503c0)
        /home/gohome/src/gev/example/pushmessage/main.go:36 +0x97
main.main()
        /home/gohome/src/gev/example/pushmessage/main.go:99 +0x85
exit status 2
Allenxuxu commented 5 years ago

你的 go 版本和环境发下，我看看能不能复现。

defsky commented 5 years ago

这个？

[root@develop pushmessage]# uname -a
Linux develop 2.6.32-358.el6.i686 #1 SMP Thu Feb 21 21:50:49 UTC 2013 i686 i686 i386 GNU/Linux
[root@develop pushmessage]# go version
go version go1.13 linux/386
[root@develop pushmessage]# go env
GO111MODULE="on"
GOARCH="386"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="386"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/gohome"
GOPRIVATE=""
GOPROXY="https://goproxy.io"
GOROOT="/usr/lib/golang"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_386"
GCCGO="gccgo"
GO386="sse2"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/gohome/src/gev/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m32 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build136523118=/tmp/go-build"
[root@develop pushmessage]#
Allenxuxu commented 5 years ago

我这边没你这个机器的环境，我看日志觉得还是 timingwheel 库 的问题。

你先试试运行下面这个，会不会 panic ：

package main

import (
    "fmt"
    "time"

    "github.com/RussellLuo/timingwheel"
)

type EveryScheduler struct {
    Interval time.Duration
}

func (s *EveryScheduler) Next(prev time.Time) time.Time {
    return prev.Add(s.Interval)
}

func main() {
    tw := timingwheel.NewTimingWheel(time.Millisecond, 20)
    tw.Start()
    defer tw.Stop()

    _ = tw.ScheduleFunc(&EveryScheduler{time.Second}, func() {
        fmt.Println("The timer fires")
    })

    select {}
}
defsky commented 5 years ago

我这边没你这个机器的环境，我看日志觉得还是 timingwheel 库 的问题。

你先试试运行下面这个，会不会 panic ：

package main

import (
  "fmt"
  "time"

  "github.com/RussellLuo/timingwheel"
)

type EveryScheduler struct {
  Interval time.Duration
}

func (s *EveryScheduler) Next(prev time.Time) time.Time {
  return prev.Add(s.Interval)
}

func main() {
  tw := timingwheel.NewTimingWheel(time.Millisecond, 20)
  tw.Start()
  defer tw.Stop()

  _ = tw.ScheduleFunc(&EveryScheduler{time.Second}, func() {
      fmt.Println("The timer fires")
  })

  select {}
}

这个也会panic

[root@develop test]# go run main.go
go: finding github.com/RussellLuo/timingwheel latest
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8049fac]

goroutine 1 [running]:
runtime/internal/atomic.Xchg64(0x9c66bcc, 0xcdfcee50, 0x16d, 0x0, 0x1)
        /usr/lib/golang/src/runtime/internal/atomic/asm_386.s:151 +0xc
github.com/RussellLuo/timingwheel.(*bucket).SetExpiration(...)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/bucket.go:74
github.com/RussellLuo/timingwheel.(*TimingWheel).add(0x9c761e0, 0x9c66620, 0x9c761e0)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:79 +0x167
github.com/RussellLuo/timingwheel.(*TimingWheel).add(0x9c76140, 0x9c66620, 0x9c76140)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:106 +0x1f2
github.com/RussellLuo/timingwheel.(*TimingWheel).add(0x9c760a0, 0x9c66620, 0x9c66640)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:106 +0x1f2
github.com/RussellLuo/timingwheel.(*TimingWheel).addOrRun(0x9c760a0, 0x9c66620)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:113 +0x29
github.com/RussellLuo/timingwheel.(*TimingWheel).ScheduleFunc(0x9c760a0, 0x81173c0, 0x9c120b8, 0x8105430, 0x9c760a0)
        /home/gohome/pkg/mod/github.com/!russell!luo/timingwheel@v0.0.0-20190930141959-b40c04085073/timingwheel.go:222 +0x2b5
main.main()
        /home/repo/test/main.go:23 +0xba
exit status 2
Allenxuxu commented 5 years ago

是的， bug 就在这里 timingwheel 库里。 但是我在 centos7 的机器上是没问题的，你是只有 centos6.5 的机器会有问题吗？

defsky commented 5 years ago

是的， bug 就在这里 timingwheel 库里。 但是我在 centos7 的机器上是没问题的，你是只有 centos6.5 的机器会有问题吗？

是的，centos 7正常，centos 6.5会有这个panic

Allenxuxu commented 5 years ago

https://github.com/RussellLuo/timingwheel/issues/13

我晚上也会再配个环境，找找问题。

Allenxuxu commented 5 years ago

老哥，可以帮忙再测测吗？

下面的代码，再运行看看，看看会不会 panic ：

ps：和之前的代码，导入的包地址不通。

package main

import (
    "fmt"
    "time"

    "github.com/Allenxuxu/timingwheel"
)

type EveryScheduler struct {
    Interval time.Duration
}

func (s *EveryScheduler) Next(prev time.Time) time.Time {
    return prev.Add(s.Interval)
}

func main() {
    tw := timingwheel.NewTimingWheel(time.Millisecond, 20)
    tw.Start()
    defer tw.Stop()

    _ = tw.ScheduleFunc(&EveryScheduler{time.Second}, func() {
        fmt.Println("The timer fires")
    })

    select {}
}
defsky commented 5 years ago

好了。

[root@develop test]# go run main.go
go: finding github.com/Allenxuxu/timingwheel latest
go: downloading github.com/Allenxuxu/timingwheel v0.0.0-20191015090400-4a0bc1c55c54
go: extracting github.com/Allenxuxu/timingwheel v0.0.0-20191015090400-4a0bc1c55c54
The timer fires
The timer fires
The timer fires
The timer fires
The timer fires
The timer fires
The timer fires
The timer fires
The timer fires
The timer fires
^Csignal: interrupt
[root@develop test]#
Allenxuxu commented 5 years ago 
...
TEXT  runtime∕internal∕atomic·Xchg64(SB),NOSPLIT,$0-20
--
  | // no XCHGQ so use CMPXCHG8B loop
  | MOVL    ptr+0(FP), BP
  | TESTL   $7, BP
  | JZ  2(PC)
  | MOVL    0, AX // crash when unaligned
...

上面最后一行，注释说明了，未对齐会崩溃。 你是32 位机器，可能内存未对齐，导致了崩溃。 我 fork了 timingwheel 仓库，调整了字段位置，使内存对齐了 ：https://github.com/Allenxuxu/timingwheel/commit/4a0bc1c55c54da30bcb68d6986315c7a0e08c312

我已经给 RussellLuo/timingwheel PR 了 ：https://github.com/RussellLuo/timingwheel/pull/14

defsky commented 5 years ago

好的 谢谢！ 等你更新了gev库后 我就可以正常用了吗？

Allenxuxu commented 5 years ago

我已经提交了代码了：将 RussellLuo/timingwheel 替换成 Allenxuxu/timingwheel 了。

RussellLuo 合并我的代码后，我再切回来，现在你可以正常使用。

感谢帮忙测试🙏

Allenxuxu commented 5 years ago

RussellLuo 已经合并我的代码，已经切回将 RussellLuo/timingwheel ，可以拉取最新 gev 代码获取。

defsky commented 5 years ago

RussellLuo 已经合并我的代码，已经切回将 RussellLuo/timingwheel ，可以拉取最新 gev 代码获取。

go get 获取到的怎么总是v0.0.2这个版本 是不是还需要发给release才能获取到最新版本呢？

Allenxuxu commented 5 years ago

已经发布 release v0.1.0 https://github.com/Allenxuxu/gev/releases/tag/v0.1.0