go-ego / riot

Go Open Source, Distributed, Simple and efficient Search Engine; Warning: This is V1 and beta version, because of big memory consume, and the V2 will be rewrite all code.
Apache License 2.0
6.11k stars 474 forks source link

restart server data don`t persistent storage #93

Closed zlykernel closed 5 years ago

zlykernel commented 5 years ago

- my OS info: LSB Version: n/a Distributor ID: ManjaroLinux Description: Manjaro Linux Release: 18.0.4 - go version go1.12.4 linux/amd64 my operate step:

1.execute cmd : go run test.go ,index can add and search 2.change code only search 3.Ctrl+c and execute cmd : go run test.go ,no data find

my console log 2019/07/05 17:20:37 Dictionary file path is empty, load the default dictionary file. 2019/07/05 17:20:37 Load the gse dictionary: "/home/zlykernel/go/src/github.com/go-ego/gse/data/dict/dictionary.txt" 2019/07/05 17:20:40 Gse dictionary loaded finished. 2019/07/05 17:20:42 创建的索引数量: 0 2019/07/05 17:20:42 建索引开始 2019/07/05 17:20:42 添加了文件标题为: title1 2019/07/05 17:20:42 创建的索引数量: 1 2019/07/05 17:20:42 after Add 创建的索引数量: 1 2019/07/05 17:20:42 建索引完毕 ,创建的索引数量 = 1 2019/07/05 17:20:42 查询: {{[content1] false 1} [{{2019-07-05 17:20:42 [0] [] []} content1 {title1 author1 2019-07-05 17:20:42 0} <nil>}]} zlykernel@zlykernel-pc  ~/go/src/livecode-riotapp/middleware/riot_server/test  go run test.go 2019/07/05 17:20:57 Dictionary file path is empty, load the default dictionary file. 2019/07/05 17:20:57 Load the gse dictionary: "/home/zlykernel/go/src/github.com/go-ego/gse/data/dict/dictionary.txt" 2019/07/05 17:20:59 Gse dictionary loaded finished. 2019/07/05 17:21:01 创建的索引数量: 0 2019/07/05 17:21:01 查询: {{[content1] false 0} []} - my code: package main

import ( "log" "net/http" "time" "os" "os/signal" // rhttp "github.com/go-ego/riot/net/http"

"github.com/go-ego/riot"
"github.com/go-ego/riot/types"

)

var ( // searcher is coroutine safe searcher = riot.Engine{}

opts = types.EngineOpts{
    Using: 1,
    IndexerOpts: &types.IndexerOpts{
        IndexType: types.DocIdsIndex,
    },
    UseStore: true,
    StoreFolder: "riot-index-data",
    // StoreEngine: "bg", // bg: badger, lbd: leveldb, bolt: bolt
}

)

//标题、作者、标签 func AddContent(title,content,author string,labels[]string)(err error) { //defer searcher.Close() //错误 log.Println("添加了文件标题为: ", title)

//err = GetError()
//为文件添加属性
attri :=types.Attri{title,author,time.Now().Format("2006-01-02 15:04:05"),0}
ct :=types.DocData{Content: content}
ct.Attri=attri
//如果类别不为空则添加类别
if labels != nil || len(labels) >0{
    ct.Labels = labels
}
//生成主键
docid := time.Now().Format("2006-01-02 15:04:05")
//插入
searcher.Index(docid, ct)
//刷新
searcher.Flush()
log.Println("创建的索引数量: ", searcher.NumDocsIndexed())
return err

}

func Add(w http.ResponseWriter, req *http.Request){ AddContent("title1","content1","author1",[]string{"false"}) log.Println("after Add 创建的索引数量: ", searcher.NumDocsIndexed()) // restoreIndex() }

func restoreIndex() { searcher.Init(opts) defer searcher.Close() searcher.Flush() log.Println("recover index number: ", searcher.NumDocsIndexed()) }

func main() { searcher.Init(opts) // restoreIndex() log.Println("创建的索引数量: ", searcher.NumDocsIndexed()) // http.HandleFunc("/search", rhttp.Search) // http.HandleFunc("/dist", rhttp.WgDist) // http.HandleFunc("/Add", Add) // log.Println("listen and serve on 8080...") // log.Fatal(http.ListenAndServe(":8088", nil))

// log.Println("建索引开始")
// Add(nil,nil)
// log.Println("建索引完毕 ,创建的索引数量 = ",searcher.NumDocsIndexed())

// searcher.Flush()

    // 捕获 ctrl-c
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
go func() {
    for range c {
        log.Println("捕获Ctrl-c,退出服务器")
        searcher.Close()
        os.Exit(0)
    }
}()

sea := searcher.Search(types.SearchReq{
    Text: "content1",
    RankOpts: &types.RankOpts{
        OutputOffset: 0,
        MaxOutputs:   100,
    }})
log.Println("查询: ", sea)

defer searcher.Close()

}

`

zlykernel commented 5 years ago

this problem has already solved! add gob.Register(types.Attri{}) before init engine like this: import "github.com/go-ego/riot/types" func init() { gob.Register(types.Attri{}) rpc.InitEngine(config) }