go-ego / riot

Go Open Source, Distributed, Simple and efficient Search Engine; Warning: This is V1 and beta version, because of big memory consume, and the V2 will be rewrite all code.
Apache License 2.0
6.11k stars 473 forks source link

How do i update a index? #65

Closed jiankuny-zz closed 6 years ago

jiankuny-zz commented 6 years ago

19

can i use RemoveDoc(id, forceUpdate) and then Index(id, types.DocData{Content: title,},forceUpdate)

is that thread safe?

vcaesar commented 6 years ago

Index addition and deletion operations are thread safe.

jiankuny-zz commented 6 years ago

seems like i can directly call

Index(id, types.DocData{Content: text,},forceUpdate)

to update a doc, a exists doc's docState=0, after call

Index()

it marks docState to 1, once you call

Flush()

it will call

AddDocToCache(forceUpdate=true)

and call

RemoveDocToCache(forceUpdate=true) 
if indexer.addCacheLock.addCachePointer >= indexer.initOptions.DocCacheSize ||
forceUpdate {
    indexer.tableLock.Lock()

    position := 0
    for i := 0; i < indexer.addCacheLock.addCachePointer; i++ {
        docIndex := indexer.addCacheLock.addCache[i]

        docState, ok := indexer.tableLock.docsState[docIndex.DocId]
        if ok && docState <= 1 {
            // ok && docState == 0 表示存在于索引中,需先删除再添加
            // ok && docState == 1 表示不一定存在于索引中,等待删除,需先删除再添加
            if position != i {
                indexer.addCacheLock.addCache[position], indexer.addCacheLock.addCache[i] =
                indexer.addCacheLock.addCache[i], indexer.addCacheLock.addCache[position]
            }
            if docState == 0 {
                indexer.removeCacheLock.Lock()
                indexer.removeCacheLock.removeCache[indexer.removeCacheLock.removeCachePointer] =
                docIndex.DocId
                indexer.removeCacheLock.removeCachePointer++
                indexer.removeCacheLock.Unlock()

                indexer.tableLock.docsState[docIndex.DocId] = 1
                indexer.numDocs--
            }
            position++
        } else if !ok {
            indexer.tableLock.docsState[docIndex.DocId] = 2
        }
    }

    indexer.tableLock.Unlock()
    if indexer.RemoveDocToCache(0, forceUpdate) {
        // 只有当存在于索引表中的文档已被删除,其才可以重新加入到索引表中
        position = 0
    }

    addCachedDocs := indexer.addCacheLock.addCache[position:indexer.addCacheLock.addCachePointer]
    indexer.addCacheLock.addCachePointer = position

    indexer.addCacheLock.Unlock()
    sort.Sort(addCachedDocs)
    indexer.AddDocs(&addCachedDocs)
}

to remove the doc which docState=1 and delete the key in docState map, then it will call

AddDocs(addCache[postion:addCachePointer])

which position=0, so it will add all doc in addCache,

if ok && docState == 1 {
            // 如果此时 docState 仍为 1,说明该文档需被删除
            // docState 合法状态为 nil & 2,保证一定不会插入已经在索引表中的文档
            continue
        }

in this case, ok=false, so it will add the doc.

vcaesar commented 6 years ago

What do you express by calling the function call? This is obviously a bug. Closing is due to duplicate ( #19 ) and non-issue regulations.