Closed GoogleCodeExporter closed 8 years ago
protected Set<BytesRef> getQueryTokenSet(String query, Analyzer analyzer) { try { final Set<BytesRef> tokens = new HashSet<BytesRef>(); final TokenStream tokenStream = analyzer.tokenStream("", new StringReader(query)); final TermToBytesRefAttribute bytesAtt = tokenStream.getAttribute(TermToBytesRefAttribute.class); final BytesRef bytes = bytesAtt.getBytesRef(); tokenStream.reset(); while (tokenStream.incrementToken()) { bytesAtt.fillBytesRef(); tokens.add(BytesRef.deepCopyOf(bytes)); } tokenStream.end(); tokenStream.close(); return tokens; } catch (IOException ioe) { throw new RuntimeException("Error occured while iterating over tokenstream", ioe); } } 4.0版 tokenStream.end(); tokenStream.close(); 将tokenStream关闭了,在读取input的过程中会抛出Caused by: java.io.IOException: Stream closed 异常
Original issue reported on code.google.com by Dino...@gmail.com on 19 Oct 2012 at 7:53
Dino...@gmail.com
关于4.0请使用最新版本的IK2012_FF
Original comment by linliang...@gmail.com on 23 Oct 2012 at 9:20
linliang...@gmail.com
貌似是最快沙发
Original comment by wangwei....@gmail.com on 23 Oct 2012 at 9:27
wangwei....@gmail.com
Original issue reported on code.google.com by
Dino...@gmail.com
on 19 Oct 2012 at 7:53