Problem string

[scottbouma]

I have uncovered a problem with certain strings in scws. The simplest example I can find is "ｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏｏ", which causes an error on http://www.xunsearch.com/scws/demo/v48.php.

But if I remove a single character from the problem string, then it succeeds.

[hightman]

It seems this character is treated as english, I will track this issue, thanks.

[hightman]

A bit strange, I could not reproduce the BUG on MacOS.

[scottbouma]

Interesting. We first encountered the issue when using the "zhparser" extension for PostgreSQL (which relies on SCWS for tokenization). We are running zhparser with PostgreSQL on Ubuntu, and the problem occurs there.

Thanks for looking into it.

On Wed, Dec 2, 2015, 6:35 PM hightman notifications@github.com wrote:

A bit strange, I could not reproduce the BUG on MacOS.

— Reply to this email directly or view it on GitHub https://github.com/hightman/scws/issues/21#issuecomment-161487991.

[hightman]

@scottbouma hi, I just recompile libscws, and the problem actually gone. see: http://www.xunsearch.com/scws/demo/v48.php

[scottbouma]

This is great! Is there an updated binary that I can download and try out on our system?

If so I can let you know in about 6 hours if it worked for us too.

On Thu, Dec 3, 2015, 2:02 AM hightman notifications@github.com wrote:

@scottbouma https://github.com/scottbouma hi, I just recompile libscws, and the problem actually gone. see: http://www.xunsearch.com/scws/demo/v48.php

— Reply to this email directly or view it on GitHub https://github.com/hightman/scws/issues/21#issuecomment-161557264.

[hightman]

Compiled on OS: ubuntu-14.04 (kernel 3.2.0-74) http://www.xunsearch.com/demo/libscws.so.1.1.0

[scottbouma]

Thanks for the fast reply! Unfortunately the link above does not work. I've attached a screenshot of the 404 error page.

We are currently downloading and building on Ubuntu using the following commands:

wget -O - http://www.xunsearch.com/scws/down/scws-1.2.2.tar.bz2 | tar xvjf

cd /scws-1.2.2 && ./configure && make install

Are we using the correct version, and doing the right steps? Did you do anything special or different when you recompiled?

Thanks, Scott

On Thu, Dec 3, 2015 at 5:12 AM, hightman notifications@github.com wrote:

Compiled on OS: ubuntu-14.04 (kernel 3.2.0-74) http://www.xunsearch.com/scws/demo/libscws.so.1.1.0

— Reply to this email directly or view it on GitHub https://github.com/hightman/scws/issues/21#issuecomment-161616952.

[hightman]

I am using scws-1.2.3-dev.

Best Regards

hightman/海鳗

---

微信/微博：hightman Github：https://github.com/hightman

在 2015年12月4日，上午12:44，scottbouma notifications@github.com 写道：

Thanks for the fast reply! Unfortunately the link above does not work. I've attached a screenshot of the 404 error page.

We are currently downloading and building on Ubuntu using the following commands:

wget -O - http://www.xunsearch.com/scws/down/scws-1.2.2.tar.bz2 | tar xvjf

cd /scws-1.2.2 && ./configure && make install

Are we using the correct version, and doing the right steps? Did you do anything special or different when you recompiled?

Thanks, Scott

On Thu, Dec 3, 2015 at 5:12 AM, hightman notifications@github.com wrote:

Compiled on OS: ubuntu-14.04 (kernel 3.2.0-74) http://www.xunsearch.com/scws/demo/libscws.so.1.1.0

— Reply to this email directly or view it on GitHub https://github.com/hightman/scws/issues/21#issuecomment-161616952.

— Reply to this email directly or view it on GitHub https://github.com/hightman/scws/issues/21#issuecomment-161710756.

[hightman]

Now I released 1.2.3, you can try it again...

[scottbouma]

Great, thanks for the update.

On Wed, Feb 24, 2016, 6:24 PM hightman notifications@github.com wrote:

Now I released 1.2.3, you can try it again...

— Reply to this email directly or view it on GitHub https://github.com/hightman/scws/issues/21#issuecomment-188546913.