search

gbhl / bhl-europe

Biodiversity Heritage Library Europe
http://www.bhl-europe.eu/
15 stars 2 forks source link

1.1.6 - Search with special characters (Simple Search) #38

Closed janahoffmann closed 13 years ago

janahoffmann commented 13 years ago

Updated description: Special characters, not diacritics, shall be treated as regular characters. Some examples: !"§$%&/()=?+*#'-.,_:;^°~<>| Special characters shall not interfere with the system.

Previous description: Search with diacritic marks, e.g. ´, `, ^

Related bug #197

akohlbecker commented 13 years ago

A diacritical mark is an ancillary glyph added to a letter, theses are only a limited subset of the special characters. The title of this ticket requests for the ability to search using special characters. What are special characters, unfortunately the only really comprehensive wikipedia article this topic is written in German: http://de.wikipedia.org/wiki/Sonderzeichen

Some examples: !"§$%&/()=?+*#'-.,_:;^°~<>|

janahoffmann commented 13 years ago

ready for testing

JiriFrank commented 13 years ago

This is a simple search option.

Bug: Search with special words is working but in the results are also these words without diacritic or just part of that word with complete different meaning. In that case the result is not what the user expected.

Is there an option search the exact word, for example “Král“(King)? In this case I have 10 books in result, but none of them includes “Král“, but just Kral with different meaning.

JiriFrank commented 13 years ago

Testing of feature 1.1.6 - Search with special characters

Feature description from the Catalogue of user requirements:

Search with diacritic marks, e.g. ´, `, ^

Testers: @AntonioGVH @fwelter @heimor @janahoffmann @JFTester @JiriFrank @LarissaS

JiriFrank commented 13 years ago

Testing of feature 1.1.6 - Search with special characters

Feature description from the Catalogue of user requirements:

Search with diacritic marks, e.g. ´, `, ^

Testers: @fwelter

akohlbecker commented 13 years ago

Hi Jiri, the original description from the catalogue of user requirements is misleading - see my comment ( https://github.com/bhle/bhle/issues/38#issuecomment-1431016 ). Jana, could you please fix the description in the catalogue of user requirements?

JiriFrank commented 13 years ago

Hi Adreas,

thank you. I will test also the special characters which you mentioned in your comment. Anyway new bug with diacritic due to this issue appeared, see my first comment.

Cheers, Jiri

Test results

Special character as !?()/+- reporting error and need to be used in combination with quotes. The error screen need to be replaced for example by information that it is necessary to use the quotes or give automatically no results. Others special characters are working properly.

LarissaS commented 13 years ago

Testing of feature 1.1.6 - Search with special characters

Searching for “N’Zau” and “Nzau” gives same result (2 records from RMCA)

Searching for “crâne” gives 5 records and includes as well French “crâne” (scull) as the name Crane

Searching for "tête" and "tete" give same result - 13 records all in French

So, this function looks ok !

-----Original Message----- From: JiriFrank [mailto:reply@reply.github.com] Sent: 1 августа 2011 г. 10:23 To: Smirnova Larissa Subject: Re: [bhle] 1.1.6 - Search with special characters (#38)

Testing of feature 1.1.6 - Search with special characters

Feature description from the Catalogue of user requirements:

Search with diacritic marks, e.g. ´, `, ^

Testers: @AntonioGVH @fwelter @heimor @janahoffmann @JFTester @JiriFrank @LarissaS

Reply to this email directly or view it on GitHub: https://github.com/bhle/bhle/issues/38#issuecomment-1698724 ###########################################

This message has been scanned by ICT - Africa Museum

9/8/2011 - Filtered through antispam by ICT

fwelter commented 13 years ago

1.1.6 special characters

,

.

(

"("

")"

&

/

"/"

"

""

"a*

"d'un nouveau genre

"d'un nouveau genre"

"l'afrique"

l'afrique

lafrique

l'''''afrique

"de l'afrique"

"de lafrique"

"l'afrique maroc"

"lafrique maroc"

" lafrique maroc"

"de lafrique maroc"

"de l'afrique maroc"

"(heteroptera,"

basalt?

"basalt?"

the program largely ignored special characters, but did not skip them totally. Exact finding of special characters was not possible. in some cases not putting the ' yielded curiously different results.

22eme

22em

1 - the result lists generally displayed titles where none of the words were contained in the title, author or year, but in the text of the abstract and notes of the item page.

2 - single journal volumes were displayed for serial runs, they blocked all intelligent search attempts and spammed the results list with over and over the same journal title.

Francisco

F. Welter-Schultes Zoologisches Institut, Berliner Str. 28, D-37073 Goettingen Phone +49 551 395536, Fax +49 551 395579 http://www.gwdg.de/~fwelter http://www.animalbase.org

akohlbecker commented 13 years ago

searching with diacritic marks works by now, but searching for special characters is not really an option since we are using. Solr usually only indexes letters and numbers. special characters are always ignored except those which are part of the query syntax, these have to be stripped out: #247

So I am closing this issue as fixed.

chris-sleep commented 12 years ago

special characters are now stripped from the index and searching

JiriFrank commented 12 years ago

1.1.6 - Search with special characters (Simple Search)

COR number: 1.1.6 Testing platform: http://bhl-test.nhm.ac.uk/portal/ Function: Simple search, Advanced search

Description:

Special characters, not diacritics, shall be treated as regular characters. Some examples: !"§$%&/()=?+*#'-.,_:;^°~<>|

Special characters shall not interfere with the system.

@AnneSch @AntonioGVH @fwelter @grahamhrbge1670 @heimor @HenningScholz @JFTester @JiriFrank @LarissaS @RalfH

RalfH commented 12 years ago

There are certain problems.

Hyphens work well; results are even given if you omit them. That's fine.

When searching "(Anthropomorphae):" no results are given but it may refer to a general problem because even "Anthropomorphae" is not found by simple search.

By the way: If you start typing with an ( in simple search you receive the attached note. It may be of interest for you.

-----Ursprngliche Nachricht----- Von: JiriFrank [mailto:reply@reply.github.com] Gesendet: Dienstag, 17. Januar 2012 13:39 An: Hand, Ralf Betreff: Re: [bhle] 1.1.6 - Search with special characters (Simple Search) (#38)

1.1.6 - Search with special characters (Simple Search)

COR number: 1.1.6 Testing platform: http://bhl-test.nhm.ac.uk/portal/ Function: Simple search, Advanced search

Description:

Special characters, not diacritics, shall be treated as regular characters. Some examples: !"$%&/()=?+*#'-.,_:;^~<>|

Special characters shall not interfere with the system.

@AnneSch @AntonioGVH @fwelter @grahamhrbge1670 @heimor @HenningScholz @JFTester @JiriFrank @LarissaS @RalfH

Reply to this email directly or view it on GitHub: https://github.com/bhle/bhle/issues/38#issuecomment-3527214

AnneSch commented 12 years ago

Works fine with accents. Only a little remark: I looked for "L'etat independant", but it was not found, because it was not written like that, but "Letat". Same with "dequateurville", which is originally "d'Equateurville".

-----Ursprüngliche Nachricht----- Von: JiriFrank [mailto:reply@reply.github.com] Gesendet: Dienstag, 17. Januar 2012 13:39 An: Scholz, Annemarie Betreff: Re: [bhle] 1.1.6 - Search with special characters (Simple Search) (#38)

1.1.6 - Search with special characters (Simple Search)

COR number: 1.1.6 Testing platform: http://bhl-test.nhm.ac.uk/portal/ Function: Simple search, Advanced search

Description:

Special characters, not diacritics, shall be treated as regular characters. Some examples: !"§$%&/()=?+*#'-.,_:;^°~<>|

Special characters shall not interfere with the system.

@AnneSch @AntonioGVH @fwelter @grahamhrbge1670 @heimor @HenningScholz @JFTester @JiriFrank @LarissaS @RalfH

Reply to this email directly or view it on GitHub: https://github.com/bhle/bhle/issues/38#issuecomment-3527214

grahamhrbge1670 commented 12 years ago

Have all the apostrophes, for example, been removed from the system at the data ingest stage. I tried a similar search to the one Anne did, using l'Atlantique as an example. A simple search set to Title field for "l'Atlantique" retrieves no results, whilst a simple search set to Title field for "lAtlantique" retrieves one result... This is confusing for me.

Graham

Just noted Andreas' comment above, about the way Solr indexes. Important to instruct users that they should remove special characters from search terms that contain them. Not something someone would do intuitively.

HenningScholz commented 12 years ago

Not working in all cases: searching for "(Anthropomorphae): " gives not result and I can not even find "Anthropomorphae". Entering in the simple search gives sometimes an error message in a pop up window (Ajax http error code 500). "botaniker," is working.

Von: JiriFrank [mailto:reply@reply.github.com] Gesendet: Di 17.01.2012 13:38 An: Scholz, Henning Betreff: Re: [bhle] 1.1.6 - Search with special characters (Simple Search) (#38)

1.1.6 - Search with special characters (Simple Search)

COR number: 1.1.6 Testing platform: http://bhl-test.nhm.ac.uk/portal/ Function: Simple search, Advanced search

Description:

Special characters, not diacritics, shall be treated as regular characters. Some examples: !"$%&/()=?+*#'-.,_:;^~<>|

Special characters shall not interfere with the system.

@AnneSch @AntonioGVH @fwelter @grahamhrbge1670 @heimor @HenningScholz @JFTester @JiriFrank @LarissaS @RalfH

Reply to this email directly or view it on GitHub: https://github.com/bhle/bhle/issues/38#issuecomment-3527214