dishmint / LexicalCases

Extract substrings matching a lexical pattern
https://www.paclets.com/FaizonZaman/LexicalCases
MIT License
2 stars 0 forks source link

SearchIndex example no longer works #295

Closed dishmint closed 7 months ago

dishmint commented 7 months ago
index = CreateSearchIndex["ExampleData/Text"]
indexResults = 
  LexicalCases[index -> All, WordToken[1] ~~ TextType["Verb"]];

This example returns 0 matches, even though the expanded pattern looks correct.


It's another case of extra whitespace. While I cleaned up boundaries for sequences of the same type. I did not handle sequences of different types. For example Word ~~ Verb

In[19]:= ExpandPattern["to be", WordToken[1] ~~ TextType["Verb"]]

Out[19]= WordBoundary ~~ 
 WordCharacter .. ~~ WordBoundary ~~ "  " ~~ WordBoundary ~~ 
 Alternatives["be"] ~~ WordBoundary

In[24]:= StringMatchQ["to be", 
 WordBoundary ~~ WordCharacter .. ~~ WordBoundary ~~ " " ~~ 
  WordBoundary ~~ Alternatives["be"] ~~ WordBoundary]

Out[24]= True

Screenshot 2024-01-27 at 4 56 51 AM

To be fixed in #276