Search Return Nothing

[hieuuk]

Hi, I'm only recently starting using Redis OM dotnet for my projects. Love the way it's coded, thanks for the great work.

I hit some issues though with query data out. I have added the full test code bellow. And you can also view it on the public github: https://github.com/hieuuk/TestRedisOm.git

This is the index inffo I get from RedisInsight

I must have done something wrong as a simple query doesn't return anything

var result = await midx
     .Where(m => m.Actors.Any(a => a.Id == 1))
     .ToListAsync();

I also wonder if it's possible to search for part of the word and not a full word or not. So this return result

// WORKING
var result = await midx
     .Where(m => m.SingleActor.Name == "Tom")
     .ToListAsync();

But this query isn't

var result = await midx
     .Where(m => m.SingleActor.Name.Contains("om"))
     .ToListAsync();

Thank you!

Full code

using Redis.OM;
using Redis.OM.Searching;
using Redis.OM.Modeling;

var actor1 = new Actor
{
    Id = 1,
    Name = "Brad Pitt"
};
var actor2 = new Actor
{
    Id = 2,
    Name = "Tom Cruise"
};

var movie1 = new Movie
{
    Id = 1,
    Title = "Mission: Impossible - Rogue Nation",
    SingleActor = actor2,
    Actors = new List<Actor>()
    {
        actor2
    }
};

var movie2 = new Movie
{
    Id = 2,
    Title = "Interview With The Vampire",
    SingleActor = actor1,
    Actors = new List<Actor>
    {
        actor1,
        actor2
    }
};

var provider = new RedisConnectionProvider("http://localhost:6379");
await provider.Connection.DropIndexAsync(typeof(Movie));
await provider.Connection.CreateIndexAsync(typeof(Movie));
var midx = (RedisCollection<Movie>)provider.RedisCollection<Movie>();

await midx.InsertAsync(movie1);
await midx.InsertAsync(movie2);

// WORKING
//var result = await midx
//     .Where(m => m.SingleActor.Name == "Tom")
//     .ToListAsync();

// RETURN NOTHING
var result = await midx
     .Where(m => m.SingleActor.Name.Contains("om"))
     .ToListAsync();

// RETURN NOTHING
//var result = await midx
//     .Where(m => m.Title.Contains("Interview"))
//     .ToListAsync();

// ERROR: Syntax error at offset 19 near With
//var result = await midx
//     .Where(m => m.Title == "Interview With The Vampire")
//     .ToListAsync();

// RETURN NOTHING
//var result = await midx
//     .Where(m => m.Title == "Interview")
//     .ToListAsync();

// RETURN NOTHING
//var result = await midx
//     .Where(m => m.Actors.Any(a => a.Id == 1))
//     .ToListAsync();

// RETURN NOTHING
//var result = await midx
//     .Where(m => m.Actors.Any(a => a.Name == "Tom Cruise"))
//     .ToListAsync();

// RETURN NOTHING
//var result = await midx
//     .Where(m => m.Title.Contains("pir"))
//     .ToListAsync();

// RETURN NOTHING
//var result = await midx
//     .Where(m => m.Actors.Any(a => a.Name == "Tom"))
//     .ToListAsync();

foreach (var item in result)
{
    Console.WriteLine($"{item.Id} - {item.Title}");
    foreach (var ma in item.Actors)
    {
        Console.WriteLine($"Actor {ma.Id} - {ma.Name}");
    }
}

Console.WriteLine("Completed");

[Document(StorageType = StorageType.Json, Prefixes = new string[] { "mov" }, IndexName = "midx")]
public class Movie
{
    [RedisIdField]
    public int Id { get; set; }
    [Searchable]
    public string Title { get; set; }
    [Indexed(CascadeDepth = 1)]
    public Actor SingleActor { get; set; }
    [Indexed(JsonPath = "$.Id")]
    [Searchable(JsonPath = "$.Name")]
    public List<Actor> Actors { get; set; } 
}

public class Actor
{
    [RedisIdField]
    [Indexed]
    public int Id { get; set; }
    [Searchable]
    public string Name { get; set; }
}

[slorello89]

Hi @hieuuk - looks like you've run into a couple of quirks in Redis OM, mostly around embedded documents within arrays, and fundamentally how full-text works in RediSearch. Thanks for sharing all your code, that really helped narrow this all down.

1. var result = await midx .Where(m => m.SingleActor.Name == "Tom") .ToListAsync(); - unfortunately RediSearch doesn't quite work like this. You can do prefix matches with a glob pattern (I think you might be able to do suffix matches as well), but just calling contains on a string in here won't do the trick. You can however set Name to Aggregatable in the Actor class, and then use the following aggregation:

result = (await aggregations.Load(x=>x.RecordShell.SingleActor.Name).Apply(x => x.RecordShell.SingleActor.Name.Contains("om"), "doesContain")
    .Filter(x => x["doesContain"] == 1).LoadAll().ToListAsync()).Select(x=>x.Hydrate()).ToList();

which will get you what you're looking for.

2. var result = await midx.Where(m => m.Title == "Interview With The Vampire").ToListAsync(); - This is another odd quirk, you'll notice if you look at our forthcoming 0.2.2 release, there is a Stopwords configuration item from #168 - A stopword in RediSearch can cause syntax errors. The in Interview With The Vampire, is unfortunately a stopword. The good news is that you'll be able to disable them pretty soon.

3. var result = await midx.Where(m => m.Title == "Interview").ToListAsync(); - This one's a bit odd, if you noticed in the README - it does somewhat intentionally call out that you can index "strings and string-like value types" (that's bools/enums/Ulids/Guids) - numerics/geo/full-text-search and the like can unfortunately NOT be indexed at this time. Interestingly, because the Id in Actor is setup as an integer, Redis OM IS creating an index for it (which is a bug), unfortunately, when you store the JSON with those illegal values for indexing, RediSearch chokes when trying to index it, and ends up not indexing the document at all, hence the "Interview With The Vampire" is not stored in Redis. Change Actor Id to a string and it'll work fine

4. var result = await midx.Where(m => m.Actors.Any(a => a.Id == 1)).ToListAsync(); - see 3, same issue

5. var result = await midx.Where(m => m.Actors.Any(a => a.Name == "Tom Cruise")).ToListAsync(); - see 3 same issue (can Searchable isn't valid on a collection)

6. var result = await midx.Where(m => m.Title.Contains("pir")).ToListAsync(); - same issue as 1

7. var result = await midx.Where(m => m.Actors.Any(a => a.Name == "Tom")).ToListAsync(); - same issue as 3

[hieuuk]

Hi,

Thank you for quick response.

Quick update on this.

1. The solution works. I did read on Redisearch about the same thing. I'm pretty new to RediSearch so I'm not sure what it mean. But using Aggretions working.

2. Although number 2, the problem might not light in the Stopwords though. I have tested by remove the stopword and I still can't get the 2nd movie to index. After a few tests, I realize, if my Actors List has more then 1 item. It won't index. I did change the Id to String for Movie and Actors which not helping in this case. I did a Flushdb just to be sure but still no luck.

I have upload the code to github and will do some more tests on this.

[slorello89]

See point 3 above, because ID is an integer, it breaks when trying to index the multi-value array it gets back when resolving that JSON path. When I changed actor’s ID to a string they indexed for me

[hieuuk]

I did got it working in the end now. It's strange though as before I did set the Actor Id and Movie Id to string and can't get it the 2nd to index even with remove the stop word. Anyways, all working now, thank you very much.

My last issue in the list before trying to roll it out to my main project. If you could help that's great. So with this is working and return 2 movies as expected.

var result = await midx
     .Where(m => m.Actors.Any(a => a.Id == "a2"))
     .ToListAsync();

This query still return nothing

var result = await midx
     .Where(m => m.Actors.Any(a => a.Name == "Tom"))
     .ToListAsync();

What kinda aggreations we can use to find the contains but in the array object, please?

I included the full code just in case:

using Redis.OM;
using Redis.OM.Searching;
using Redis.OM.Aggregation;
using Redis.OM.Modeling;

var actor1 = new Actor
{
    Id = "a1",
    Name = "Brad Pitt"
};
var actor2 = new Actor
{
    Id = "a2",
    Name = "Tom Cruise"
};

var movie1 = new Movie
{
    Id = "m1",
    Title = "Mission: Impossible - Rogue Nation",
    SingleActor = actor2,
    Actors = new List<Actor>()
    {
        actor2
    }
};

var movie2 = new Movie
{
    Id = "m2",
    Title = "Interview With The Vampire",
    SingleActor = actor1,
    Actors = new List<Actor>
    {
        actor1,
        actor2
    }
};

var provider = new RedisConnectionProvider("http://localhost:6379");
//await provider.Connection.DropIndexAsync(typeof(Movie));
//await provider.Connection.CreateIndexAsync(typeof(Movie));
var midx = (RedisCollection<Movie>)provider.RedisCollection<Movie>();

await midx.InsertAsync(movie1);
await midx.InsertAsync(movie2);

// WORKING
//var result = await midx
//     .Where(m => m.SingleActor.Name == "Cruise")
//     .ToListAsync();

// RETURN NOTHING
//var result = await midx
//     .Where(m => m.SingleActor.Name.Contains("om"))
//     .ToListAsync();
// THIS WORKING
//var aggregations = new RedisAggregationSet<Movie>(provider.Connection);
//var result = (await aggregations.Load(x => x.RecordShell.SingleActor.Name).Apply(x => x.RecordShell.SingleActor.Name.Contains("om"), "doesContain")
//    .Filter(x => x["doesContain"] == 1).LoadAll().ToListAsync()).Select(x => x.Hydrate()).ToList();

// RETURN NOTHING
//var result = await midx
//     .Where(m => m.Title == "Interview")
//     .ToListAsync();
//WORKING
//var aggregations = new RedisAggregationSet<Movie>(provider.Connection);
//var result = (await aggregations.Load(x => x.RecordShell.Title).Apply(x => x.RecordShell.Title.Contains("Inter"), "doesContain")
//    .Filter(x => x["doesContain"] == 1).LoadAll().ToListAsync()).Select(x => x.Hydrate()).ToList();

// ERROR: Syntax error at offset 19 near With
//var result = await midx
//     .Where(m => m.Title == "Interview vampire")
//     .ToListAsync();

// WORKING
//var result = await midx
//     .Where(m => m.Title == "Interview")
//     .ToListAsync();

// WORKING
//var result = await midx
//     .Where(m => m.Actors.Any(a => a.Id == "a2"))
//     .ToListAsync();

// RETURN ONLY MOVIE 1
var result = await midx
     .Where(m => m.Actors.Any(a => a.Name == "Tom"))
     .ToListAsync();

//var aggregations = new RedisAggregationSet<Movie>(provider.Connection);
//var result = (await aggregations.Load(x => x.RecordShell.Actors.Name).Apply(x => x.RecordShell.SingleActor.Name.Contains("om"), "doesContain")
//    .Filter(x => x["doesContain"] == 1).LoadAll().ToListAsync()).Select(x => x.Hydrate()).ToList();

foreach (var item in result)
{
    Console.WriteLine($"{item.Id} - {item.Title}");
    foreach (var ma in item.Actors)
    {
        Console.WriteLine($"Actor {ma.Id} - {ma.Name}");
    }
}

Console.WriteLine("Completed");

[Document(StorageType = StorageType.Json, Prefixes = new string[] { "mov" }, IndexName = "midx")]
public class Movie
{
    [RedisIdField]
    [Indexed]
    public string Id { get; set; }
    [Searchable(Aggregatable = true)]
    public string Title { get; set; }
    [Indexed(CascadeDepth = 1)]
    public Actor SingleActor { get; set; }
    [Indexed(JsonPath = "$.Id")]
    [Indexed(JsonPath = "$.Name")]
    public List<Actor> Actors { get; set; } 
}

public class Actor
{
    [RedisIdField]
    [Indexed]
    public string Id { get; set; }
    [Searchable(Aggregatable = true, Sortable = false)]
    public string Name { get; set; }
}

[slorello89]

Hi @hieuuk - so that one's a bit stickier because your searching a collection rather than a single scalar, the Apply functions don't apply cleanly across the whole collection (for now I believe they'll only operate on the first element of the resolved array). Fortunately, since you are working with a prefix, you can drop down to the raw query API and do a prefix search for all the Tom's in redis:

var query = new RedisQuery("midx");
query.QueryText = "@Actors_Name:{Tom*}";
var movies = provider.Connection.Search<Movie>(query);
foreach (var movie in movies.Documents.Values)
{
    Console.WriteLine($"{movie.Title}");
}

[hieuuk]

Thanks. Sorry, had quite a busy week. This looks promising. I will try to get it working.

[slorello89]

Going to close this as it's more or less a duplicate of #264 now