Refresh clears ListBox selection

Hi, I'm trying to decouple my VMs from WPF libraries and DynamicData seems to be a great way to do that. I've managed to figure out most of what I was doing before using your snippets and examples, but I've hit something I don't know how to solve.

I had two observable collections, one with all the items and another with the items selected in that collection (bound using an attached behaviour on ListView, which worked fine before). I also have a search text filter for these items, but I want to keep the selected items in the list even if they don't match the search text, so my filter method takes both of those things into account.

I have used .AutoRefreshOnObservable(_ => ...) to listen to changes to the search text using this.WhenValueChanged(...) and the selected items observable using .ToObservable(), which is updating the filter correctly, but is also deselecting all my items when it refreshes.

I found two stackoverflow questions which seemed relevant to my issue:

Clearing on sort which you already responded to, but isn't the accepted answer (it seems like I want a move rather than remove+insert to keep the item in the selection list)
Clearing on add which you also responded to, but resetThreshold: int.MaxValue didn't work for me.

I'd really like to be able to continue using this way of presenting my data to the View, but I can't if the collection clearing on changes is unavoidable, so any assistance would be appreciated, thanks.

Selection is a genuine pain. I have often resorted to a little custom code around the bind to keep items selected. I can't remember exactly what I did however as it's been about 5 years since I did any UI work

So, your goal for the ListBox is to have items be automatically selected or not depending on whether they match the search text? And you need to plumb that info back into ListBox.SelectedItems somehow?

Can you provide any code samples/snippets for what you have now?

Selection is a genuine pain. I have often resorted to a little custom code around the bind to keep items selected. I can't remember exactly what I did however as it's been about 5 years since I did any UI work

I'm glad it's not just something I've done wrong, but I hope you can remember what it was you did, because I'm not sure how I can prevent removals when they could be genuine deselections.

So, your goal for the ListBox is to have items be automatically selected or not depending on whether they match the search text? And you need to plumb that info back into ListBox.SelectedItems somehow?

The search text doesn't (shouldn't) change which items are selected, but items which are already selected won't be deselected by the search filter. Basically the search filter is: item.Matches(searchText) || SelectedItems.Contains(item);

I can't provide the exact code unfortunately.

Well, I can demonstrate what I would do to achieve this...

public record MyListItem
{
    public required int Id { get; init; }
    public required string Text { get; init; }
}

public class MyListItemModel
    : INotifyPropertyChanged
{
    public static MyListItemModel Create(
        int                                 itemId,
        IObservableCache<MyListItem, int>   itemsSource);

    public int Id { get; }
    public string Text { get; private set; }
    public bool IsSelected { get; set; }
}

...

using var itemsSource = new SourceCache<MyListItem, int>(static item => item.Id);

using var searchText = new BehaviorSubject<string>(string.Empty);

var itemModelsCache = = itemsSource
    .Connect()
    .DistinctValues(static item => item.Id)
    .Transform(itemId => MyItemViewModel.Create(itemId, itemsSource))
    .AsObservableCache();

using var itemModelsSubscription = itemModelsCache
    .Connect()
    .Sort(...)
    .Bind(out var itemModels)
    .Subscribe();

var filteredItems = itemModeslCache
    .Connect()
    .Filter(searchText
        .DistinctUntilChanged()
        .Select(static searchText => new Func<MyListItemModel, bool>(itemModel
                => itemModel.Matches(searchText) || itemModel.IsSelected)));

Well, I can demonstrate what I would do to achieve this...

Yeah, I had been thinking about wrapping my items in a class Selectable<T> which tracks the selection per item but that would require custom styling for the ListBoxItem/ListViewItem control to bind the IsSelected property and down-select the DataContext for the presenter to find the right DataTemplate. I also wasn't completely sure it would solve the problem if the items still got deselected by the ListBox/ListView when the bound observable collection refreshed.

Your example is adding IsSelected to the same VM object but seems like it would amount to the same thing. I'm just not sure what .DistinctUntilChanged() adds to the search.

I think I might need to look at the reference source to see how Selector handles maintaining the selection when the source updates.

UPDATE:

I figured I would listen to the collectionchanged events on my bound r/o collection and each time it refreshes there are a bunch of Replaces. I tried using BindingOptions and UseReplaceForUpdates: false, but it was still doing it, and when I looked at the stack trace in ObservableCollectionAdaptor<T>.Adapt() there's a TODO: pass in allowReplace ..., so I'm guessing that's not implemented? I don't think it would solve my problem anyway, but I'm wondering if I want a Replace change event at all if the object is the same?

I'm going to try filtering out those change events with a wrapper ObservableCollection to see what happens.

So wrapping my bound ReadOnlyObservableCollection<T> in this seems to fix my issue, but it's a hack:

public class ReadOnlyObservableCollectionNoReplaceSameObject<T> : IReadOnlyList<T>, INotifyCollectionChanged
{
    private readonly IReadOnlyList<T> _items;
    private readonly INotifyCollectionChanged _ncc;
    private readonly Dictionary<NotifyCollectionChangedEventHandler, NotifyCollectionChangedEventHandler> _filterLookup = new();

    public ObservableCollectionNoReplace(ReadOnlyObservableCollection<T> collection)
    {
        _items = collection;
        _ncc = collection;
    }

    /// <inheritdoc />
    public int Count => _items.Count;

    /// <inheritdoc />
    public T this[int index] => _items[index];

    /// <inheritdoc />
    public IEnumerator<T> GetEnumerator() => _items.GetEnumerator();

    /// <inheritdoc />
    IEnumerator IEnumerable.GetEnumerator() => ((IEnumerable)_items).GetEnumerator();

    /// <inheritdoc />
    public event NotifyCollectionChangedEventHandler CollectionChanged
    {
        add => _ncc.CollectionChanged += FilterOutReplace(value);
        remove => _ncc.CollectionChanged -= GetFilter(value);
    }

    private NotifyCollectionChangedEventHandler GetFilter(NotifyCollectionChangedEventHandler value)
    {
        return _filterLookup.TryGetValue(value, out var handler) ? handler : null;
    }

    private NotifyCollectionChangedEventHandler FilterOutReplace(NotifyCollectionChangedEventHandler value)
    {
        NotifyCollectionChangedEventHandler handler = (o, e) =>
        {
            if (e.Action != NotifyCollectionChangedAction.Replace || e.NewItems[0] != e.OldItems[0])
            {
                value(o, e);
            }
        };

        _filterLookup.Add(value, handler);
        return handler;
    }
}

Could you add an option to the BindingOptions to exclude replacing the same item (or just exclude it if it never makes sense)?

I do have one remaining issue, which is that .AutoRefreshOnObservable(_ => this.WhenValueChanged(x => x.SearchText)) triggers a refresh as expected, but .AutoRefreshOnObservable(_ => SelectedItems.ToObservable()) doesn't. Am I doing something wrong?

but that would require custom styling for the ListBoxItem/ListViewItem control to bind the IsSelected property and down-select the DataContext for the presenter to find the right DataTemplate.

To me, this doesn't sound so much like a problem as "the normal work you have to do to build a View layer". I'm also not sure what you mean by "down-select" here.

Even if the VM is structured like you describe, as a Selectable<T> class that has .IsSelected and .Data (or whatever it might be called) properties, the View-layer binding for that, which leverages existing data templates for T, is pretty straightforward.

<ListBox>
    <ListBox.ItemContainerStyle>
        <Style TargetType="ListBoxItem}">
            <Setter Property="IsSelected" Value="{Binding IsSelected}"/>
        </Style>
    </ListBox.ItemContainerStyle>
    <ListBox.ItemTemplate>
        <DataTemplate>
            <ContentControl Content="{Binding Data}"/>
        <DataTemplate>
    </ListBox.ItemTemplate>
</ListBox>

I'm just not sure what .DistinctUntilChanged() adds to the search.

.DistinctUntilChanged() isn't really relevant to this issue at all, it's just a normal clause to add to a value stream that triggers a lot of downstream processing, in order to prevent redundant re-processing.

I tried using BindingOptions and UseReplaceForUpdates: false

So, you're saying that your source stream is issuing updates instead of refreshes, when the item hasn't actually changed, only its state has? That there would be your core problem, stemming from something you're not doing properly in the creation of these objects, or in your composition of operators, or from a bug in DD.

Have a peak at what the changeset stream looks like going INTO the binding. From what you describe of your scenario, you should only be seeing Refresh changes, not Updates. If you're seeing Updates it's probably that your stream has a .Transform() that is reconstructing and replacing items in the stream, when it shouldn't.

if (e.Action != NotifyCollectionChangedAction.Replace || e.NewItems[0] != e.OldItems[0])
{
   value(o, e);
}

This is a convincing proof of what the problem is, but it's not here, it's upstream. See above.

Could you add an option to the BindingOptions to exclude replacing the same item (or just exclude it if it never makes sense)?

Wouldn't have any practical use, as this kinda thing shouldn't happen in the first place. See above.

.AutoRefreshOnObservable(_ => this.WhenValueChanged(x => x.SearchText)) triggers a refresh as expected, but
.AutoRefreshOnObservable(_ => SelectedItems.ToObservable())

You say the second one doesn't trigger a Refresh. Does that mean it triggers nothing at all, or triggers something other than a Refresh?

Also, I HIGHLY recommend against doing either of these, it's going to murder your performance. This will cause a separate changeset, each with a single Refresh changed, to be emitted for every item in your collection, whenever the trigger occurs, instead of a single changeset with all of the Refreshes in it at once.

What is it that you're trying to achieve with these two? Filtering? If so, change-detection for filtering criteria should be handled within the .Filter() clause itself. E.G...

items
    .Filter(
        predicateChanged:   Observable.Return<Func<MyListItem, bool>>(
            item => item.Name.Matches(SearchText)
                || SelectedItems.Contains(item)),
        reapplyFilter:      Observable.Merge(
            this.WhenValueChanged(x => x.SearchText)
                .Select(static _ => Unit.Default),
            SelectedItems
                .ToObservable()
                .Select(static _ => Unit.Default));

(This is much more verbose than it should be, BTW, due to lack of enough variety in overloads for this operator. That's on my to-do list).

Even if the VM is structured like you describe, as a Selectable<T> class that has .IsSelected and .Data (or whatever it might be called) properties, the View-layer binding for that, which leverages existing data templates for T, is pretty straightforward.

Yes, that's basically what I meant, and no it's not that complicated, but it's annoying to have to add that to a view which should handle selection without it to solve a bug. I think it's a reasonable way to handling selection if you don't want to make SelectedItems bindable.

Have a peak at what the changeset stream looks like going INTO the binding. From what you describe of your scenario, you should only be seeing Refresh changes, not Updates. If you're seeing Updates it's probably that your stream has a .Transform() that is reconstructing and replacing items in the stream, when it shouldn't.

I wasn't sure how to peek at what's going on inside the pipe, do I just make a new extension method and observer/observee that does nothing to put in the chain and put some breakpoints there, or is there some way to inspect them in the DD API already?

Also, I HIGHLY recommend against doing either of these, it's going to murder your performance. This will cause a separate changeset, each with a single Refresh changed, to be emitted for every item in your collection, whenever the trigger occurs, instead of a single changeset with all of the Refreshes in it at once.

What is it that you're trying to achieve with these two? Filtering? If so, change-detection for filtering criteria should be handled within the .Filter() clause itself. E.G...
items
    .Filter(
        predicateChanged:   Observable.Return<Func<MyListItem, bool>>(
            item => item.Name.Matches(SearchText)
                || SelectedItems.Contains(item)),
        reapplyFilter:      Observable.Merge(
            this.WhenValueChanged(x => x.SearchText)
                .Select(static _ => Unit.Default),
            SelectedItems
                .ToObservable()
                .Select(static _ => Unit.Default));
(This is much more verbose than it should be, BTW, due to lack of enough variety in overloads for this operator. That's on my to-do list).

I'm quite happy to do this entirely differently as my understanding was that it was refreshing the entire pipeline just to refresh the filter and that seemed overkill.

In another VM I tried to create an observable filter from multiple owner VM properties changing using: .WhenAnyPropertyChanged(nameof(A), nameof(B)).Select(_ => (Func<T, bool>)DoFilter) but it didn't work. It did work when I used: .WhenChanges(x => x.A, x => x.B, (x, _, _) => (Func<T, bool>)DoFilter)), but having to cast a method to its own type annoyed me so I used .AutoRefreshOnObservable(_ x.WhenValueChanged(...)) as described previously, which seemed to work correctly except for the selection.

I couldn't observe the selection collection changing in the property changed ones anyway, so it seemed a better fit, but the collection isn't triggering a refresh.

However, I can't find the overload you're talking about, did you mean this?

.Filter(Observable.Merge(
        this.WhenPropertyChanged(x => x.SearchText).Select(_ => Unit.Default), 
        SelectedItems.ToObservable().Select(_ => Unit.Default))
    .Select(_ => (Func<T, bool>)DoFilter))

I tried it and it avoids the unwanted downsteam Replaces, but it still wasn't updating on the selection changing. I then changed ToObservable() to ToObservableChangeSet() and it seems to be working now 👍

Now I understand it better and it works, I would recommend two additonal extension methods

.AsDefault() for .Select(_ => Unit.Default)
IObservable<IChangeSet<T>> Filter<T>(this IObservable<IChangeSet<T>> source, Predicate<T> filter, IObservable<Unit> refreshOn, params IObservable<Unit>[] more) => source.Filter(Observable.Merge(more).Merge(refreshOn).Select(_ => predicate)); (use two parameters to avoid conflicting with the predicate only overload, also ensures at least one refresh source).

is there some way to inspect them in the DD API already?

I'm just talking about looking at the changesets passing between operators.

.Do(changeSet => { /* breakpoint here */ })

However, I can't find the overload you're talking about, did you mean this?

Your version doesn't look like it would be meaningfully different. It makes more sense to read, but it also generates a new redundant Func<> allocation each time the search text or set of selections changes. My version would avoid that.

The overload I was referring to is...

public static IObservable<IChangeSet<TObject, TKey>> Filter<TObject, TKey>(
        this    IObservable<IChangeSet<TObject, TKey>>  source,
                IObservable<Func<TObject, bool>>        predicateChanged,
                IObservable<Unit>                       reapplyFilter,
                bool                                    suppressEmptyChangeSets = true)
    where TObject : notnull
    where TKey : notnull

I would recommend two additonal extension methods

.AsDefault() (or .SelectUnit() as I usually call it) is very much something I've used in my own projects, and I think there's a version internally in DD as well. But I'm hesitant to make it a public part of DD, as it really doesn't have anything to do with DD. If anything, it ought to be a part of System.Reactive itself, and I've considered submitting a PR to see if they'd add it. But at the same time, it's such a simple extension to make for anyone who wants it, that it's a pretty low priority.

That overload is basically what I was looking for, and was kinda surprised it doesn't exist. Although, I definitely wouldn't put the params arg in there. The only thing the operator would do is Observable.Merge() them together, and if we force the consumer to do that before calling the operator, we save on the array allocation for the params arg.

However, there's also a new .Filter() overload I'm working on right now that makes the proposed one a bit superfluous. To an extent, we don't want to go TOO far into providing overloads to accommodate every possible scenario. Limiting operator overloads can help promote good practices in how people consume the library. Still, I'll put it on our proverbial list to consider.

I'm not quite sure if what you've got now is the most effective solution, but I'm glad you're working regardless. Let us know if you need any further advice.

RolandPheasant / DynamicData.Snippets

Refresh clears ListBox selection #9