GroupBy / Filtering out Duplicates

[chiplay]

As the title suggest - do you have any thoughts on how Obscura might be extended to allow for those two use cases:

1. GroupBy Proxies - Grouping collections into groups of collections per an attribute value - eg. all delivery models with the same 'date' attribute get placed into a 'collection' attribute on a new model, that's part of the proxy collection. This works great with Marionette where a collection view can use another collection view for it's itemView.
2. Filtering out Duplicates - is it possible to use filterby (or a variation) to filter out all duplicates from a collection based on an attribute value

Thanks!

[jmorrell]

Hmm... I can definitely see why you might want to this, and I think Obscura can help, if perhaps not provide the full solution. I'll play around with these and see what I can come up with, though I probably won't find time until after new years.

For 2, would this be the desired behavior?

• You decide to filter out a collection, removing any models that have the same value for a key, let's say 'a'
• You add three models to the original collection (not the proxy) in this order

{ a: 1, b: 1 }
{ a: 1, b: 2 }
{ a: 1, b: 3 }

• The proxied collection now looks like:

{ a: 1, b: 1 }

• We remove the first object: { a: 1, b: 1 } from the original array, so the proxy uses the next one that was added

{ a: 1, b: 2 }

[chiplay]

That would be amazing - thanks for your openness to new features!

We are using Marionette with Backbone.Stickit to power all of templates with model bindings and Backbone.Associations to handle our deeply nested model / collection object graph. This combo works really well with Obscura as it allows us to keep the original connections to deeply nested models, instead of having them recreated with new cids after filtering into new collection instance.

The way I'm specifically using the groupBy is as follows:

var groupedCollection = this.collection.groupBy(function(model) {
  return model.get('date');
});

var collectionGroups = _.map(groupedCollection, function(models, date) {
  return new Backbone.Model({ date: date, models: models });
});
this.collection = new Backbone.Collection(collectionGroups);

The final collection is part of a Marionette CompositeView, that has a nested CompositeView for its ItemView. That nested CompositeView has a template binding for the 'date' header and a container for the individual ItemViews that are part of the group. It would be pretty nifty to have something like this:

var proxy = new Obscura(this.collection);
proxy.setGroup('date');

Where proxy would become a collection of models that have two attributes - 'date' (the groupBy key) and 'models' (this proxied group of models for that key)

As for case 2, I think your spot on. The only other feature I could see that would be useful would be allowing the sort to correctly interface with the filter, so that in your example, if the sort was set to 'b' and 'desc', the proxied collection would show { a: 1, b: 3 }.

Let me know how I can be of any help! I'd love to help collaborate on this library in the future as I'm sure we'll be using it extensively for our current project.

[jmorrell]

Haven't forgotten about this btw. Just not going to have any spare cycles until Sunday.

[chiplay]

No worries - thanks again!

[jmorrell]

Okay, I don't think 2 is possible at the moment due to the cache, which assumes that a given model will always return the same response unless either the filter or the model has changed. In this case that would not be the case.

I'm starting to suspect that the cache might have been a premature optimization. JS might be fast enough these days to just re-filter the entire collection every time any model changes, but I need to think about this.

re: having a grouped-by proxy. Here's a first sketch at what that interface might look like as it's own module. I could then pull it in as another transform in Obscura, sandwiched between the filtering and sorting transforms. Any feedback on the proposed API would be helpful.

var collection = new Backbone.Collection([ /* ... */ ]);
var grouped = new Grouped(collection);

// Give it just a key, let it group by value alone
grouped.setGrouping('a');

// Give it a function that accepts a model and returns something
// that can be used as a key.
//
// We can't use groupBy here because that's already in-use by 
// Backbone.Collection :(
grouped.setGroupBy(function(model) {
  return getKey(model);
});

// Clear grouping
grouped.setGroupBy(null);

// or this way
grouped.clearGroupBy();

// By default it creates Backbone.Collections as the sub-container, 
// but let us override this by passing in a different constructor.
var grouped = new Grouped(collection, { type: Obscura });

// Will likely need a way to get a key for any given position?
// If no grouping is set, this will return null.
grouped.getKeyForIndex(i);

// Ex: 
var collection = new Backbone.Collection([
  { id: 1, a: 1, b: 1 },
  { id: 2, a: 1, b: 1 },
  { id: 3, a: 2, b: 1 },
  { id: 4, a: 2, b: 1 },
  { id: 5, a: 3, b: 1 }
]);

var grouped = new Grouped(collection);

grouped.setGroupBy('a');

console.log(grouped.length); // 3

console.log(grouped.getKeyForIndex(0)); // 1
console.log(grouped.getKeyForIndex(1)); // 2
console.log(grouped.getKeyForIndex(2)); // 3

console.log(grouped[0] instanceof Backbone.Collection); // true

console.log(grouped[0].pluck('id')); // [1, 2]
console.log(grouped[1].pluck('id')); // [3, 4]
console.log(grouped[2].pluck('id')); // [5]

grouped.setGroupBy('b');

console.log(grouped.length); // 1

// Need to nail down how this behaves as the underlying collection changes.

[jmorrell]

Created repo that holds the skeleton for this at: https://github.com/jmorrell/backbone-grouped-collection

[chiplay]

Thanks @jmorrell - I'll check it out soon as I have some margin. Thanks again for your work on this great library.

cc @jeffthink

[jmorrell]

Closing because I don't think I'm ever going to have time to get around to this.

After some thinking I don't want to add this functionality to Obscura. It expands the scope too much and would be confusing. Suddenly after grouping your sort function is not acting upon collections instead of models? The proxy idea is strange enough at time without that additional mental load.

I think the repo I started above could work well paired with Obscura (grouped collection could be passed as input to obscura, or vice versa), but I'm unlikely to write it myself, though I'd likely jump in if someone else started working on it.