search

mixxxdj / mixxx

Mixxx is Free DJ software that gives you everything you need to perform live mixes.
http://mixxx.org
Other
4.52k stars 1.28k forks source link

Smart crates / save and recall search queries #5575

Open mixxxbot opened 2 years ago

mixxxbot commented 2 years ago

Reported by: rryan Date: 2010-10-15T20:48:14Z Status: Confirmed Importance: Wishlist Launchpad Issue: lp661460 Tags: library

Provide a way to save a current search query as a Smart Playlist/Crate that updates based on that query. Will be more useful once we have search operators. I actually think this only makes sense to do a Smart Crate because a Smart Playlist is essentially a crate.

mixxxbot commented 2 years ago

Commented by: borfo Date: 2013-07-17T16:53:41Z

It would be useful if smart crates ran their queries once at the start of the mixxx session, and cached the results for the duration of the session, rather than running the crate-construction query every time you open the crate - I have a very large music collection, and searches of the whole collection aren't very fast - if these smart crates cached their contents somehow, that would really speed up searches within the crates, since I would only be searching a subset of my library.

mixxxbot commented 2 years ago

Commented by: rryan Date: 2013-07-17T17:12:30Z

I'd suggest something slightly different -- that they are cached but only update when tracks in the Mixxx library change. This way if you had a crate that was like [playcount>5] and you played a track in the same session bumping its playcount from 5 to 6 then it would appear in the crate.

mixxxbot commented 2 years ago

Commented by: borfo Date: 2013-07-28T11:47:40Z

That works for me.

mixxxbot commented 2 years ago

Commented by: daschuer Date: 2013-09-23T06:56:13Z

7179

mixxxbot commented 2 years ago

Commented by: naught101 Date: 2014-01-12T00:26:21Z

Just noting that both Zotero and Thunderbird do this quite well, if anyone is looking for UI inspiration.

Also, it would be really useful to have a couple of default smart crates - particularly "Songs not in a crate", and "Never played songs". They'd be really helpful for organising a library initially.

mixxxbot commented 2 years ago

Commented by: uklotzde Date: 2015-02-16T10:14:04Z

More ideas on (and beyond) the 'smart crates' feature.

Dynamic content

Customizable column configuration

Mixxx query syntax: Add crate containment property

Mixxx query syntax: Add new operators

mixxxbot commented 2 years ago

Commented by: ferranpujolcamins Date: 2015-02-19T10:00:05Z

A filter that evaluates to true for songs that are in a particular crate and multi-parent crates are essentially the same (A child crate is a crate that selects tracks that match its query AND its parents query). They both present a problem when cross-referencing crates, e.g: Crate A = songs that are not in crate B Crate B = songs that are not in crate A

This cannot be evaluated. Did you thought about this when you came up with this ideas? How could we solve this in a way that is efficient and is easy for the user to understand why he/she can't configure a crate the way he/she wanted? Some users might get confused about why their crate doesn't work. In fact this would introduce the users with the task of "debugging" a smart crate expression, as some complicated ones could lead to this kind of evaluation cycles.

If we limit the crates structure to be a tree (just one parent per crate), we can easily prevent this issues. In fact, in this case the only problematic operation would be to move a crate down one of its child branches. This can be managed just by breaking all the child relations of that create before moving it down one of its branches.

We could keep the “is in crate” filter if we don't support it on smart crates queries but just in the search field and as a library column.

So, is it really worth to support multiple parent crates? What are the use cases that would benefit from that and which couldn't be achieved in a reasonably easy way with a tree-like crate structure?

mixxxbot commented 2 years ago

Commented by: daschuer Date: 2015-02-19T11:46:08Z

I am not sure if I get the issue correct, but I think it would be the best to separate the crates and smart crates.

Think of crates in terms of file system folders containing symlinks to the original files, manually setup by the user. We may allow to manually group crates or create a new crate within a upper crate. It sounds reasonable to limit this to a tree structure only, just like it would be in a file system. .. or like a vinyl DJ that puts a sorter into his real crate.

Smart crates are for me sticky filtered filtered views of the library. They do not need to be applied on other filtered views. They also do not need to apply on Crates or playlists, if we allow to set a filter "member of crate/playlist" Bug #⁠1402133 Maybe we should name it not smart crate.

A related aspect of this bug is the hierarchical view of the library. We have some bugs that propose different nature of such a hierarchical tree. I have collected the browse related bugs here: Bug #⁠412453 Here is the multi level crate Bug #⁠671632 and virtual folders: Bug #⁠1228789

For me, a complete solution will feature this:

mixxxbot commented 2 years ago

Commented by: ferranpujolcamins Date: 2015-02-20T01:09:40Z

Hi Daniel! Let me focus on the crates, which is what I'm working on :)

I agree with you that if this is implemented then we don't that kind of parent child relation of crates I described (child crates only match tracks that are in its parent).

Not to confuse this with crates folders which are just a display feature, i.e. they don't change the behavior nor the content of the crates. It is a feature common to both smart and regular crates. As a bonus, selecting a folder could show in the library all the tracks of its sub-folders together, but for this to be useful, smart crates and regular crates should not be in separate sections of the browser (I don't see a problem in that, provided that smart crates have a distinctive icon).

But my main concern now is the "member of crate/playlist" filter: If crate A references crate B via this filter, and crate B references crate A, neither crate can be evaluated. How could we handle this elegantly and making it easy for the user to understand whats the problem? Imagine a user configuring a smart crate and seeing a pop-up saying that he/she just created a circular reference between smart crates...weird.

mixxxbot commented 2 years ago

Commented by: daschuer Date: 2015-02-20T09:02:03Z

Let me focus on the crates, which is what I'm working on :)

Focusing on a doable topic is a good idea and a requirement for a GSoC projekt.

All these crates natures are confusing. I hope we can clarify that somehow. Did you mean "Let me focus on "smart crates"?

"Crates" are for me it is just a place where the user can collect manually some tracks. Lets stick with this term, since it is what we have in Mixxx right now. It is implemented as an own persistent library association table.

In the bug title we have "Smart Crates". We may replace it a more more significant term like "Filter" or "Dynamic ... " (Uwe please correct me) This will be stored as a search phase, which is applied to the library every time the user opens a "Smart Crate"

And a third term we have the fixed library "hierarchy". This might be an auto generated set of filters descending into the natural library structure.

At a "Crate", the user is responsible for each single track in the content.
At "Smart crates" Mixxx is responsible for the content and the user can control this by a fancy filter sting or similar. In the "Hierarchy" we have a natural tree build form the track meta-data, not changeable by the user.

We may merge all this together, but IMHO it is a good Idea to keep it separate.

For me there is no need to have a filter on "Smart Crates". The user might wish to search for a track in a "smart crate", but such add-hock searches, basically extend the filter phrase of the original saved filter.

The "member of crate/playlist" feature can be limited to "crates". If we wish to allow to filter for "member of smart crate", this will just merge the two filter phrases of both smart crates.

"Smart playlists": I think we do not need them, since we have already the Auto-DJ crates and the Add Random feature. It would be great if we can finally add a "Smart crate" to the Auto-DJ, but that is a separate issue.

mixxxbot commented 2 years ago

Commented by: ferranpujolcamins Date: 2015-02-20T14:41:12Z

Hi Daniel, I agree in almost every point. Thank you for clarifying the concepts, I'll use your terminology from now on.

Regarding the member of crate/playlist filter, I'm sorry to keep insisting on the same thing but my concerns are about the user experience.

If users want to use this filter with a smart crate and we don't allow it, they might wonder why they can't. Sure we can inform of that in the manual or even show an info pop-up if they try to do this, but they might feel frustrated.

On the other hand, if we allow the filter to match smart crates, the issue gets even more difficult. As soon as an user tries to do some smart crate configuration that would lead to the reference cycle situation, we should warn him/her and let him/her know which smart crates are affected.

I asked if someone had any idea on how to handle this.

mixxxbot commented 2 years ago

Commented by: daschuer Date: 2015-02-20T21:41:02Z

If users want to use this filter with a smart crate ...

I think we can allow it. The interface can be similar to the crate filter, but the backend will be different, since we will probably have no association table for a smart crate. I hope Uwe can jump in here, since I am not a database expert.

mixxxbot commented 2 years ago

Commented by: Be-ing Date: 2017-05-26T06:12:30Z

Saving and recalling arbitrary queries in the library redesign branch takes care of this. https://github.com/mixxxdj/mixxx/pull/1117

mixxxbot commented 2 years ago

Commented by: naught101 Date: 2018-01-04T02:35:12Z

Are you sure you posted the right pull request there Be? It doesn't really discuss filtering/searching/smart crates other than filtering in relation to AutoDJ...

Daniel: re filtering on smart crates. I think this would be very useful. For example, if I have a "genre:dubstep" smart crate, I would like to be able to filter on key:Am. It doesn't make sense to have 12 key smart-crates for each genre though.

Also, it would be really useful to have "filters macros", which are saved filters that can be added to the current filter. For example a "mid-tempo" filter macro might be "bpm:>100 bpm:<120" or similar. The ability to go into any crate, then quickly apply that filter would be great (it's maybe not the best example, because I could do it with sorting by BPM, I guess. I feel like there are likely to be other examples though).

This could probably be done just as easily by having a search-bar history feature, which would operate similarly to a web-browser's form/address bar auto-fill functions (where it provides a drop-down with matches from previous entries).

mixxxbot commented 2 years ago

Commented by: Be-ing Date: 2018-01-04T03:46:12Z

Yes, saving and restoring search queries accomplishes the same thing but doesn't use the same terminology or interface as other software like Serato. IMO it is better because it could potentially be operated easily from the keyboard.

The "filter macro" idea seems really useful. I have been thinking it would be useful to be able to name saved search queries so typing those user-defined names would be a shorthand for the full query. So instead of using your mouse to open the menu of saved queries, looking for the one you want, then clicking it, you could just focus the search bar (Ctrl + F with en_US locale, not sure about other locales) and type "midtempo", potentially as part of a larger query. We may want to add a new syntax for invoking saved queries to clearly distinguish them from doing a plain text search, for example maybe "query: midtempo" or "q: midtempo".

Keeping a history for queries and autofilling queries are good ideas too! I think making the search query functionality similar to what users are already familiar with from web browsers is a good idea and could make all these functionalities quite intuitive. I don't think we should constrain the design to resembling web browsers because the purpose is not quite the same, but drawing that comparison and taking inspiration from the design of web browsers is a good idea.

mixxxbot commented 2 years ago

Commented by: ywwg Date: 2018-08-20T17:00:32Z

what's the status of this feature / gsoc project? Is it still alive?

mixxxbot commented 2 years ago

Commented by: Be-ing Date: 2018-08-20T18:14:11Z

This is entangled with the massive library redesign branch which still has major performance issues. :( https://github.com/mixxxdj/mixxx/pull/1726

mixxxbot commented 2 years ago

Commented by: ywwg Date: 2018-08-20T18:40:23Z

is there any way to break that huge PR into smaller pieces that can be committed one at a time? ~17000 line change is ... a lot

mixxxbot commented 2 years ago

Commented by: ywwg Date: 2018-08-20T18:42:45Z

or can we make it so the library is a runtime or compile-time selectable option? That way the new code can live in the tree but we don't lose the working one

ronso0 commented 1 year ago

addressed by #3171 (searchbar combobox widget, not Smart Crates) and #4458

fwcd commented 1 year ago

Fwiw, I think saved searches/smart crates could still be useful to support in a more structured way, e.g. as a node in the sidebar or under crates/playlists.

For example: Say I regularly play House tracks in a certain BPM range. In that case it would be nice to be able to add the search to the library, perhaps even with a custom name. Search history feels too ephemeral to me to really serve this use case well, especially if I e.g. want to go back to tweak the query.

Personally I use iTunes smart playlists for this, but it would be nice to have a Mixxx-native solution, especially since iTunes doesn't have good support for filtering by key.

ronso0 commented 1 year ago

Yep, that's why this bug is still open : )

Eve00000 commented 4 months ago

This is how it looks like in iTunes afbeelding As you can see there can be a lot of criteria, parallel to a sql query statement with 'and' 'or' ... My opinion; a smart crate in Mixxx must be equal to iTunes, a smartcrate with 'Genre like "Jive"' won't be sufficient. So a smart-crate should have a query-builder screen with fields, operators, orderselectors (up down). The crate-table in the database can be used if extra fields are added: 'smart (boolean)', 'query (string)' and 'cont-update (boolean)' Besides the query-building-screen an option on the searchtoolbar 'create smart crate with query' would make an easy start

ronso0 commented 4 months ago

I think most of this can be done with Mixxx filters (incl. OR operator and special BPM filtering in Mixxx 2.5+), just Mixxx has no GUI for this.