Idea: Add a coverflow mode

[jrbastien]

Add an option to present the covers in coverflow mode rather than in library view. There was once a plugin for this in Rhythmbox but I think it is now obsolete.

http://lifestream.nbonnici.info/items/view/316

[fossfreedom]

This is actually a stretch goal I have for v0.9 - see the milestone description for the aims of v0.9.

Coverflow doesnt work for a large number of albums - its takes too long to scroll through.

Thus the way I'm currently thinking that this should operate is that where the icon-view would display (for example) 20 or less covers then the iconview would be hidden and the space replaced by a coverflow display for the remaining albums.

If you change the filter or use the toolbar buttons and the result would be more than 20 albums, then the coverflow would disappear replaced with the icon-view again.

For folks that dont like coverflow - this automatic switching would be configurable. Similarly the number of albums before switching would be configurable..

@asermax and myself briefly discussed this in issue #103 and we've got the coverflow code we could potentially use.

Not sure though yet where or how to make the changes.

[asermax]

I think we should first put in place a generic mechanism that allows us to plug diffirent views using the same model, an to be able to switch between them through a somewhat consistent interface. Once we have an stable definition of this mechanism, we can start playing with the different views, otherwise if we rush into making the views first, we will end having to refactor half of it when we realise that it isn't flexible enough.

So, for this specific topic: I'm all in with the idea, but we should't rush it. Also, I think there should be a generic way to switch between views instead of some automatic behavior, at least at first.

[jrbastien]

@fossfreedom, when you say the cover view does not work with a large number of albums. Is it a python or linux limitation? This works pretty well on my Ipod Classic with 500 + albums. It also used to work well in iTunes before they made the UI refactoring.

I think when there is a large number of covers, it just accelerate when scrolling through without stopping. It also does not display all the covers to help you gain some speed. They only appears when you stop.

[fossfreedom]

I suppose it may work - it depends upon the "mouse" swiping control.

I've never found a linux coverflow that has convincingly worked. The library of code described above doesnt have any "mouse" control. We'll have to implement this ourselves.

The link you gave also never had mouse control ... when I tested this, it was a simple left/right keyboard arrow coverflow action.

Most of the coverflow's seem to be "web-based" - the code appears to be proprietary and buried in a server somewhere..

The demo of ubuntu-touch looked interesting with its coverflow... but it only works with QT - this would be a massive overhead to integrate this - and will not work for ubuntu 12.04 due to the old QT libraries available.

[jrbastien]

Wasn't Songbird open source at one point? It used to have a coverflow mode that wasn't so bad. I don't know the technology it used though.

EDIT: Another one that works well is in CALIBRE. I have quite a lot of digital books for which I have no problem scrolling though.

[fossfreedom]

songbird was replaced with nightingale ... and that is written in C++... my C++ knowledge is 15 years out-of-date!

[asermax]

Any and all examples are useful, doesn't matter much which language d: most languages are pretty much the same to read (not to code tho).

And at the end, we will have to write it pretty much from scratch, since we have to adapt it to our models and interfaces. The important part to lay down is the graphic stuff, which we can get a hold of from all the examples out there; all the rest we can plug it in as we see fit, as long as we build the core component extendable (hooks for moving through it programatically with ease as a main example). I don't see much difficulty in adding mouse gestures, keyboard shortcuts (for example a "Go To Letter" shortcut) and any other stuff like that.

[fossfreedom]

@jrbastien - calibre - as in the e-book manager? didnt know that had a coverflow. Love to see that in action.

Calibre-ebook is written in python - which is very useful.

[jrbastien]

Yes, Calibre as in the e-book manager. I like @asermax's suggestion to implement a "Go To Letter" shortcut. This is probably the best missing feature from all the coverflows I have seen!

[jrbastien]

There is a demo of the coverflow in Calibre here: http://www.youtube.com/watch?v=DVe6LkmXR-Y. This is an old version, this is now at the top of the application with the info of each book below. That could works nicely in RB, showing the track listing instead.

[fossfreedom]

@jrbastien - that looks exactly the sort of coverflow I had in my mind ... cheers.

I'll have a look at the source code and see if there is anything we can "borrow" for this plugin.

[fossfreedom]

hmm.

ok some notes.

The actual coveflow widget is a mixture of QT4 python code and C++.

The real widget is C++ from this project:

• http://gitorious.org/ofi-labs/pictureflow/trees/master/pictureflow-qt

This FAQ describes how to make python bindings for this C++ code

• http://code.google.com/p/pictureflow/wiki/FAQ.

In fact it looks like ubuntu-touch coverflow is similarly created using the same C++ / QT python bindings

coverflow.py shows how the calibre coverflow is actually constructed together with the mouse bindings used to control the swiping action

• http://bazaar.launchpad.net/~kovid/calibre/trunk/files/14884/src/calibre/gui2

Using SIP - http://pyqt.sourceforge.net/Docs/sip4/using.html#a-simple-c-example

EDIT:

and after install qt4-dev-tools this program run against /usr/share/backgrounds/ produced a very nice coverflow with mouse swipe actions - nice!

• http://www.rkblog.rk.edu.pl/w/p/pictureflow-listing-images-pyqt4/

no idea though how a QT widget is supposed to be inserted into a GTK widget such as a grid or box.

EDIT:

I wonder if we can use the webkit control with CSS transforms like this:

http://scottgale.com/coverflow-css-3d-transforms/2011/05/24/

http://askubuntu.com/questions/97430/connect-webkit-webview-form-to-a-python-callback

[fossfreedom]

ok - after much ebb-and-flow trying to decide on the best course for this, I've settled (most probably :) ) on the following:

• https://dl.dropboxusercontent.com/u/3575524/ContentFlowRework.tar.gz

Extract it (tar -zxvf ...) and run within the ContentFlow folder python content.py)

Its basically a HTML webpage with javascript controlling the coverflow motion - use the bar or the scroll on your mouse or touchpad.

It seems to work very well with the rather elderly webkit in ubuntu 12.04 so I have every confidence on its longevity.

Obviously since its "web" based, it works with files. So our plugin will need to write out the covers into temporary files that the webkit app can see and use.

Also, I presume the other difficulty would be "speed" - we dont want to write out every cover into a file - probably just a select of files that get updated as the user scrolls. My javascript is non-existent, so no idea really what to do in this area.

Also there appears to be some good examples of alternative coverflow's we can achieve as well

• http://www.jacksasylum.eu/ContentFlow/addons.php

This is released on the MIT license, so we can include this/modify etc without any difficulties with copyright etc.

[asermax]

A couple of things:

• The covers are already files, they are saved on rhytmbox cache. The fact that we transform them into Pixbufs doesn't stop us from using the files, but seems kinda cumbersome.
• This seems nice for static content, but how easy is to add dinamic content? And I don't mean just the pictures, but also the corresponding text and any other info we would show on screen. Have you tried using it in this way?
• Also, events. On the cover search pane we also use webkit and is a real pain to comunicate back and forth with the event handlers on the python side. If this is going to manage the same actions it already does on the default view, it seems like it will be something difficult to do.

Those are my concerns regarding to using a js cover flow on top of webkit. Using clutter or somthing like that seemed a little more ideal from my perspective; then again, idk how much time I will be able to dedicate to help you out, so decide for the option that better fits your necessities.

[fossfreedom]

I've considered using clutter - however, clutter does not play nice with Compiz which is used in Ubuntu. In 12.04, using clutter it displays as an overlay. Laying any window ontop of the clutter "stage" goes screwy with black artifacts.

Mind you it clutter seems to work ok in gnome-shell ... but I suppose that is expected since mutter/clutter are from the same devs.

QT coverflow works brilliantly ... but it doesnt appear to be able to integrate within a GTK3 window - you can only display QT effectively within its own window.

As to text - yes, you are at the limits of what can be displayed by CSS & HTML but I dont think this will be a show-stopper.

I need to do some off-line experiments with webkit and how to pass long lists of filenames and getting feedback on what is clicked.

I'm open to finding an alternative to webkit. Designing something from scratch is beyond my expertise - I much prefer integrating & manipulating existing stuff. I havent found anything really useful beyond the original rhythmbox coverflow jrbastien linked to at the top - however this needs a complete rewrite to make it work with gtk3 :/

[fossfreedom]

@asermax

just to start off - remember, until a few days ago, my HTML knowledge was that the web somehow was using pages of the stuff (!), javascript was just a word and CSS was something I've never heard of... :)

after further investigation, I've got to this point:

• https://dl.dropboxusercontent.com/u/3575524/coverflow.tar.gz

extract and run in the coverflow folder python coverflow.py

What there is, is a coverflow with the following "functionality" - very loose term!

• python creates the html using the filenames of the pictures from ~/.cache/rhythmbox/album-art
  • webkit.webview responds to a click event passing what was clicked to python - I presume we could something similar where two clicks could open/close the track-view
  • similarly it responds to a double click event - so should be able to hook this up to a "play album" call.
  • a right click event is detected and its possible to pass the mouse X & Y position together with what was clicked back to python - thus we could use our standard right-click popup stuff.
  • by clicking a cover, it sends a response to python. Python sends back a bunch of data and we dynamically create table rows (removing first what was there). Thus I think this shows we can have "dynamic" text and stuff.

Passing stuff back and forth will need to be via JSON strings - we'll need to invent a couple of functions that packages up data and responds to strings that start with the event name e.g. "rightclick:::{stuff: [item, screenX, screenY]}" and "doubleclick:::{stuff:[item]})

Have a look - seems to me this is a fairly good "off-the-peg" coverflow solution we could tailor to our needs.

[asermax]

More comments d:

• Seems like the dynamic content isn't a problem then. We have to take care of making a consintent interface for the view, since all the views will have to respect some kind of interface if we want to make them pluggable.
• The events are easily catcheables using js (I work making web pages, so I kinda know that xD) but the comunication with the python side is still bothersome. Having to use the title and making a protocol on top of it is, on one hand complicated, and the other, completely different from how the other components of the plugin work. That's my principal concern; I don't say it is impossible, just that we should be extra carefull with that.

Other than that, I agree that it's a possible solution for this view.

On a side note, it seems to break with a large amount of images. On the first try running it, it crashed on me with a segmentation fault. The second time it ran, but showed just a couple of images, while the others looked like this: This error was shown (js related, probably):

** Message: console message: undefined @0: INDEX_SIZE_ERR: DOM Exception 1: Index or size was negative, or greater than the allowed value.

Also, the sensibility was really high, It wouldn't stop moving xD

[fossfreedom]

That crash is worrying me - could be a show-stopper for the whole thing. I've only got 443 albums so I cant reproduce this.

I dont know if it due to the fact that 3500 images with a "src=[filename]" html element is being loaded. If it is, then perhaps we can create the html without a "src=" value (or just empty string), and tuck the filename away is a new img attribute. Then somehow dynamically fill the "src=" value with the filename when scrolling.

However, if it is simply a crash since the html size is too big then we are scuppered.

The sensitivity can be controlled by using one or more of the FlowOptions here

• http://jacksasylum.eu/ContentFlow/docu.php?show=options&#options

[asermax]

I think the sensibility issue had more to do with the large amount of images. It's probably calculated based on that: if there are more images, the flow should go faster, but I guess that having so many images was taken into account.

Anyway, I think one posible way of solving this issue and the crash/problem loading images, is using preemptive loading: start loading just a couple of images (100 for example) and when reaching the edges, throw an event to the python side to load another batch of images. When reaching the top limit (the max amount of images you can actually have there), unload the ones that are on the opposite edge. Idk how well prepared is for this the javascript side, but I thought I would throw it out there as a posible idea.

[fossfreedom]

@asermax - ok - finally found some time to look at this (!)

• https://dl.dropboxusercontent.com/u/3575524/coverflow.tar.gz

It now loads covers in batches - each batch is 50 covers.

Let me know if this fixes the crash you noticed.

I had a quick look at removing items as well - but I couldnt figure out why the visible cover kept changing. So at the moment, the coverflow just dynamically loads batches as you increasingly scroll though your collection.

[asermax]

It doesn't crash now, but its behavior is pretty inconsistent. I'm on albun 300 an suddendly it jumps back to 100, the sensibilty rises when new albums are loaded and I guess that eventually it will start throwing the same error as before. Maybe keeping only 100 albums at a time is a good compromise to avoid this kind of issues.

Anyway, the coverflow is obviously not a good choice for browsing a big library, so idk if it's really worth taking the big library case into account, at least at first, and just adapt it to the use case of an small library or a filtered view.

[fossfreedom]

hmm - I dont see this so its a little difficult to debug from my side - I'll throw in some more output tracing to see what covers are being added for each batch.

I have noticed the coverflow is much smoother on ubuntu 13.04 compared to ubuntu 12.04. So maybe its the newer webkit that is making the difference.

I've never thought coverflow is a clever idea for larger libraries (see my first comment to jrbastien at the top). Yes - I've thought it would be best to link it to one of the current search options (album-model filter-changed signal??) that when the number of covers displayed is (for example) below 200 then the display (optionally) switches to coverflow.

I'm looking at the current code at the moment but am not sure how to integrate it with the current code-structure. I would welcome your thoughts on this.

BTW - the only other "view" I'm imagining for this plugin is an "artists view" - display all the artists visually by the iconview - click on the artist and then the display switches to album iconview/coverflow and shows the albums that the artist has a track on. That is my "v1.0" vision!

[asermax]

I don't think that doing an auto-switch is a good idea, unless it's parametrized and configurable.

Anyway, my idea regarding views was of laying down a common interface that views should respect: some methods to initialize and hide/destroy it, someway for switching views, maybe an unified GUI for the switching; also, if the idea is sharing the context menu and other stuff (maybe the toolbar), each view should implement certain hooks to allow this context aware elements to plug themself into the view. Now that most of the plugin was refactorized and most of the stuff is isolated in classes/modules, it should be easier to put something like this in place. Neverthless, I think that laying down the views requires that we reorganize the modules and packages structure, since putting everything in a single View module isn't a good idea, and the current modules will probably grow larger than what they already are. I'll try to think of a different configuration and post it on the refactoring issue, feel free to do the same so we can reach a consesus for the best way to implement this.

[fossfreedom]

ok - making some headway with this.

The following is a list of tasks on my to-do list - will be edited as we make progress during development:

• Need graphical based radio-buttons to switch between views. Need @jrbastien help here to design these and other icons for each theme. Basics are there - just need the icons + load icon code in view-manager class. -- use text based instead with labels that change colours
• Add a "need to filter" display of coverflow when the number of albums exceeds a user-definable figure.
• Catch updates to album details and update the requisite coverflow album representation.
• noted that sometimes selecting an icon in iconview does not update the selection after switching from coverflow
• This batch mechanism inherited from prototyping seems overkill and unmaintainable. Consider removing this to be a simple load "x" amount of album covers upto the user-definable figure. The advantage is that we can switch back-on the circularflow option as well as the scrollbar - the disadvantage is the extra load time
• There appears to be 3 filterchange events on first load of the plugin - the coverflow is loaded each time the filterchange event is executed. Why are the 3 filterchange events? - overcome by waiting for the covers to be loaded
• If coverflow is the default view - on first display - no covers are displayed. Seems like the covers are added after the filter-change method is executed. User at the moment has to search/change a filter for the covers to appear - overcome by waiting until the covers have been loaded
• bug #195 needs resolving ASAP to stop crash when a user adds a cover manually/searches and updates a new cover.
• clicking on a icon twice does toggle the cover-entry view - but the entry view does not expand. User at the moment has to drag the entry-view bar upwards to display the tracks. Why? Dont really userstand what is going on with the new panedcollapsible widget that should cope with this.
• link flow-automatic to paned-collapsible so that clicking on an item does not automatically open/close the track-view

Some on the nice-to-have list items (may not be possible to do in the time remaining before release)

• user-definable black vs white background
• coverflow supports add-ons - consider adding a vertical & horizontal carousel add-on - users can then switch to these if required.
• add a highlight around the selected item --> decided against this, dont believe this works very well
• support updating of covers from covers tab (drag and drop, double clicking --> this link will help) --> as below OR
  • in coverflow mode, the covers tab on the entry view should be hidden - doesnt make sense for it to be visible since you cant update covers via coverflow --> coverflow now supports drag and drop from the coverview
• it looks fairly straightforward to display a play symbol when hovering over the selected icon - should be straightforward to link this to a one-click to play an album. --> nope - after further research no real idea on this. This is a nice-to-have
• for the top toolbar - the view icons should be centered within the space between the filter buttons and the search entry. How?
• for the side toolbar - the view icons should be centered - How?
• user defined scale-factor for the image size.

[jrbastien]

Good to hear back from you Fossfreedom. Personally, I have been taking it easy this summer and I did not work on the wiki re-org I promised but I can certainly help with these icons. Will the master branch work on Ubuntu 12.04 RB 2.96 or do I need a newer system?

[fossfreedom]

hi there - nice to talk to you again after a long while. Hope all is well.

Yes please for the icons. Yes, the master branch should work for all versions of rhythmbox since v2.96 including the forthcoming RBv3.0 coming in October.

We have the coverflow view now (still in heavy development - state of play on the to-do list above) and the standard icon-view.

In the future I would like to add one more view called "artist view" - basically, display all artists in an icon view. When you click the artist, it displays the albums for the artist.

The last icon needed is a "properties" icon - clicking this displays a menu allowing you to download all album covers, display the preferences screen and some other frequent used preferences options (to-be-defined).

[jrbastien]

I will need to see it in action to better understand where these icons will fit and how you intend to use them.

But unfortunately, when I run my install script, it just deletes the folder in PLUGIN_PATH="/home/${USER}/.local/share/rhythmbox/plugins/coverart_browser/" and do not create it again.

I don't see any change in install.sh so I'm having a hard time to figure out what is happening. The script I have been running since I started working on this project is:

!/bin/bash

sudo rm -rf /home/jrbastien/.local/share/rhythmbox/plugins/coverart_browser rm -rf ~/.cache/rhythmbox/cb_genre cd /home/jrbastien/Téléchargements rm -rf coverart-browser git clone https://github.com/fossfreedom/coverart-browser.git -b master cd coverart-browser sh ./install.sh

[jrbastien]

OK, I can manually copy the files and it is working. I see the 2 icons you need. Could we move them in the middle or somewhere else? If we are to put them next to the other one, then what we need for consistency is just one button that would change its icon depending upon the selected view.

[fossfreedom]

yep - I will flesh out the toolbar stuff asap so that you will have something to play with.

The first icon will be the properties button

The row of three toggles will be the v1.0 release - thus each toggle would be an icon itself.

Maybe differentiate the toggle buttons from the other icons by making the icons twice as wide?

on itunes you see the toggle buttons in a row at the top with the "icons" on the left hand side (music and some cloud thing)

[fossfreedom]

ok - I've made some changes to allow you to design the icons more easily

The new icons have the following filenames in each theme folder (img/dark, img/light & img/standard):

 artistview.png
 flowview.png
 iconview.png
 properties.png

Currently they contain some random png images I found lurking in /usr/share/icons

[fossfreedom]

@asermax

re the following observation of the coverflow view.

clicking on a icon twice does toggle the cover-entry view - but the entry view does not expand. User at the moment has to drag the entry-view bar upwards to display the tracks. Why? Dont really userstand what is going on with the new panedcollapsible widget that should cope with this.

I've been looking at this far too long - any hints on what the possible issue is? I've added a few print statements in the panedcollapsible widgets looking at the self.collapsible_y value - but the values look the same between the iconview and coverflow when expanding and not expanding.

I was thinking it could be something to do with the overlay - but not really sure. Thoughts?

[asermax]

Seems like the do_button_release_event callback is being called on the PanedCollapsible just before the call to the _on_collapsible_expanded is made, and sets the collapsible_y value to the current position of the handle, which is down the bottom of the screen. Idk why it's being called tho, most probably haves to do with some side effect on the CoverFlowView (I'm just assuming here, since it works correctly on the other view).

On a side note, moving the handle on the Paned for the CoverIconView is really sluggish right now, like if it were redrawing each time the position of the handle is updated.

[fossfreedom]

indeed - if I comment out the whole do_button_release_event code then the toggle of the entry-view seems to work in both views.

Is this event-code really needed? - seems to work ok without it.

Re your side note - havent really noticed this.

[asermax]

That callback saves the current position when the lower pane is left open, something some users might do, specially if using the cover flow view(no need to hide it since it only uses a little portion of the screen). Is up to you to keep it or remove it, I think it covers a posible use case and the best solution is finding the actual issue causing that callback to fire incorrectly.

[jrbastien]

I have been away this weekend but I have started to look at the icons tonight. Yes, I properties icon would fit with the other ones but for the view I have a suggestion. I admit I shamelessly took the idea from iTunes. Why don't we use text instead? I would provide you with the background buttons and you would just need to change the buttons and the color of the text upon selection.

The challenge for you would be to stretch or squeeze the png depending of the translation text.

Here is a quick and dirty mock-up showing the effect of the coverflow selected first, then the Tiles view (for lack of a better term):

[fossfreedom]

@asermax - this is just speculation - I'll have a look closer later.

Maybe there is some sort of cascading event thing happening. On the coverview - it is using the javascript click event to intercept when a user clicks on the cover. Maybe this is equivalent to the "mousedown" event? Thus - it sends a message to python - when in-turn sends an expand signal. The Gtk.Expander is interpreting the current mouse state as mouse-down and hence the next signal is the mouse-up/release event which we are seeing.

Yes ... speculation! Lets see what I find.

[fossfreedom]

@jrbastien ... indeed a challenge.

Not sure on how to do this - I'll have a "google" to see if I spot if someone has done something similar to gain inspiration :)

note to self - some old gtk2 code here to change the font colour. I think the buttons autoresize anyway depending upon the text (need to double check) - http://stackoverflow.com/questions/7258963/how-to-set-a-gtk2buttons-text-colour/7369328#7369328

[jrbastien]

@fossfreedom , some clarifications

In this design, all buttons are of the same width. The longest text of the 3 controls the width. In a first phase, we could simply make them wide enough that they would support most of the languages.

To make them adjustable with the language, a pseudo algorithm (for a non programmer like me) would be:

• Get text length of each three.
• Take the longest (here it is 9)
• 9 is our base reference since in English "Coverflow" is of 9 character and the widest.
• Buttons remain 100% of wide
• If longest text is 12, we simply stretch all button images by 12/9 (133%).
• Notice that it does not account for characters being of different width but that should be close enough

[jrbastien]

I just thought of a much easier solution:

If it is possible for you to use transparent buttons? we could just put the whole buttons bar as a background image. I would provide you with the 3 states. You would just need to change the background image and the text color.

I would also provide you with 3 sets (a small, medium, large). We will just need to figure out what will be the best way to determine when to use either one of the sets.

For you, it is easier because you don't have to calculate coordinates. Just use centered text. For me, this is fun because I can create different designs for the buttons!

[fossfreedom]

@jrbastien - sorry - a bit remiss of me to get back to you. I'm playing with some stuff at the moment. When done, I'll try to get back to you with something more concrete that we can work together with.

[jrbastien]

Not a problem. I'm slowly progressing myself on the design of the properties icons. I will open 2 separate issues when I'm ready to address the properties icon design and the toggle buttons design. This has polluted this thread enough.

[fossfreedom]

ok - this one is now "done".

The iconview (tile-view) needs a good regression test and the coverflow needs a good workout as well as polishing out some of the idiosyncrasies.

Bug reports would be appropriate now.