Odd behavior when filtering by only one file type or one result type

[vee-L]

When filtering by only one of these (you can select multiple file types or result types, in which case the bug doesn't happen) and then clicking search, you get correct results for the first page, but as soon as you change pages, it resets as if there was never any filtering done. It's probably a very specific pagination thing, I'll look into it after dealing with the urgent stuff.

[musicnerd]

Not sure if this was already noticed and considered a part of 'odd behaviour' but when doing an advanced search using the "voice types" filter, it doesn't seem to work. I had a list of 61 results that included both strictly vocal and non-vocal styles and when I limited it to "vocal" it suddenly said I had zero results. Were you guys already aware of this issue?

[musicnerd]

Furthermore just doing a plain search for Bach, Johann Sebastian (as prompted) gives me only 8 results, but on the side filter it suggests that there are 571 pieces that fit this description?! Surely this shouldn't be medium priority! This is a basic search function. I would like to gather pieces together and I can't. This is the most basic usage of the website and it should work.

[vee-L]

The 571 pieces is the bug, I'm pretty sure there are actually only 8 available results on the main database at the moment. Will take another look.

[agpar]

I had a look, and the problem seems to be that the vast majority of pieces are hidden - that is, won't show up in search results to non-superusers. Here's a breakdown:

'composer': hidden_pieces/total_pieces,
--------------------------------------------------
'Bach, Johann Sebastian': 563/566,
'Beethoven, Ludwig van': 17/17,
'Byrd, William': 89/91
'Casadesus, Henri': 1/1
'Chopin, Fryderyk Franciszek': 198/198,
'Handel, George Frideric': 51/53,
'Haydn, Joseph': 71/73
'Mozart, Wolfgang Amadeus': 3/5
'Scarlatti, Domenico': 557/557,
'Schubert, Franz': 105/106,
'Tchaikovsky, Peter': 1/1

As you can see, all but 3 of Bach's pieces are hidden (+ the composer page itself, +4 non-hidden movements = 8 results).

In all, 1656 of the 2854 pieces on the database are currently hidden. Could this possibly be intentional?

[musicus]

If they are from certain collections.

[imlorenz]

It is intentional. In addition to hiding specific pieces, we have also hidden the entirety of the Yale collection over copyright liabilities for now.

[musicus]

Yes. I didn't want to mention them by name, since they requested to be removed...

[fujinaga]

Mystery solved!

Thank you @lexpar.

On Aug 25, 2016, at 10:34 AM, Alex Parmentier notifications@github.com wrote:

I had a look, and the problem seems to be that the vast majority of pieces are hidden - that is, won't show up in search results to non-superusers. Here's a breakdown:

'composer': hidden_pieces/total_pieces,

'Bach, Johann Sebastian': 563/566, 'Beethoven, Ludwig van': 17/17, 'Byrd, William': 89/91 'Casadesus, Henri': 1/1 'Chopin, Fryderyk Franciszek': 198/198, 'Handel, George Frideric': 51/53, 'Haydn, Joseph': 71/73 'Mozart, Wolfgang Amadeus': 3/5 'Scarlatti, Domenico': 557/557, 'Schubert, Franz': 105/106, 'Tchaikovsky, Peter': 1/1

As you can see, all but 3 of Bach's pieces are hidden (+ the composer page itself, +4 non-hidden movements = 8 results).

In all, 1656 of the 2854 pieces on the database are currently hidden. Could this possibly be intentional?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/ELVIS-Project/elvis-database","title":"ELVIS-Project/elvis-database","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/ELVIS-Project/elvis-database"}},"updates":{"snippets":[{"icon":"PERSON","message":"@lexpar in #224: I had a look, and the problem seems to be that the vast majority of pieces are hidden - that is, won't show up in search results to non-superusers. Here's a breakdown:\r\n\r\n\r\n'composer': hidden_pieces/total_pieces,\r\n--------------------------------------------------\r\n'Bach, Johann Sebastian': 563/566,\r\n'Beethoven, Ludwig van': 17/17,\r\n'Byrd, William': 89/91\r\n'Casadesus, Henri': 1/1\r\n'Chopin, Fryderyk Franciszek': 198/198,\r\n'Handel, George Frideric': 51/53,\r\n'Haydn, Joseph': 71/73\r\n'Mozart, Wolfgang Amadeus': 3/5\r\n'Scarlatti, Domenico': 557/557,\r\n'Schubert, Franz': 105/106,\r\n'Tchaikovsky, Peter': 1/1\r\n\r\nAs you can see, all but 3 of Bach's pieces are hidden (+ the composer page itself, +4 non-hidden movements = 8 results).\r\n\r\nIn all, 1656 of the 2854 pieces on the database are currently hidden. Could this possibly be intentional?"}],"action":{"name":"View Issue","url":"https://github.com/ELVIS-Project/elvis-database/issues/224#issuecomment-242409506"}}}

[musicus]

We only spent the first half of the summer hiding all the pieces. Again, a breakdown in communications.

[ahankinson]

If these are removed they should also be excluded from the facet counts. There is facet.query which may be useful: facet.query=public:true or something along those lines.

[agpar]

I've written a patch that will fix this. There was some trickiness in the code causing 2 searches to happen, one of which was not filtered.

Will fix the last comment from @ahankinson.

[agpar]

@musicnerd As for the voice-types filter, it is limited to those pieces which are explicitly marked as vocal or non-vocal. This is a new(ish) feature that was added last summer sometime. Thus, most pieces on the database have an unknown vocalisation.

The issue with the counts on the right hand side is the one I mentioned having fixed in the comment above.

[musicnerd]

I see. Would there be a way (for now) to automatically assign the voice type based on the 'instrument voice' field? So that whatever possibilities show up in the instrument voice field belong to either instrument (e.g. 'orchestra') or voice, or possibly both? Or are there also not enough 'instrument voice' data?

[imlorenz]

@lexpar When I took my first foray into fixing the metadata for the ELVIS Database, I noticed this as well. I will be correcting that issue once I am finished making sure that the Wegman files are correct.

[agpar]

That's a good suggestions @musicnerd. We can run a quick loop over the database applying that heuristic. Should update a great deal of the pieces.

@imlorenz, what issue are you talking about? I'm getting a little mixed up here!

[imlorenz]

Sorry for the confusion. I meant the issue with the voice-type filters. When I went through the whole Database collection I noticed that a majority of both instrument voices and voice types were left unknown. As far as the voice filters, I could have probably fix them by tomorrow at the latest. This would save @lexpar the issue of making the loop when it is an problem that needs to be fixed on the Database anyway. @musicnerd Is there a specific time constraint that you are under, or would having it be fixed by tomorrow afternoon be okay?

[musicnerd]

time fix for this issue by tomorrow is plenty early. A timeline for the fix for ALL the bugs I'm hoping like a few weeks at most... but maybe that's just wishful thinking...

[agpar]

It probably is wishful thinking. @vee-L will be working 10 hours a week after this week.

How do you plan to fix this @imlorenz? Do you mean going through them manually? Is that the alternative to me writing a loop?

[musicnerd]

I'm still a bit confused about some files being hidden due to copyright. Firstly, we have currently 77 pieces from the kernscores database coming up on a search, so it can't be that they didn't want us to use ANY of their stuff (or are those supposed to be hidden and somehow aren't?) Also, in that list of 77 includes Scriabin pieces from the early 20th century, so if there's copyright stuff going on I don't see why Bach is a problem but Scriabin isn't? I'm new here so forgive my ignorance of these issues.

[imlorenz]

@lexpar Yes, I would have to go through the pieces manually and give them a voice type. Since instrument voices and voice types are two separate filters--the former being a specification of the latter--I think it would be best to keep them as two distinct filters, as opposed to looping through one another.

[fujinaga]

Anything that originates from CCARH cannot be redistributed: http://www.musedata.org/legal/lcr.html

On Aug 25, 2016, at 3:09 PM, musicnerd notifications@github.com wrote:

I'm still a bit confused about some files being hidden due to copyright. Firstly, we have currently 77 pieces from the kernscores database coming up on a search, so it can't be that they didn't want us to use ANY of their stuff (or are those supposed to be hidden and somehow aren't?) Also, in that list of 77 includes Scriabin pieces from the early 20th century, so if there's copyright stuff going on I don't see why Bach is a problem but Scriabin isn't? I'm new here so forgive my ignorance of these issues.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/ELVIS-Project/elvis-database","title":"ELVIS-Project/elvis-database","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/ELVIS-Project/elvis-database"}},"updates":{"snippets":[{"icon":"PERSON","message":"@musicnerd in #224: I'm still a bit confused about some files being hidden due to copyright. Firstly, we have currently 77 pieces from the kernscores database coming up on a search, so it can't be that they didn't want us to use ANY of their stuff (or are those supposed to be hidden and somehow aren't?) Also, in that list of 77 includes Scriabin pieces from the early 20th century, so if there's copyright stuff going on I don't see why Bach is a problem but Scriabin isn't? I'm new here so forgive my ignorance of these issues."}],"action":{"name":"View Issue","url":"https://github.com/ELVIS-Project/elvis-database/issues/224#issuecomment-242503770"}}}

[musicnerd]

At the moment, if you type 'kernscores' (it will prompt you to KernScores) in the general search box, then you get a list of 77 results of stuff within this collection. This is CCARH's collection, so I'm guessing this should all be hidden from the average user? Anyone want to fix that?

[imlorenz]

They are currently hidden.

[musicnerd]

@imlorenz, I'm not sure that they are hidden. Until Friday I was a non 'super user' and basically had access to anything that any outsider could view, and I was able to see this list of the 77 pieces. Is there a way to check that they are truly hidden?

[imlorenz]

@musicnerd Yes, that was my fault. When I initially starting hiding pieces on the Database, I didn't realise that the KernScores collection was related to CCARH, so I didn't think to hide them. When you told me about this on Friday, however, I hid them. So now they should not be visible to the public.

Also, I believe that I have finished changing the voice-type filters for the pieces currently not hidden on the Database with a few exceptions. I have not yet updated the Wegman Mass and Model files, because there are many errors with them and a good portion of them duplicates (I'm in the process of taking care of that now). Also, any pieces that are listed under movements--such as many of the files in the Digital Du Chemin collection--have not been updated because the Database won't allow it. So that is something we will have to look into further on. I hope this helps, and thanks for bringing this to my attention!

[musicnerd]

Oh I see. OK well hopefully it's all good now. Thanks! :)