search

lingdb / Sound-Comparisons

Exploring phonetic diversity across language families —
http://www.soundcomparisons.com
Other
13 stars 8 forks source link

All Sound Files 'Automagically' Stop Working #439

Closed PaulHeggarty closed 7 years ago

PaulHeggarty commented 7 years ago

This is a mysterious but very serious problem that we have experienced a few times in the past, and has now happened again, so it would be good to try to work out what is causing it.

It is a serious, critical bug that takes down all sound files in some but not all sections of the site. In this case, the entire Brazil and Celtic sections are dead, but not the other studies.

As Laura and I can confirm, the sound for Brazil (for the 8 of 11 languages currently loaded) was working properly on Friday. None of us have touched anything in the meantime, but now the sound play links no longer even appear, as if the site cannot see them any more. But the sound files are still present on the server, as they were, with the correct names.

Could this be to do with permissions being updated automatically on some time check? Could it be related to the fix for #437 recently?

Bibiko commented 7 years ago

The permission of sound files is correct since other studies are working. I found a permission bug based on removing the docker system which is partly related. But I'm just digging into that.

Bibiko commented 7 years ago

Could it be a data problem? At least this site shows some transcriptions:

http://www.soundcomparisons.com/#/en/Celtic/language/Léon%3A%20Plouarzel

Bibiko commented 7 years ago

I looked into the online database. For Celtic only 144 transcriptions are stored (IxElicitation 1,2,3,4,5,6,7,8,9,10,11,20) and for Brazil no transcriptions are stored.

PaulHeggarty commented 7 years ago

This is correct, but it's not the problem. The problem is precisely that when there is no transcription, the normal behaviour should be that the site looks for a sound file, and if one exists, then you get the blue play arrow, not 'soon'. 'Soon' is only for when the sound file is missing. On Friday, all Celtic languages but two had working blue play arrows for all words, and so did 8 of 11 Celtic languages. They have now all been replaced by 'soon'. It seems the site no longer 'sees' the sound files.

PaulHeggarty commented 7 years ago

The point is that in most cases, we have sound files long, long before we ever get round to the transcriptions. (Indeed we use the site to help with producing the transcriptions.) We have a great set of Celtic recordings, which we want online now, because it will take Cormac ages, and a few months of time and funding, to do the transcriptions, and that may not be until 2018. But we want the recordings up there and playing now.

PaulHeggarty commented 7 years ago

To confirm and clarify: this issue is about sound files, not about transcriptions. Or at least, it should not be about transcriptions!
However, that may be the clue to the solution. When Laura runs your script, it should automatically create a dummy, placeholder record for every sound file, but with a blank transcription. Perhaps it's that that allows the site to look for a corresponding sound file and display the blue play icon.
Is the problem actually that the dummy blank transcription records have been deleted by some auto clean-up on the database, or perhaps even an SQL sent recently to try to purge the Mapudungun data, but which was over-enthusiastic and killed all records with a blank transcription field in all other studies too?

Bibiko commented 7 years ago

Ok, I got it and I've a vague suspicion that the admin command "Missing sound files" could have to do with it.

Bibiko commented 7 years ago

Finally, after a very long and stony way through the code, I found the bottleneck and I fixed it for now by removing lines of code which are actually wrong (erroneous SQL statements) and which doesn't make sense to me. That's why I keep this issue open until I discussed it with @runjak https://github.com/lingdb/Sound-Comparisons/commit/13c0070ad7ff2cc6948480c2b9f1b56b0986da21

@PaulHeggarty @LauraWae Could you please check the relevant data? [Don't forget to clear the cache]

PaulHeggarty commented 7 years ago

Great, and yes, that has fixed it. (Didn’t need to clear the cache, actually.) Back to how it should be. This also sounds like this means it won’t recur, which is a relief. That one has being bugging us for a few years now. Good stuff!

LauraWae commented 7 years ago

Confirmed, too, from here. Thanks!