othmar52 / slimpd

PHP/JS based MPD-web-client for large music collections
GNU Affero General Public License v3.0
41 stars 11 forks source link

Unknown Artist autocorrection on Russian Tracks #64

Closed filatovvo closed 6 years ago

filatovvo commented 6 years ago

Hello, i changed enable_guessing to 0 in ./core/config/local.ini, removed /localdata/cache/conf*, rescan all my library with hard-reset, but when i open track info, there everithing OK in Raw TAG Data, But in Track Entry (autocorrected tag attributes) Artist == Unknown Artist, and i don`t know how to fix it, can you help me? Screenshot

othmar52 commented 6 years ago

Hi, If i remember correctly this enable_guessing isn't implemented yet. So i assume it has no effect at all and you are doomed to live with activated guessing/autocorrection.

I can reproduce the described problem which seems not to be caused by sliMpd's autocorrection stuff but by a restriction of sliMpd's internal string handling.

you can try to hack the string simplify function by

approach 1: simply return the inputstring

function az09($string, $preserve = "", $strToLower = TRUE) {
    return strtolower($string); // add this line
    $charGroup = array();

approach 2: add preserve characters in default argument

function az09($string, $preserve = "абвгдеёжзи<add all chars here>", $strToLower = TRUE) {
    $charGroup = array();
    ...

approach 3: extend the config with matching alphanumerics

$charGroup[] = array("b","Б", "Б");
$charGroup[] = array("j","Й", "й");

but maybe those 3 approches does not work at all... With a bit luck approach 3 - which is an elegant solution and no dirty hack like 1 and 2 - works. would be great if you can tell me if you had success

Don't expect the problem beeing solved in near future. currently too much other stuff i am on...

filatovvo commented 6 years ago

I commented $string = preg_replace("/[^a-zA-Z0-9". $preserve ."]/", "", $string); in az09 function, and my problem was solved, i`ll try approach 3 little bit later.

approach1,2 - Not working, slimpd update crashed on /opt/slimpd/core/php/Modules/Albummigrator/TrackArtistExtractor.php(359): Slimpd\Repositories\ArtistRepo->getUidsByString

filatovvo commented 6 years ago

But now i have some other issue on ./slimpd update. I have Over 3TB files on my Network Hard Drive, and got this Screenshot. I have changed Tags on this mp3 file, and don`nt know how to solve this without Hard-Reset. It takes too much time to fully hard-reset my library.

othmar52 commented 6 years ago

The disabling of this preg_replace makes a few queries vulnerable for sql injections

you have to change ArtistRepo.php#L142 to

$query = "SELECT uid FROM " . self::$tableName ." WHERE az09=\"" . $this->db->real_escape_string($az09) . "\" LIMIT 1;";

maybe there are some other queries where az09 column is used without escaping special characters

btw. incremental update shouldnt be an issue as the basis for sliMpd's update is the database file of your mpd installation. and mpd's database has the last-modified information per track included

filatovvo commented 6 years ago

By the way your approach 3: extend the config with matching alphanumerics works well

$charGroup[] = array("a","А","а");
$charGroup[] = array("b","Б","б");
$charGroup[] = array("v","В","в");
$charGroup[] = array("g","Г","г");
$charGroup[] = array("d","Д","д");
$charGroup[] = array("e","Е","е","Ё","ё");
$charGroup[] = array("zh","Ж","ж");
$charGroup[] = array("z","З","з");
$charGroup[] = array("i","И","и","Ы","ы");
$charGroup[] = array("y","Й","й","Ъ","ъ");
$charGroup[] = array("k","К","к");
$charGroup[] = array("l","Л","л");
$charGroup[] = array("m","М","м");
$charGroup[] = array("n","Н","н");
$charGroup[] = array("o","О","о");
$charGroup[] = array("p","П","п");
$charGroup[] = array("r","Р","р");
$charGroup[] = array("s","С","с");
$charGroup[] = array("t","Т","т");
$charGroup[] = array("u","У","у");
$charGroup[] = array("f","Ф","ф");
$charGroup[] = array("h","Х","х");
$charGroup[] = array("ts","Ц","ц");
$charGroup[] = array("ch","Ч","ч");
$charGroup[] = array("sh","Ш","ш","Щ","щ");
$charGroup[] = array("`","Ь","ь");
$charGroup[] = array("e","Э","э");
$charGroup[] = array("yu","Ю","ю");
$charGroup[] = array("ya","Я","я");
filatovvo commented 6 years ago

i think it must be in repo, for the first time.

And another problem Screenshot1 Screenshot2 If there is "-" the track title, slimpd thinks that it`s artist name ;)

othmar52 commented 6 years ago

btw. incremental update shouldnt be an issue as the basis for sliMpd's update is the database file of your mpd installation. and mpd's database has the last-modified information per track included

this did not work before 101e2078d5f04a37c623500b061155e80a18ef7c. now it should be fixed. funny that i never tested this must-have

filatovvo commented 6 years ago

Great ;)

othmar52 commented 6 years ago

so now you have to follow these steps:

1) pull the latest changes 2) add those missing lines to your sphinx.conf 3) restart sphinx daemon 4) run ./slimpd update to update modified tags 5) run ./slimpd remigrate to get rid of those "Unknown Artist" 6) update your shinx index by running core/scripts/sphinx_rotate.sh