joedixon / laravel-translation

Translation management for your Laravel application.
MIT License
694 stars 230 forks source link

req(IMP): speed up file to database sync #256

Open chrillep opened 1 year ago

chrillep commented 1 year ago

addGroupTranslation (and or more places) could benefit from upsert https://laravel.com/docs/9.x/eloquent#upserts

would scale and speed up sync 3x-10x times. syncing truncated translation table with 17765 translations takes 15,288.38 ms

If you would like to perform multiple "upserts" in a single query, then you should use the upsert method instead.

All databases except SQL Server require the columns in the second argument of the upsert method to have a "primary" or "unique" index. In addition, the MySQL database driver ignores the second argument of the upsert method and always uses the "primary" and "unique" indexes of the table to detect existing records.

Is your feature request related to a problem? Please describe. I'm always frustrated when it takes to long to sync lots of translations.

Describe the solution you'd like use upsert instead of looping through every key

Describe alternatives you've considered using a translationSeeder that uses upsert

example

alter table translations
    add constraint translations_language_group_key
        unique (`language_id`, `group`, `key`);
<?php

declare(strict_types=1);

use Illuminate\Support\Collection;
use JoeDixon\Translation\Language;
use JoeDixon\Translation\Translation as TranslationModel;

public function addGroupTranslations(string $language, Collection $groups): void
{
    if (! $this->languageExists(language: $language)) {
        $this->addLanguage(language: $language);
    }
    $languageId = Language::where(column: 'language', operator: $language)
        ->firstOrFail()
        ->id;

    $translationGroups= [];
    foreach ($groups as $group => $translations) {
        foreach ($translations as $key => $value) {
            if (is_array($value)) {
                continue;
            }
            $translationGroups[] = [
                'language_id' => $languageId,
                'group' => $group,
                'key' => $key,
                'value' => $value,
            ];
        }
    }

    TranslationModel::upsert(
        values: $translationGroups,
        uniqueBy: ['language_id', 'group', 'key'],
        update: ['value']
    );
}
chrillep commented 1 year ago

@joedixon any thoughts on how to accomplish this without breaking the current API for File|Database driver without duplication?