wagtail / wagtail-localize

Translation plugin for Wagtail CMS
https://wagtail-localize.org/
Other
216 stars 83 forks source link

Issue String.MultipleObjectsReturned #758

Closed Nigel2392 closed 4 months ago

Nigel2392 commented 6 months ago

When using Translation.import_po, there can be an issue with some databases and filtering. Some databases by default, are case insensitive. This means that the translated string Home, would also retrieve home - get() fails because of String.MultipleObjectsReturned.

This can be fixed by filtering by the data_hash, instead of the data itself.

This means that instead of filtering like:

                string = String.objects.get(
                    locale_id=self.source.locale_id, data=entry.msgid
                )

We should filter like so _(For some reason data__exact does not work?!??!)_

                string = String.objects.get(
                    locale_id=self.source.locale_id, data_hash=String._get_data_hash(entry.msgid)
                )

Fix (wagtail_localize.models.Translation.import_po (LineNo. 1228)):

    @transaction.atomic
    def import_po(
        self, po, delete=False, user=None, translation_type="manual", tool_name=""
    ):
        """
        Imports all translatable strings with any translations that have already been made.

        Args:
            po (polib.POFile): A POFile object containing the source translatable strings and any translations.
            delete (boolean, optional): Set to True to delete any translations that do not appear in the PO file.
            user (User, optional): The user who is performing this operation. Used for logging purposes.
            translation_type ('manual' or 'machine', optional): Whether the translationw as performed by a human or machine. Defaults to 'manual'.
            tool_name (string, optional): The name of the tool that was used to perform the translation. Defaults to ''.

        Returns:
            list[POImportWarning]: A list of POImportWarning objects representing any non-fatal issues that were
            encountered while importing the PO file.
        """
        seen_translation_ids = set()
        warnings = []

        if "X-WagtailLocalize-TranslationID" in po.metadata and po.metadata[
            "X-WagtailLocalize-TranslationID"
        ] != str(self.uuid):
            return []

        for index, entry in enumerate(po):
            try:
                string = String.objects.get(
                    # Filter by hash instead of by data.
                    locale_id=self.source.locale_id, data_hash=String._get_data_hash(entry.msgid)
                )
                context = TranslationContext.objects.get(
                    object_id=self.source.object_id, path=entry.msgctxt
                )

                # Ignore blank strings
                if not entry.msgstr:
                    continue

                # Ignore if the string doesn't appear in this context, and if there is not an obsolete StringTranslation
                if (
                    not StringSegment.objects.filter(
                        string=string, context=context
                    ).exists()
                    and not StringTranslation.objects.filter(
                        translation_of=string, context=context
                    ).exists()
                ):
                    warnings.append(
                        StringNotUsedInContext(index, entry.msgid, entry.msgctxt)
                    )
                    continue

                string_translation, created = string.translations.get_or_create(
                    locale_id=self.target_locale_id,
                    context=context,
                    defaults={
                        "data": entry.msgstr,
                        "updated_at": timezone.now(),
                        "translation_type": translation_type,
                        "tool_name": tool_name,
                        "last_translated_by": user,
                        "has_error": False,
                        "field_error": "",
                    },
                )

                seen_translation_ids.add(string_translation.id)

                if not created:
                    # Update the string_translation only if it has changed
                    if string_translation.data != entry.msgstr:
                        string_translation.data = entry.msgstr
                        string_translation.translation_type = translation_type
                        string_translation.tool_name = tool_name
                        string_translation.last_translated_by = user
                        string_translation.updated_at = timezone.now()
                        string_translation.has_error = False  # reset the error flag.
                        string_translation.save()

            except TranslationContext.DoesNotExist:
                warnings.append(UnknownContext(index, entry.msgctxt))

            except String.DoesNotExist:
                warnings.append(UnknownString(index, entry.msgid))

        # Delete any translations that weren't mentioned
        if delete:
            StringTranslation.objects.filter(
                context__object_id=self.source.object_id, locale=self.target_locale
            ).exclude(id__in=seen_translation_ids).delete()

        return warnings
Nigel2392 commented 6 months ago

Relevant discussion was created before. Ref: https://github.com/wagtail/wagtail-localize/discussions/606