search

FreeCAD / FreeCAD-translations

Repository tracking localization issues and progress
5 stars 3 forks source link

[Discussion] Crowdin and github integration. #119

Open Kuzma30 opened 2 years ago

Kuzma30 commented 2 years ago

I propose to discuss the integration of translation updates. I've tried something, and there are some results. What I did

  1. Created an empty mail for tests, I can provide login and password if needed

  2. Under this email I created a test project on https://crowdin.com/freecadtest. Tests decided to make on fork https://github.com/Kuzma30/FreeCAD. Tests conducted in one language (Ukrainian) and on one file (Spreadsheet). The configuration file https://github.com/Kuzma30/FreeCAD/blob/master/crowdin.yml was created (online when the integration was connected) and supplemented. After synchronization, I received partially translated file (translation was obtained automatically from Spreadsheet_uk.ts file from github. What are the problems when importing a) Loss of translation author b) loss of Approved status c) loss of translated variants This error may be solved (I think) with translation memory. I can't test this on empty project.. The following errors were also noticed. For example https://github.com/FreeCAD/FreeCAD/blob/f8ae185c15480b1b6cc4209d33a8e726d3a7ba80/src/Mod/Spreadsheet/Gui/Resources/translations/Spreadsheet.ts#L662 Line 

    <source>&lt;html&gt;&lt;head/&gt;&lt;body&gt;&lt;p&gt;The symbol is used to separate lines, usually a single quote (&apos;) or a double quote (&amp;quot;). Must be a single character.&lt;/p&gt;&lt;/body&gt;&lt;/html&gt;</source>

    On the server it (https://crowdin.com/translate/freecad/27907/en-uk?filter=basic&value=0#q=Character%20used%20to%20delimit%20strings) looks like

    <html><head/><body><p>The character used to separate lines, usually a single quote (') or a double quote (&quot;). It must be a single character.</p></body></html>.

    Accordingly, a translation was made for this line. On the test server (https://crowdin.com/translate/freecadtest/88/en-uk?filter=basic&value=3#q=Character%20used%20to%20delimit%20strings,%20) it looks like in the original .ts file. The problem is that in this case translation for such strings is lost.

  3. Then on local machine I run command to update Spreadsheet.ts. New lines with untranslated text appeared in the file. They also appeared on the translation server after the synchronization.

  4. Crowdin create pull request in separate branch. Then it can be merged.

  5. For automation update .ts files on github and auto create .qm after update I think we can use Actions mechanism ( test this later)

luzpaz commented 2 years ago

CC @yorikvanhavre

Kuzma30 commented 2 years ago
  1. Looks like official script start work better and write "more".
  2. Is there official script for making .qm files?
Kuzma30 commented 2 years ago

A few words about commits from crowdin. In the settings, you can choose the frequency of commits from an hour to one day. Commits are created in a separate branch. Created separately for each language and translation file. Then they can be squashed into one through the github menu. This has been tested on my fork.

yorikvanhavre commented 2 years ago

I'm all for using the github integration system too, but there are several things we need to check too: 1) what happens the day we move to another platform: gitlab seems ok for now, but what if we have our own git server? 2) qm files should indeed be removed from source and built at build time. Probably we need a cmake macro for that. Maybe that could be a first point to address, as it doesn't depend from the crowdn stuff?

I'll have a look at your fork ASAP...

Kuzma30 commented 2 years ago
  1. In this situation we can use https://support.crowdin.com/cli-tool/. You can disable github integration at any time without lost of translations memory.
  2. I am thinking on it and also wanted to suggest this. I will try realize this, but I am bad programmer ;) I can modify my version of script, but I don't know how to run it during cmake/build.
Kuzma30 commented 2 years ago

I modify updatets.py to run lrelease in directories. Its works on local machine. But I don't know how to run it during build or configure stage (with cmake).

cat updateqm.py 
#!/usr/bin/python3

# -*- coding: utf-8 -*-
# (c) 2010 Werner Mayer LGPL

#***************************************************************************
#*                                                                         *
#*   Copyright (c) 2010 Werner Mayer <wmayer@users.sourceforge.net>        *
#*                                                                         *
#*   This program is free software; you can redistribute it and/or modify  *
#*   it under the terms of the GNU Library General Public License (LGPL)   *
#*   as published by the Free Software Foundation; either version 2 of     *
#*   the License, or (at your option) any later version.                   *
#*   for detail see the LICENCE text file.                                 *
#*                                                                         *
#*   This program is distributed in the hope that it will be useful,       *
#*   but WITHOUT ANY WARRANTY; without even the implied warranty of        *
#*   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the         *
#*   GNU Library General Public License for more details.                  *
#*                                                                         *
#*   You should have received a copy of the GNU Library General Public     *
#*   License along with this program; if not, write to the Free Software   *
#*   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  *
#*   USA                                                                   *
#*                                                                         *
#***************************************************************************

# Changelog:
# 0.1 Initial Release, based on updatets script

from __future__ import print_function

Usage = """updateqm - run lrelease for all .ts files found in the source directories

Usage:
   updateqm

Authors:
  (c) 2022 FreeCAD Volunteers
  Licence: LGPL

Version:
  0.1
"""

import os, sys

directories = [
        {"tsname":"FreeCAD", "workingdir":"./src/Gui/Language"},
        {"tsname":"AddonManager", "workingdir":"./src/Mod/AddonManager/Resources/translations"},
        {"tsname":"Arch", "workingdir":"./src/Mod/Arch/Resources/translations"},
        {"tsname":"Draft", "workingdir":"./src/Mod/Draft/Resources/translations"},
        {"tsname":"Drawing", "workingdir":"./src/Mod/Drawing/Gui/Resources/translations"},
        {"tsname":"Fem", "workingdir":"./src/Mod/Fem/Gui/Resources/translations"},
        {"tsname":"Image", "workingdir":"./src/Mod/Image/Gui/Resources/translations"},
        {"tsname":"Mesh", "workingdir":"./src/Mod/Mesh/Gui/Resources/translations"},
        {"tsname":"MeshPart", "workingdir":"./src/Mod/MeshPart/Gui/Resources/translations"},
        {"tsname":"OpenSCAD", "workingdir":"./src/Mod/OpenSCAD/Resources/translations"},
        {"tsname":"PartDesign", "workingdir":"./src/Mod/PartDesign/Gui/Resources/translations"},
        {"tsname":"Part", "workingdir":"./src/Mod/Part/Gui/Resources/translations"},
        {"tsname":"Path", "workingdir":"./src/Mod/Path/Gui/Resources/translations"},
        {"tsname":"Points", "workingdir":"./src/Mod/Points/Gui/Resources/translations"},
        {"tsname":"Raytracing", "workingdir":"./src/Mod/Raytracing/Gui/Resources/translations"},
        {"tsname":"ReverseEngineering", "workingdir":"./src/Mod/ReverseEngineering/Gui/Resources/translations"},
        {"tsname":"Robot", "workingdir":"./src/Mod/Robot/Gui/Resources/translations"},
        {"tsname":"Sketcher", "workingdir":"./src/Mod/Sketcher/Gui/Resources/translations"},
        {"tsname":"Spreadsheet", "workingdir":"./src/Mod/Spreadsheet/Gui/Resources/translations"},
        {"tsname":"Start", "workingdir":"./src/Mod/Start/Gui/Resources/translations"},
        {"tsname":"TechDraw", "workingdir":"./src/Mod/TechDraw/Gui/Resources/translations"},
        {"tsname":"Test", "workingdir":"./src/Mod/Test/Gui/Resources/translations"},
        {"tsname":"Tux", "workingdir":"./src/Mod/Tux/Resources/translations"},
        {"tsname":"Web", "workingdir":"./src/Mod/Web/Gui/Resources/translations"}
        ]

LRELEASE = ""

def find_tools():

    print(Usage + "\nFirst, lets find lrelease tool on your system")
    global LRELEASE
    if (os.system("lrelease -version") == 0):
        LRELEASE = "lrelease"
    elif (os.system("lrelease-qt5 -version") == 0):
        LRELEASE = "lrelease-qt5"
    else:
        raise Exception("Cannot find lrelease")
    print("\nQt tool lrelease have been found!\n",
          "\t" + LRELEASE   + "\n")
    print("==============================================\n")

def make_qmfiles(entry):

    global LRELEASE
    print ("\n\n=============================================")
    print (f"MAKING QM files FOR {entry['tsname']}")
    print ("=============================================")
    cur = os.getcwd()
    log_redirect = f" 2>> {cur}/qm_create_stderr.log 1>> {cur}/qm_create_stdout.log"
    os.chdir(entry["workingdir"])

    execline = []
    execline.append (f"{LRELEASE} *.ts {log_redirect}")
    print(f"Executing commands in {entry['workingdir']}:")
    for line in execline:
        print (line)
        os.system(line)
    print()
    os.chdir(cur)

def main(mod=None):

    find_tools()
    path = os.path.realpath(__file__)
    path = os.path.dirname(path)
    os.chdir(path)
    os.chdir("..")
    os.chdir("..")
    print("\n\n\n BEGINNING MAKING QM FILES\n\n")
    if mod:
        for i in directories:
            if i["tsname"] == mod:
                print("WARNING - Running QT lrelease for ",mod,"ONLY")
                make_qmfiles(i)
                break
    else:
        for i in directories:
            make_qmfiles(i)
    print("\nIf updateqm.py was run successfully. Now you have latest translations ")
    print("stderr output from lrelease can be found in qm_create_stderr.log")
    print("stdout output from lrelease can be found in qm_create_stdout.log")

if __name__ == "__main__":
    if len(sys.argv[1:]) > 0:
        main(sys.argv[1])
    else:
        main()
Kuzma30 commented 2 years ago

CC @yorikvanhavre Can I make pull request with this version of script?

Kuzma30 commented 2 years ago 
2. qm files should indeed be removed from source and built at build time. Probably we need a cmake macro for that. Maybe that could be a first point to address, as it doesn't depend from the crowdn stuff?

Done. https://github.com/Kuzma30/FreeCAD/commits/qm_create_during_configure ping @yorikvanhavre

Kuzma30 commented 2 years ago

@yorikvanhavre can you just update .ts files and Crowdin strings (In the usual way)? According to my test crowdin server looks like it will be 1k+ new string to translate. May be translator can do this before v0.20 release?

yorikvanhavre commented 2 years ago

I think qm files should be built through a cmake macro or function, I am not sure it can use your py script. In any case, it's best to do this after the 0.20 release, as we don't add new features at the moment. We're still fixing details with the translations scripts with @chennes, I'll do a new crowdin merge ASAP.

kaktusus commented 2 years ago

We all look forward to the Crowdin update

The new 0.2 release has a chance to stop being a hole and have nicely translated user interfaces in many languages.

Kuzma30 commented 2 years ago

I think qm files should be built through a cmake macro or function, I am not sure it can use your py script. In any case, it's best to do this after the 0.20 release, as we don't add new features at the moment. We're still fixing details with the translations scripts with @chennes, I'll do a new crowdin merge ASAP.

In comment with link to fork you can see how to run script during cmake configure. I test this on my local Ubuntu 22 mashine.

yorikvanhavre commented 2 years ago

Ok I'll have a look!

andrii-bodnar commented 2 years ago

Hey everyone,

There is a Crowdin GitHub Action that allows you to easily automate source files pushing to Crowdin and translations download.

I can do a pull request with the integration setup. I would only need a green light from you and the minimal setup of this repo (create a Crowdin project and add some Actions secrets).

The Action is based on Crowdin CLI, so the existing crowdin.yml config can be reused.

luzpaz commented 2 years ago

Thank @andrii-bodnar we appreciate the volunteering. Out of curiosity is there a way to do dry-run testing?

Kuzma30 commented 2 years ago

@yorikvanhavre As we have 0.20 release, what is your decision with github integration? Did you test script for creation of qm files? https://github.com/Kuzma30/FreeCAD/commit/6cda6e3dd50f1532e9b57c3a2c57120231180dbf

yorikvanhavre commented 2 years ago

Let's wait until the release is complete please :sweat_smile: I'll attack that ASAP

Kuzma30 commented 2 years ago

@yorikvanhavre I test adding tags and changing default commit text.

pull_request_title: '[skip ci] Crowdin upload'
pull_request_labels:
  - crowdin
  - l10n

Adding this to config file (https://github.com/Kuzma30/FreeCAD/blob/master/crowdin.yml) produce such PR https://github.com/Kuzma30/FreeCAD/pulls I also test how to save approved translation. (I think it will be work for your too) I change in settings Duplicate Strings to Show, but auto-translate them Then, when I add new file to translate (from realthunder branch) it save translations and approve status.

Kuzma30 commented 2 years ago

@chennes Please, can you test this script on Windows? https://github.com/Kuzma30/FreeCAD/commit/6cda6e3dd50f1532e9b57c3a2c57120231180dbf

Idea - creating qm files during cmake configure process

chennes commented 2 years ago

It doesn't work if you don't have the official Python installed from the Microsoft Store -- you need to modify your command to use the version of Python that the configuration found.

Kuzma30 commented 2 years ago

It doesn't work if you don't have the official Python installed from the Microsoft Store -- you need to modify your command to use the version of Python that the configuration found.

Like last updatets.py version?

chennes commented 2 years ago

I don't know what you mean: cMake will have found some version of Python, your command should use that one, not try to use whatever is in PATH

Kuzma30 commented 2 years ago

@chennes Made small modification, please test (on Windows) https://github.com/FreeCAD/FreeCAD/commit/0f6f3ccb663d83f74430213c032f5f2f252a8d06 If this will work I made some cleanup to script and add new directory to list.

Kuzma30 commented 2 years ago

Let's wait until the release is complete please sweat_smile I'll attack that ASAP

@yorikvanhavre Is v.20 release complete?

Kuzma30 commented 2 years ago

@yorikvanhavre Still waiting (for feedback as minimum) 235789_600