Closed romaninsh closed 5 years ago
See also #315
From my admittedly really short research, here are some things regarding gettext. If your research indicates otherwise let me know.
We should pick a solution that works everywhere. So if gettext can be missing in certain environments that is a problem. About the additional gettext library (2) in general I avoid adding too many files, except if there's a real benefit. Gettext can offer us a lot but we must see if our project will use that power. If we have a few strings that we want to translate then I think gettext might be overkill, except if there's a way for a really simple implementation.
In a non-gettext scenario here's what I would do:
$app->setLocale()
function or $app->locale
property that would be set before App
loads.App
load and depending on the locale, the correct localization file would be loaded and variables would be added to an array or variable list. In case no locale was specified, default (strings.en.php) would be loaded. We could even avoid the file load in case of default by having the default English strings within the views themselves.This system could be enhanced with sprintf()
if needed in order to change the position of certain elements. But it might not even be needed. This solution is lightweight, will work everywhere and I think it's a good fit for our project. I would also be able to implement it. Translating to another language would be as simple as copying the file, changing the fr to the language code and translate the strings within the file. Pretty simple for a PHP person.
If we're sure about gettext availability everywhere, there might be similar gettext usage scenarios that we might consider. As I haven't used gettext @abbadon1334 is in a better position to propose something.
I can provide Greek and French translations. I would not mind having my French translation inspected by a native speaker. I'm sure @abbadon1334 could provide Italian and @PhilippGrashoff German. It's a good starting point. Spanish would be a language to look for. And Chinese....
Another WALL OF TEXT is coming ( like winter )
i suggest all to read this chapter, there are some exceptions in localization that any specific library already solves and if you don't consider it and do one of your own you will have problem in the future :
IMHO i think there is more value to add third party library that solve a specific domain and solve it well, Going in this way as a good side effect, during the usage maybe we can solve an issue or collaborate on something; In place of close our code to our implementation and make us even more asocial than average devs can be :)
We need in App :
interface TranslatorInterface
{
/**
* @param string[] ...$args
* @return string
*/
public function translate(...$args): string;
}
class App
{
/** @var TranslatorInterface */
private $translator;
private function setTranslator(TranslatorInterface $translator): void
{
$this->translator = $translator;
}
public function __(...$args): string
{
return $this->translator->translate(...$args);
}
}
after this we can create the TranslatorNulled in this way, which will be the default translator. to avoid in code any checking of types.
class TranslatorNulled implements TranslatorInterface
{
/**
* @inheritDoc
*/
public function translate(...$args): string
{
return $args[0];
}
}
after add this 2 method we can start searching :
and redirect all to $this->app->__(...$args)
i think this is the solution with less code possible, that open atk4/ui to localization
After that we can create as many as we want implementation of TranslatorInterface and add support for GetText, other libraries or reinvent the wheel because all like to do it and be proud of it (included me)
But we give guidance and unbreakable restriction to do it giving an interface, make translator private and expose only translate method, force devs to be stick with this implementation.
i think that in future can be something like this:
$app = new App();
/* define translator */
$translator = new Atk4TranslatorGetText();
/* blah blah */
/* define translator */
$app->setTranslator($translator);
$app->run();
this solution is in line with Symfony Contracts TranslatorInterface.
if we think more on high level integration with atk4, there is another solution
/**
* Interface TranslatorInterface
*/
interface TranslatorInterface
{
/**
* Set language code
*
* @param string $code
*/
public function setCode(string $code): void;
/**
* Set language code to fallback in case translation is not present
*
* @param string $code
*/
public function setCodeFallback(string $code): void;
/**
* Add file to load in translations
*
* @param string $path
*/
public function addFile(string $path): void;
/**
* Call at the end of initialization
* some libraries probably need a method after all config is done
*/
public function onInitEnd(): void;
/**
* @param string[] ...$args
* @return string
*/
public function translate(...$args): string;
}
class App
{
public $translator_language = 'en'; // <-- can be protected?
public $translator_fallback = 'en'; // <-- can be protected?
/** @var TranslatorInterface */
private $translator;
/** @var string */
public $translator_class = \atk4\ui\Translator\Nulled::class; // <-- can be protected?
/** @var string[] */
public $translator_paths = []; // <-- can be protected?
/**
* @param TranslatorInterface $translator
*/
private function setTranslator(TranslatorInterface $translator): void
{
$this->translator = $translator;
/** can be only one that set code + fallback */
$this->translator->setCode($this->app->translator_language);
$this->translator->setCodeFallback($this->app->translator_fallback);
foreach ($this->app->language_file as $file)
{
$this->translator->addFile($file);
}
$this->translator->onInitEnd();
}
private function initTranslator()
{
$translator = $this->translator_class;
$this->setTranslator(new $translator());
}
public function __(...$args): string
{
return $this->translator->translate(...$args);
}
}
all is defined in the fields of Container App. If Dev extends the app and define in extended class all the needed parameters, inner methods to setup the language will be fired and do all automation needed.
$App = new App([
'translator_class' => \atk4\ui\TranslatorGettext::class,
'translator_paths' => './locale',
'translator_fallback' => 'en'
]);
$App = new UserApp(); // <-- extended App
$App->run();
N.B. but remain the problem that you cannot add something before defining layout, even if is something that is functional and not related to any kind of UI, i'm wrong?
interface TranslatorInterface
{
/**
* @param string[] ...$args
* @return string
*/
public function translate(...$args): string;
}
class App
{
public function getLanguage() : string
{
return 'language code from somewhere (config | user-session)';
}
public function __(...$args): string
{
return $this->translator->translate(...$args);
}
}
class TranslatorNulled implements TranslatorInterface {
public function translate(...$args): string
{
return $args[0];
}
}
class TranslatorReallyReallySimple implements TranslatorInterface {
use \atk4\core\AppScopeTrait;
use \atk4\core\ConfigTrait;
use \atk4\core\InitializerTrait;
public function init()
{
parent::init();
/**
* LOAD CONFIG - language
* LOAD CONFIG - language fallback
* LOAD CONFIG - language files
*/
}
public function translate(...$args): string
{
return $this->config->getConfig($args[0]);
}
}
Did you see any other ways to do this?
@abbadon1334 great research and proposal.
My proposal is a dead-simple solution which has worked for me in the past even for sites with 3-4 languages. If we want a complete translation system it's indeed better to go with a library. My only question is if we do need something that elaborate. Perhaps having it even if it's not needed is good, I guess this more is up to you and @romaninsh to discuss and decide.
Here are some comments:
My proposal is a dead-simple solution which has worked for me in the past even for sites with 3-4 languages. If we want a complete translation system it's indeed better to go with a library.
the 3rd solution, the full atk and dead simple one, is a draft of what you said, look in core/ConfigTrait is only in dev branch, is nearly all done.
Every solution i propose to implements an Interface, default for all solution it will be TranslatorNulled, that return the string without translations ( * probably we need to call sprintf to make it works with arguments).
My only question is if we do need something that elaborate. Perhaps having it even if it's not needed is good, I guess this more is up to you and @romaninsh to discuss and decide.
Is up to all who want to discuss it, i'm the last one arrived :)
In my opinion even if we go with something complete it should be easy to use to encourage translation. We should be based on an interface as you did so add-on authors can also implement it and use it. Personally I prefer non-compiled translation files because they encourage translation (you just duplicate and translate - even a non-programmer can do it).
i agree, i answer this at the end of this post.
Regarding the last note about models and persistences I don't know how you have it in your mind, but I wouldn't like to need a database just to translate 20-30 strings. If we talk about a small SQLite database with all the languages that will be distributed with ATK, then we should see which method is faster and encourages translation more. If there's a central language database, we should also see how this will work for add-ons. For example, if tomorrow I create an add-on, it's easy for me to implement an interface and create some text files with translations. However, if all translations are meant to be on a central ATK database this enforces coupling and If my add-on is not accepted as official I might have a problem. So perhaps clarifying what you meant will help us.
Anyway, i think there is a big part of people that work on MVP (atk4 is perfect for this) and probably even configure a translation system is something in plus to do.
For this purposed i propose to return directly the indexed string, because in this way if you don't want to setup the translations you don't need even the localization file.
If you want to use translations, you can substitute TranslatorNulled with another implementation or implements your own because you hate the world and want to blow up King's Landing :D
Translations
If we stick to what others do, make easier to switch to atk4, we need to find a good balance between simple and easy to be changeable because i'm sure there will be some PSR even on this, look at a beautiful and simple Framework like FatFree which is less appreciated only because has his own way of doing things.
If we want to even manage translation in a CRUD we need to find a file format that can be used in atk4/data (*) or define a atk4/Model for Translation with field_id (string) which i think is not a really bad idea, but on this i want to hear @romaninsh and @DarkSide666.
I try to think about what can be added to atk4 that already has its own translations and i have no clue about what it can be.
(*) every format i see has some multidimensional arrays format, if you want to do a good work.
Awesome. I personally have no other questions or comments and I'm sure we'll have a great localization system for ATK. @abbadon1334 thanks for all the time you've been putting into this.
@acicovic is a pleasure, i think solution 3 and your description of execution is equal, i just wait for others to discuss and after that we can start deciding what to do.
I think there is much more work to find all strings in code than implement this.
Does localization also include things like local date formats, number (dot or comma, thousand separator) formats, first day of week (Monday or Sunday) etc. ?
Does localization also include things like local date formats, number (dot or comma, thousand separator) formats, first day of week (Monday or Sunday) etc. ?
yes, but i think that localization variables will be stored directly in persistence_ui i think there is already a start there, i think that must take the right format from there for normalization/cast.
This idea of localization and translations, pops up because we speak with Sean in the last meeting about calendar date format.
Dates, currencies and date formats is where things can get tricky.
For example, I might want to have an environment in French but the currency to be dollars and the dates formatted in a yyyy-mm-dd format. So we should be able to customize this through properties or easily create our own custom localization files.
Dates, currencies and date formats is where things can get tricky.
For example, I might want to have an environment in French but the currency to be dollars and the dates formatted in a yyyy-mm-dd format. So we should be able to customize this through properties or easily create our own custom localization files.
wait..., there is a major refactoring on data cast e field type in atk4/ui and atk4/data, i think number format must be done in data, while in ui we can have field for currency that get a default format with currency sign.
look at this : https://github.com/atk4/data/pull/375
I think is best to focus on translation first, after that probably even other PR will be solved or much ahead in dev and we can work together on a better solution.
BIG NOTE : we need to consider for localization Fomantic/VUE evolution.
Notes from meeting (ATK Weekly 11)
Solution 1: only translate STRINGS generated or hard-coded in ATK namespace $form->addButton($form->app->_('Save')); // button label is translated
Solution 2: magically translates during init() or getCaption() of model/field/getCaption $form->addButton('Save'); // button will be translated
routed through $app->_();
(which MAY call _() or __() depending on the library you use)
" 25 items"
"item count: 25"
"% item(s)"
$form->addButton('Delete 25 items');
$form->addButton($app->_('Delete % item(s)', [25]));
$form->addButton(['caption'=>_('Delete % items')]);
$obj->__toString()
function App::($arg, ) { if ($arg instanceof TranslatedString) { return $arg; } return new TranslatedString(parent::($arg)); }
ATK way - Solution 2 Romans - Solution 1 Gowrav - Solution 2 Franchesco - Solution 2 Alain - Solution 2
{$id} -> converts to View->id automatically {}Welcome to my add-on{/} {$header}
_('Hello') => 'Привет'
gettext.po Hello=Привет
$model->addField('name', ['caption'=>'Name']);
2a. argument caption = should init() automatically translate? yes: gowrav, romans
2b. caption to hold english version, but getCaption to translate on the fly? yes: franchesco,
in atk4/somefile.php throw new exception([$app->_('something bad happened')]);
Model data..
Not sure if I understood everything correctly from notes above, but here are my 2 cents.
TranslatableTrait
in atk4/core and apply it to all other classes. This trait should implement _()
proxy method which calls $this->app-_()
if possible.That way we can simply call $this->_()
in any class and should not always do something like $this->app ? $this->app->_('Save') : 'Save'
And yes, I vote for using gettext(). I think it's quite ok and developers should be used to it (Wordpress). But if we separate translation engine from App class (use app only as proxy to Translator class) then developer will be able to inject his own Translator class and use something else than gettext.I'm using the Agile Toolkit, Data and so on for our company's new CMS, mostly for UI and some operations, we're not english speakers so obviously the first thing I looked into was getting translations working.
Currently, the fork I've made has a few modifications:
View
class, along with various field classes and other classes that do not inherit from it now have an additional trait, which contains two static functions responsible for text translation."Sort by"
would now be self::ts("Sort by")
So, a file like View, if it had text to translate, it'd look like this:
/atk4/ui/View/{lang_code}.json
{ "Original Text": "Translated Text" }
I've been using this solution for a while, if you're going to implement your own solution I'd ask that it's an interface and you can somehow completely redirect everything.
@pkly sure!
@DarkSide666 i like the idea of Trait, because it can be used everywhere and is not coupled only to App, and you are right about use of standalone atk4/api or atk4/data, but i think we have a problem
interface TranslatorInterface
{
/**
* @param string[] ...$args
* @return string
*/
public function translate(...$args): string;
}
class TranslatorNull implements TranslatorInterface {
public function translate(...$args): string
{
return /* return string as it is */;
}
}
trait TranslatorTrait {
public $_translator_trait = true;
protected $translatorClass = TranslatorNull::class; // <-- how to define this? we don't have any singleton, why start now?
public function translate(...$args): string
{
return /* return string translated */;
}
}
Questions :
I start a draft on core/ConfigTrait to implements not only simple configs but even instance and factory referencing them by Interface name and not Class name, like Slim3 do for ContainerDI, i will push something later and we can talk further ( config file will look something like this : https://github.com/abbadon1334/nemesi-php/blob/master/config/logger.php ).
@abbadon1334
Why not make it a class that gets passed as a parameter to /atk4/App (default: null), implements interface from /atk4/core.
If /atk4/App notices the translation class is null, simply generate the default one (sitting in /atk4/core) That way you could evade using a singleton and allow for the App class to control the translations somewhat.
@abbadon1334
Why not make it a class that gets passed as a parameter to /atk4/App (default: null), implements interface from /atk4/core.
If /atk4/App notices the translation class is null, simply generate the default one (sitting in /atk4/core) That way you could evade using a singleton and allow for the App class to control the translations somewhat.
as i said the problem is not how to pass to an App that use TranslateTrait, like you already said, atk4 will do the magic just in this way :
new App(['TranslationClass' => $TranslationClass]);
// After this all elements will have a reference $element->app and access to relative method @__
I just point my finger on usages of atk4/core/TranslatorTrait other than ui like api | data | reports etc.. etc.. UI implementation is easy because all of us already have done it custom and already know the real cases
Probably i put too much thinking but i prefer to put on the table every situation and possible use when we are still analyze and probably "lose" time.
Well, translator trait or class or whatever shouldn't be dependent on App because not always we will have app. But if we implement it as trait then we can add trait to any php class.
Even more, we can have 2 Views in same UI page and each View text could be localized in separate language. That would be awesome.
And not sure if that's possible somehow, but by default if App or View have localization trait set, then maybe it could pass that localization trait/class to any child object you add with $object->add() if that child object don't already have translator linked.
Well, translator trait or class or whatever shouldn't be dependent on App because not always we will have app. But if we implement it as trait then we can add trait to any php class.
agree
Even more, we can have 2 Views in same UI page and each View text could be localized in separate language. That would be awesome.
this is what i try to figure out, because all framework work as a container and can contains more than 1 app, is stated multiple times in the documentation and i don't want to break this interesting feature
And not sure if that's possible somehow, but by default if App or View have localization trait set, then maybe it could pass that localization trait/class to any child object you add with $object->add() if that child object don't already have translator linked.
I have a little fear that we need to implement something on our own because from what i have seen gettext and all library around it use gettext like a static reference object scoped to php because it is called at SO level, someone can try to find info regarding this?
PR wip in atk4/core => https://github.com/atk4/core/pull/93 Waiting feedback
@abbadon1334 I looked at the changes you've proposed but I don't really understand why you'd have that last part that loops over arguments to simplify sprintf. I don't believe that using something like
$this->caption = sprintf($this->_("This is some %s thing"), "aaaa");
is really that bothersome, it also helps to keep things simple.
You should probably just keep it to string $string, ?int $plural = null
I'm also unsure if having the plural case is absolutely needed, but I don't think it's a problem. The translation system I've made for my project uses a different approach, but similar, it uses the class name to match the string to a file, which allows for a few things:
I know that atk4 doesn't really use singletons, so maybe it's not the best approach.
If you go ahead and implement it as a trait it should be alright, since most of the strings are in the classes and not really in objects from what I've seen, so I can modify it to call my translation system with minimal changes if required.
I could also provide a Polish translation for the whole UI/Code/Data/DSQL set of libraries, since that's what I'm currently depending on in my project.
1st : some language can have more than 1 plural form 2nd : sometimes you need even zero form, not only singular and plural
we don't want to use singleton, if we use singleton we lose one of the best thing in atk, i mean the possibility of running multiple app as independent "containers of definitions", the code it's just a base to discuss but i think we need to avoid direct calls to sprintf. I think to help user sometimes in atk4 we allow too many ways to accomplish the same result, i try in translation to give only one way to do things to stop disorienting users, i think you notice that i marked final the default class because i want the user to implements the interface and not extend our specific implementation.
i have some work to do, but in a few days i will write some examples of using the translation system, to further discuss about it, the idea is to remain simple, it will be something like this :
but i will be much specific with some examples in a few days
Sure, keeping multiple forms for plurals and zero is fine, but I don't really get the whole extra sprintf usage.
And like I said, I also don't think that putting a singleton into atk4 is a good idea, since it makes no real sense and provides barely any benefits.
the idea is to make nested translation possible, can be wrong and can be unuseful, i think only when we use it or figure out the usage possibilities we can see if it is need or not.
you show this case :
$this->caption = sprintf($this->_("This is some %s thing"), "aaaa");
why not :
$this->caption = $this->_("This is some %s thing", ["aaaa"]);
and in the method _ it will do something like :
$caption_translated = $this->_("This is some %s thing"); // if not present it return the string as it is
$aaaa_translated = $this->_("aaaa"); // if not present it return the string as it is
return sprintf($caption_translated,$aaaa_translated);
in this way if there is a translation for aaaa it will be translated too.
i update the PR https://github.com/atk4/core/pull/93 , can you take a look @pkly ? you were right if we start do that sprintf we enter in an hell because we need to include even context of translation :D
i add tag hangout agenda, because i need to know some opinions, if i have time i will prepare some case usage to discuss it.
@abbadon1334 I looked through it, besides that one exception that I don't like I've also listed a suggested implementation idea in the comments, although I'm unsure if it's something that you'd be for.
The question is, do you want to have translations outside of UI, if not, I believe that my idea is not half bad - that way you only rely on dummy traits in Atk4/core, and when used together with Atk4/ui it'd simply switch to it.
I can write a usage example of my idea, if you'd like.
Translation mechanism should be usable outside of app and UI.
For example, if I only use atk4\data, but not atk4\ui then I should be able to use translated exception messages coming from atk4\data. Theoretically I can use atk4\data or atk4\dsql alone in some other UI framework.
We've also talked about it on the atk/core pr listed above, which would make all the elements use a trait and a translation class, which should support using core + data (without ui)
Closing this. Further discussion is in : https://github.com/atk4/core/pull/93
After merged, we can implement a follow-up PR here to take advantage of new traits.
Many of our users need non-english version of ATK. Also we need a guide for external add-ons, on how they should approach translations.
For now - need answers to the above questions. To be discussed in next hangout.