Database stores unchanged language strings

sumwonlost commented 6 years ago

I have a number of language issue which I have outlined numerous times the last #1789.

I am still having issues with saving changes to db and now to Merge File.

To nut this out, for now I am not concerned about main language files, I am only concerned with Plugin language files.

After installing plugin (any), I then wish to change language files.

I select Language from Main Menu and it loads the list of Languages installed then click on EDIT ICON for the Language I wish to change and loads the Languages Module.

I then select the plugin I wish to change and it CREATES a language file (language.xml) for the selected language in plugin_name/languages folder and loads Languages page for me to make EDITS.

I make the EDITS to the fields I only want EDITED (the Language String Name field background turns green and a Edit Icon appears next to String Textbox) and SAVE. This always works the 1st time I do this for ANY Language and saves ALL (both unedited and edited) languages fields for the Plugin in db. All good so far, although I cannot see the need to save UNEDITED fields.

Before going to another Language, lets Merge File firstly.

Here is issue.

I am confronted with 1 option - Write over original language file (language.xml).

If CHECKED it writes these to to language.xml in main /languages folder and NOT to languages.xml in plugin_name/languages folder?

If left UNCHECKED it CREATES a new file language-custom.xml in main languages folder.

Neither method Merges File to plugin_name/languages/languages.xml, but lets continue with the above.

Having Merged File, what is point of having these in DB? so I manually DELETE these from db as it should work from the Merged Files from above.

Having cleared both Language and Browser caches, I go to Store to see changes BUT none are shown.

So I ask a question?

Plugins Language files do not work from main languages.xml file, so why the the need to be Merged here when it does not work? Also if I Delete plugin, it does not Remove these entries from languages.xml file. So in effect Merging to here is pretty much pointless and useless. Similarly with language-custom.xml, what is its purpose, when it also does nothing?

Merge File should merge to plugin_name/languages/languages.xml which works correctly when I manually move these here to test and not to languages folder as it currently does.

There is NO need for the current Option, although I would like to see Option (checkbox) to Delete DB entries after it merges file to modules/plugins/plugin_name/languages/languages.xml which was created earlier as outlined above.

Moving on, now I want to I want to change language fields for same plugin BUT for a different language.

I follow above method but select a different language and make same same changes as 1st language, it DOES NOT always SAVE these changes to db and if Merge File was working correctly as these have not been saved to db it will not merge edited language strings to file.

Got me baffled, although I do have a more appropriate f word I can use.

sumwonlost commented 6 years ago

I also notice when it creates modules/plugins/plugin_name/languages/languages.xml that the title is same as code in language file, don't know if this matters?

abrookbanks commented 6 years ago

The merge tool is for master language files only. Not plugins. The only reason ever to use this is if you want to create a new language pack to go here. https://www.cubecart.com/extensions/languages

Are you asking for a feature request to save out plugin translations to file?

I appreciate what you have written... but what are you actually trying to do?

abrookbanks commented 6 years ago

If you are attempting to create a multilingual plugin (extension/module or whatever you want to call it) please create the languages manually with a text editor. Say for example you want Spanish and Italian you need 3 files. CubeCart won't create the files if they already exist.

modules/plugins/extension_name/language/module.definitions.xml

<?xml version="1.0" encoding="UTF-8"?>
<definitions version="1.0">
<group name="extension_name">
<string name="key"><![CDATA[Value]]></string>
</group>
</definitions>

modules/plugins/extension_name/language/es-ES.xml

<?xml version="1.0" encoding="UTF-8" ?>
<language version="1.0">
<info>
    <title>es-ES</title>
    <code>es-ES</code>
    <character_set>utf-8</character_set>
    <version>1.0.0</version>
    <minVersion>5.0.0</minVersion>
    <maxVersion>6.*.*</maxVersion>
</info>
<translation>
    <group name="extension_name">
        <string name="key"><![CDATA[Translated Value]]></string>
    </group>
</translation>
</language>

modules/plugins/extension_name/language/it-IT.xml ...

sumwonlost commented 6 years ago

What am I trying to do?

I am not trying to put all into 1 language file language.xml, I used it as shorthand to represent individual language files, so if I was using Spanish the language.xml is es-ES.xml file, if I was using Italian the language.xml would be it-IT.xml file. Sorry for confusion.

I agree there would be 3 files as per your example.

File 1 - modules/plugins/extension_name/language/module.definitions.xml your example shows 1 Phrase string, for explanation assume there are 2 Phrase strings.

The 1st Phrase is common in all 3 files The 2nd Phase is different in es-ES.xml which is different in it-IT.xml

What I am eluding to is why do I have to include both phrases in es-ES.xml and it-IT.xml files?

Why cannot I only included the Edited phrases only, similar to how CubeCart main language files work?

It should use the phases from selected language, if phrase does not exist it should then use the phrase from module.definitions.xml.

The rest is explained in email I sent you which would do away with language strings table.

abrookbanks commented 6 years ago

I expect the translation files can contain only the translated strings. Try it.

Why does it matter? They use negligible disk space.

abrookbanks commented 6 years ago

.. CubeCart only creates the files if they don't already exist. Therefore when you create an extension you need to build it with the translations included. The translated XML files only need the translated strings.

I don't think any changes are necessary.

sumwonlost commented 6 years ago

I have managed to get plugins to save to db. It is a bit long winded but it works.

Navigate Languages to plugin for selected Language and it creates the language.xml and load Phrases then save without Editing.
Clear Language cache
Navigate back to Languages to selected plugin again and save without editing, this has worked each time and saved to db.
Now make Edits and save, it saves correctly.

As example I am testing using 3 English languages (minor variations) x 2 plugins. It insert into db every phrase for the 3 languages x 2 plugins. That is 978 entries. Imagine how many entries there would be if I was using 10 languages x 10 plugins, this could run into 10,000's of entries.

What is then the purpose of the various language.xml files when these are duplicated in db?

I have only edited 48 strings, if it only saved edited fields I have reduced my db from 978 to 48 entries.

Should it not use the language.xml it created replacing the edited phrases from db???

Round and round I go and I keep coming back to what is the purpose of language.xml files?

Now I have phrases in db, I will do further testing.

abrookbanks commented 6 years ago

what is the purpose of language.xml files

The database is used to store changes to the languages. I suppose you could say that it's not necessary to store the values that are not changed. I think your argument for this is fair but I don't think it causes any performance issues and I expect your database is still very small. Is it any bigger than 10Mb? How big is the language strings table?

I'll have a look at this in the new year but it seems pretty low priority to me.

abrookbanks commented 6 years ago

I tested this and it only saves the group. So by changing one string from the main group and one from plugin put just 49 entries in the DB.

I changed the subject and I think we should code it to only save the changed ones.

sumwonlost commented 6 years ago

It's not about the size of my databases as I am in development stage and foreseeing issues. It is also about helping improve CubeCart to become better as all stores win.

I am glad you agree on this point. Now next bit of logic.

You must surely agree, if it isn't in db then it use the string from language.xml otherwise where does it get the other non-edited string from?

I assume edited language strings in db supersede those in language.xml.

Therefore if the edited string in db was Merged to language.xml by the above logic, the edited string should now come from language.xml which is superseded by the same duplicated string in db, which is now in effect redundant. Therefore on Merge File (which should also work with plugins) it should merge to language.xml and delete these from db. Now the db size has been further reduced and language.xml has been updated with edited strings. As there no longer are language strings in db for Group/Plugin for selected language, and thus the language.xml strings cannot be superseded.

Following the above, one now asks, why even have a db and why not on save, edited phrases are merged directly to language.xml?

Merry Christmas to all.

abrookbanks commented 6 years ago

why even have a db and why not on save, edited phrases are merged directly to language.xml

Because then it can't be restored to its original value as there is no record of what it was.

sumwonlost commented 6 years ago

The Edit Phrases Form loads module.definitions.xml, it compare to language.xml, different strings are replaced and highlighted. Click restore it replaces it from module.definitions.xml. The original strings are always in module.definitions.xml, similar with main files, these are in definitions.xml.

sumwonlost commented 6 years ago

To test the above,

Install plugin then

Go to Languages -> Select Language (Edit) -> Select Plugin (from dropdown), it creates the language.xml and loads language strings from I assume from module.definition.xml. Now leave page without saving (for now as Merge does not work correctly).
Next in text editor open the created language.xml and edit strings and save.
Clear languages cache. Then go to website and confirm changes to language.xml are visible.
Goto step 1. Now it does not need to create language.xml as it exist and it doesn't nor overwrite the current language.xml (good so far) and it load language strings. It does not display the edited strings from step 2, therefore the above assumption they come from module.definition.xml (there are your original values) and currently it then go to db and find saved strings and compare these to those in module.definition.xml and replace edited strings and highlight them.

Why not instead of going to db to find edited strings, it compare to language.xml and replace edited strings and highlight them. Make further edits or restore strings then Save, instead of adding to db it Merge changes to language.xml. Therefore no need for db.

Advantage, if a new string is added in future updates, you do step 4, the new string will be displayed from updated module.definition.xml but when it compare, as it does not exist in language.xml it should be highlighted, the field blank. Make edits or restore then Save. It Merge to language.xml.

Another advantage is Users are able to edit/update downloaded language.xml files and able to create and share created language.xml files for plugins.

Possible disadvantage (not sure) is that it might take slightly longer to do comparison of xml files than using db, but seriously how often would you do this once these have been done?

If you agree to here, the last issue is migration. The Merge File Tab now does as follows:

If db entries exist for selected group/module for selected language it display Tab else do not display.

On click Tab it load page as it currently does with db entries, on Save it merge these to language.xml and delete these from db. Thus the Tab should no longer display.

You could add another Tab which is displayed once all migrations have been done (0 entries in db) which clicked deletes the db (table).

abrookbanks commented 6 years ago

I'll be able to look at this early in the new year.

sumwonlost commented 6 years ago

Thanks. 1 other thing, it can be further compressed.

CubeCart main language files work if string does not exist in language.xml it use the value from definitions.xml. Modules should do same. I will explain slight variation to above once you have had a look and give me your feedback.

abrookbanks commented 6 years ago

This has been significantly improved. Please see movie here: http://static.cubecart.com/issue-1838.mp4

Closing but I will reopen is needs be.

cubecart / v6

Database stores unchanged language strings #1838