Replace content translation with entity translation (and add an upgrade path)

[SebCorbin]

Once #49 gets in, what stopping us from using entity translation instead of content translation

Pros: unified way of handling translation (nodes, taxonomy, ...) no need of i18n

Cons: still hard to manage translations of menu, block and other things

---

Related

• https://github.com/backdrop/backdrop-issues/issues/4198 Content translation improvements

As people weigh in here with reasons they prefer (or were forced) to use entity translation, I would love if it we could add those reasons to that issue too -- since many of those improvements will be able to be included before 2025 :)

[rudiedirkx]

Content translation and entity/field translation are completely different things. You want want completely different nodes for different languages (content translation). Or you might want the same nodes with only different body texts (entity/field translation).

Entity translation should have something for a translatable title though.

[jenlampton]

I don't think we want to go the way of entity/field translation for Backdrop. I think different nodes (or users, or terms, depending on what we are translating) is going to be the preferred method for content translation - at least in core. If people really want field translation then current D7 contrib can be made backdrop compatible, and we can re-evaluate for backdrop 2.x

[joejoseph00]

All of the Drupal 7 sites I have built for large organisations are using the entity_translation module which is now stable at version 1.0, is very well supported by contrib (search api and solr, media, file_entity) has extensive test coverage. It also has an upgrade module entity_translation_upgrade which converts node translation content into entity_translation content (works very well, but does require reading the README.txt file, updating views and node title to title field using the 'title' module.

Entity translation is the preferred way.

There's also third party modules like the entity_translation_unified_form module that allows entering two or more languages of node content all in one edit page using the same interface language for two or more languages.

[jenlampton]

Thanks for the feedback. Can you let us know why content translation did not work for your use case?

[joejoseph00]

Hi Jenlampton, long-winded answer: my clients started with content translation, but I upgraded all of their content to Entity Translation using the entity_translation_upgrade module that is included with Entity Translation.

The reasons for such an upgrade were multifold, the government distribution of Drupal that most of my clients are using chose to switch from content translation to Entity Translation in about 2014, no longer supporting content translation in that distribution.

There were reasons for this choice: Content translation separates comments between languages because there is a node for each language. Entity Translation makes it easy to do both ways, you can have everyones comments show up for all languages OR comments per language.

Also, entity_translation_unified_form provides a user friendly content editing experience for multilingual content, it only works with Entity Translation, does not work with content translation.

Also, content translation does not take advantage of the field translation capabilities. entity_metadata_wrapper makes it easy to create and read multilingual nodes programmatically, especially makes sense when using entity_translation.

also, the whole notion of entity is based on having field translation, always talking the same entity , whereas content translation doesn't really fit that definition.

Entity Translation is now very well supported by Views and the rest of contrib.

I along with several of my peers spent countless hours contributing to contrib modules to push up fixes and patches that are now in contrib module releases.

Now, it is possible to use Entity Translation and expect everything to work perfectly provided you follow the README.txt in the related modules .

[mazzech]

I just finished my first multilingual project in Backdrop CMS, content translation works pretty well for brochure sites. I even use it nowadays in D7 projects, sometimes having dedicated nodes per language is simply better (e.g. Landing Pages). However, entity translation is much better whenever it comes to reference any kind of entities. For example, if an event calendar has a node for event description and a node per date (recurring events). It also better for any kind of signup, ordering, commerce (even for a small number of products without payment). But as said, I still like to have both options in D7.

[klonos]

FWIW, I also agree that entity_translation is more robust, and that it does work better for more complex sites. I still think that it is a 2.0 subject though, and to that end, I am happy that there is already an upgrade path that has been field-tested and works well. This means that most likely there'll be less work for us when it comes time to make a decision.

In the meantime, I think that 1st step would be porting it to contrib. We should then watch for usage stats.

I also think that things evolve organically. If Backdrop ends up having https://github.com/backdrop-contrib/reference in core, and if entity_translation works better with it, then things will happen 😄.

My 2c

[joejoseph00]

Yes, entity referencing is easier with entity_translation especially in views.

although yes, it's nice to have both options, they both work well.

[olafgrabienski]

Interesting! @joejoseph00, do you have any plans to port the "Entity translation" module to Backdrop?

In contrast to the recent posts, the issue was opened to discuss the removal of content translation in favor of entity translation, which isn't a realistic option as far as I know. Should we close this issue?

[joejoseph00]

olafgrabienski, I agree, wouldn't make sense to remove content translation, they both can exist. I don't have any current plans to port Entity Translation over to Backdrop but I could maybe join in on the effort at some point.

[matthewv789]

I haven't used Backdrop (but am considering after some frustrations with D8, and liking your whole philosophy much better), but would also vote for entity translation (per-field translation). The site I am working on is multi-lingual, and in each node there are fields I want to translate, and fields I don't. Dates, entity references, images... they are all shared, and shouldn't be translated, so in the content type they are defined as not translatable. (The entity references, of course, refer to the same language version of the referenced entity, automatically.) If I ever update those items, I don't want to have to do it twice, or end up with one or more languages out of date or inconsistent because I forgot to make all the same changes there. Multilingual I think is one of Drupal 8's strongest points compared to other systems, so that's one area it could be looked to for inspiration.

[jenlampton]

I don't see how that is any less likely if you have to translate 10 separate things instead of only one. Wouldn't entity translation make that problem much worse?

[joejoseph00]

jenlampton, for content translation there is the i18n_sync module (submodule of i18n) that does mitigate some of that , it allows configuring fields as synced between translation sets so it's only shown on the source language.

With that said, it was a monstrous effort to get entity_translation to fully work with D7 and D7 contrib, contrib finally caught up and it's very polished now.

There's a module called entity_translation_unified_form that makes content authoring very convenient for bilingual or multilingual sites.

However, content translation does work well also. To go into the details of the pros and cons of both approaches would take quite an in-depth analysis.

Here is a very basic explanation: https://evolvingweb.ca/blog/content-translation-drupal-7

another basic explanation https://www.o8.agency/blog/drupal-multilingual-challenges-and-considerations

comments from 2014 about D7 issues with entity_translation are things from the past, contrib has caught up now and entity_translation gets first class treatment in D7, it works, although you'll have to find a good documentation on setting it up.

it's supported by pathauto_i18n , the title module, metatags , features , facetapi_i18n and more

[jenlampton]

@joejoseph00 it sounds like most of the problems with UX complexity have been solved by contrib solutions (which have sorted out their kinks in D7). But less-confusing interfaces don't entirely make up for an overly-complex underlying system. I'm still trying to understand a use-case where the more complex system is necessary.

in each node there are fields I want to translate, and fields I don't. Dates, entity references, images... they are all shared, and shouldn't be translated, so in the content type they are defined as not translatable.

If we include support for un-translated fields in a content-translation system the end result would be the same.

The entity references, of course, refer to the same language version of the referenced entity, automatically.

This should behave the same way with a content-translation system.

If I ever update [shared and untranslated fields], I don't want to have to do it twice or end up with one or more languages out of date or inconsistent because I forgot to make all the same changes there.

It seems to me that having 100 separate things to translate instead of a single node would make this problem worse, and not better. It's been a long time since I've done a complex multi-lingual site. Can you post a screenshot of the translation page for one of your sites that's using entity translation? Maybe that would help :)

[joejoseph00]

@jenlampton I will give you a few realworld use cases.

If you have field translation with entity_translation and you're creating a view with a taxonomy term as a contextual filter you can create just one view for all languages. Whereas with content translation you have to create a view for each translation of the taxonomy term and the queries get complicated if it's a contextual filter to another node entity you have to negotiate through a tnid , it starts being a bit of a site builder issue.

Now, I'm not saying you should do entity_translation because it was an insane collaborative effort and thousands of man hours of programmers from around the world over several years to work out the kinks , it wasn't until about 2014 where a skilled developer could patch the modules to make it work very well, by 2015 you could get by without patches, by 2017 nearly all the bugs were worked out and the support for this is what I call "prestine" , it's so prestine that the newer major releases haven't caught up to D7 yet. I've found core bugs in newer major releases that were resolved in D7 years ago. Look, you can get everything done in content translation, it works, but it's sometimes clunky when it gets to the nitty gritty details for site builders. I've done a lot of multilingual site building and these days it's quite easy. however ya, content translation works too, just not in the same way. field translation is the expression of entity, an entity has fields and the fields have translations not the entities themselves, all the same entity id, a lot easier for sql query building, developing without having to deal with a tnid which is duplicating information about the entity.

I have a lot of government clients that are required to have everything in two languages where there's basically no difference between the two, on the admin side and on the client side. Take a tag cloud for instance, if you want all related documents to show up for that tag cloud in either language with entity translation the entity it'self can be configured to have the same term reference, the taxonomy term is also field translated so no need to have a different term id for a translated term. The term will show up in both languages.

If you do this in content translation you've really gotta be more careful about how the vocabularies are set up and eventually somehow you're going to be dealing with a tnid to find a reference to something else.

also, the entity api , with metadata wrapper, it's super easy to use, no need to use fancy array indexes for languages to get translations, afaik, entity api metadata wrapper is especially optimised for entity translation. It's basically the same thing that's in D8 except the D7 entity api is maybe a bit simpler to use, not always great documentation available for entity wrapper using entity api but it's pretty good.

really to fully appreciate both content translation and entity_translation you'd have to go through building a complex sites with both . I recall a lot of the pain I went through with entity translation was due to bugs and maybe not reading the available documentation, the bugs which are all now fixed for the most part, I haven't come accross any show stopping bugs in D7 entity_translation. Again, try out the entity_translation_unified_form module, someone I know wrote it and I was the one who encouraged him to share it. There's D7 and now a D8 version available.

[indigoxela]

If I get this thread right, it's suggested to have both. We could port entity_translation as a contrib module and see, what our users / site builders prefer - content translation or entity translation.

It doesn't seem necessary at all to remove content translation, entity translation is just another option to set things up (I struggled with entity translation in D8 btw, not everything is golden with it).

The D7 module page reports 44,442 sites using it. That seems an indication of quality to me. We just need someone to port it. ;-)

The one thing I worry about is this issue's title. "Remove in favor of" might not be the best wording to describe the actual discussion going on here. "Adding an option"... might be better.

Not every website needs internationalization, so it's fine to have these modules in contrib.

[olafgrabienski]

the issue was opened to discuss the removal of content translation in favor of entity translation, which isn't a realistic option https://github.com/backdrop/backdrop-issues/issues/52#issuecomment-458053252

I agree, wouldn't make sense to remove content translation, they both can exist https://github.com/backdrop/backdrop-issues/issues/52#issuecomment-458147059

The one thing I worry about is this issue's title. "Remove in favor of" might not be the best wording to describe the actual discussion going on here. "Adding an option"... might be better https://github.com/backdrop/backdrop-issues/issues/52#issuecomment-549707564

Based on the above mentioned statements, I'm going to change the issue title.

[mazzech]

If we include support for un-translated fields in a content-translation system the end result would be the same.

@jenlampton so it would be possible to mark certain fields as untranslatable? I assume image fields could be an important use case, here's one of my D7 multilingual sites which I will go either into D8 or Backdrop in 2021;-) Same page, two languages

https://michelitc.com/de/produkte/displays https://michelitc.com/en/products/displays

Currently, if the customer wants to add an additional image at the bottom, he simply uploads one. Because I marked images as non-translatable in Entity Translation. If content translation could support this use case somehow, this would be a huge step forward.

Entity Translation shines when it comes to creating relations, however, I still use Content Translation for some sites.

[joejoseph00]

For content translation field sync between translation sets there is a submodule in the i18n project called i18n_sync , configure this and this will do the trick. Works well, check and see if i18n_sync is available for backdrop if not probably easy to port it over.

[findlabnet]

If image field become untranslatable - what about tags for this image such as 'alt' or 'title'? In example by @mazzech tags lefts blank, but it not seems as best practice.

[jenlampton]

it would be possible to mark certain fields as untranslatable?

Yes, it's possible... probably even right now. @findlabnet it wouldn't be forced -- just an option. If you want to translate images or reference fields you can, but for those who don't want to they could be excluded from the node edit form.

[jenlampton]

Thank you for the detailed explanation! This is fantastic :)

a view with a taxonomy term as a contextual filter

Aha -- lightbulb! Thank you @joejoseph00. It might be possible to solve this problem using a global 'language' context. We could have one ID in the URL, but another that gets passed into things like views and layouts, we may be able to avoid any complicated handling in those systems themselves.... but I don't want to swear by that until we have a chance to try it and see :)

The one thing I worry about is this issue's title. "Remove in favor of" might not be the best wording to describe the actual discussion going on here. "Adding an option"... might be better

We aren't going to support two different translation systems in core. We need to make a decision, do one -- and then do it really well. Right now "the option" to use entity translation exists, it is installing a contrib module. :) If we switch to entity translation in core, then the option to use content translation will be to install a contrib module.

the issue was opened to discuss the removal of content translation in favor of entity translation, which isn't a realistic option

It could be, as long as we provide an upgrade path. (so changing the title back). It sounds like there's already an upgrade path built for moving from Content translation to Entity translation in contrib. But as @klonos mentioned, this would be something that needs to happen in 2.x, so we're talking not until 2024.

I conceptually prefer having a simpler, more stable solution in core, and allowing those with more complicated needs to install a more complicated contrib solution. However, I want to make sure that our "simpler more stable solution" meets the needs of at least 80% of the sites that need translation -- and I'm pretty sure that right now content translation is not enough.

I am fairly certain we can get to the 80% mark with content translation by adding some of the features we've discussed here, things that entity translation already provides. Plus, many of these improvements can be done right now, in 1.x.

I'd love to hear more use-cases where people feel they are hitting the limits of what content translation supports. I've pulled a few things out of the above comment thread and added them to this new issue for 1.x. But let's keep adding things.

[joejoseph00]

@jenlampton I highly recommend not pursuing entity_translation , purely out of pragmatic reasons. content translation works well and is supported by existing backdrop modules.

A move to entity_translation is a huge endeavor , it's not simply a module, it's a set of modules and support from contrib as well, it was partially implemented in core as well. Also, it was several years to get this working in contrib. There's not just the entity_translation module but also the title module (for fieldable titles) and the migrate module, plus documentation, the entity_api, entity_translation_unified_form (however could port this to content translation).

There's nothing 'wrong' with content translation per say. It's two approaches, in the end, it's about delivering content, both do the job. The approaches to site building are a bit different in each approach. content translation 'done right' is a lot better than a botched entity_translation.

If you're seriously determined to go with entity_translation I'd recommend sticking with D7 (LTS or a fork of a recent D7) or going to D8. Otherwise, content translation does the job.

just be sure, do not under estimate how big of a job it would be to port all this over and then once you get someone like me in there doing real world apps , if it's not prestine, I'd probably have to switch back to content translation anyway. I only helped with entity_translation patches because my clients were paying for this and these clients had big pockets, it was 'just barely good enough' at the time, and I spent a lot of time making it 'better' , ie bug fixes over a period of several years. Now it's 'prestine' good in D7 thanks mostly to Stefanos and Plach writing huge amounts of simpletests.

[jenlampton]

@joejoseph00 we've got some time to figure it out. I'd love to see entity_translation and all the supporting modules ported to Backdrop, but if we can improve the core content translation too, we'll have some good choices when it comes time for 2.x.

[cptX]

Please don't make entity translation instead of node translation. D8/D9 went this way and now they are lost in a million new problems. This actually is the reason for me coming to Backdrop. Entity translation broke many basic things in a multilingual site!!! At the moment the comment system of multilingual D9 is trash. They just don't work. Look here how many problems I found in D9 regarding comments and I spent almost a month trying to solve them but without success https://www.drupal.org/project/drupal/issues/3196886. The issue is so major that I left D9 for Backdrop because of this!!! When you have node translation you have the comments of that node automatically, the counter works, the comments are filtered by language by definition. In D9 every part of a page (entity) is translated separately. Beyond the complexity that this adds by making a different language version of every entity, you get thousand of problems with comments. Here are some of them I have discovered in D9:

1. Not able to filter comments per language
2. If you write a comment in a translated page you get redirected to the non-translated page when you save
3. Previewing a comment is not working if you are in a translated page (white screen of death)
4. Deletion of a comment in a translated page redirects you back to the non-translated one
5. Number of comments shows the sum for all languages not the ones corresponding to the current language (this should be addressed together with 1.). Also this sum is shown only in the non translated page. In the translated page no counter is shown.
6. Removing title from comments without changing twig files. Other frameworks have no title in comments so this should be an option.
7. Permalinks sometimes include the language code and sometimes not, often leading to "page not found" errors. Node comments behave differently compared to Forum comments regarding language code in permalinks.

So please reject the idea to move to entity translation. The implications are so complex and huge that nobody can predict them. I'm 100% sure D9 will take at least 10 more years to solve them. The above issues are already 4,5 years old and not solved. Actually some of them are not solvable without breaking again the backward compatiblity. I have openned issues that nobody every answered yet. So I decided to come to Backdrop where node translation is the way and all the above issues are solved!!

[indigoxela]

Please don't make entity translation instead of node translation. D8/D9 went this way and now they are lost in a million new problems.

@cptX you might have noticed that there's not much activity in this issue, nobody's actually working on this. All efforts regarding internationalization go to the "traditional" (or should I say "robust") way D7 is also using. And Backdrop would never recklessly do breaking changes like replacing the complete translation workflow, as backwards compatibility is one of the cornerstones of B's philosophy.

It's still possible to discuss ways to enhance things in B. That's what - to my understanding - happened here.

[mazzech]

@cptx thank you for reporting this, I was not aware that D9 has major i18n problems... I have 2 multilingual sites in the making, node translation works pretty well, I also believe there's no need for switching to entity translation (I did several projects in D7 with ET, it was not working 100% either)

[joejoseph00]

Ya for most sites the backdrop content_translation is going to do a great job. There are some tradeoffs either way.

I agree with keeping backdrop using content translation, it is a risky and expensive proposition to go full entity translation.

entity_translation in D7 was a huge effort and took years to become polished like it is today. Backporting would be a lot of work. Stephanopoulos did a great job on ET module writing simpletests.

D8/D9 content translation is entity translation and it took also a huge amount of work to make it polished also. I have been doing mostly D8/D9 projects lately and a few D7 projects.

Really impressed with the great work on Backdrop keep up the great work!

[olstjos]

for the record D9 doesn't have major i18n problems, it's working well overall, there's always going to be some edge cases and possible improvements to make but overall very pleased now, it's on par with D7 entity_translation if not better. Stephanos and I spent years improving it in D7, especially Stephanos who did most of the work and wrote a lot of automated tests.

[indigoxela]

for the record D9 doesn't have major i18n problems, it's working well

@olstjos sorry if comments here gave the impression that there are. We're not trying to diminish all the good work that has been done and is still going on in Drupal. That would be misplaced in this issue queue, anyway - for several (hopefully obvious) reasons. :smiley:

[cptX]

Sorry @olstjos but D9 multilingual sites do have major problems with Comments, and according to one of the comment module maintainers the problem is also in Forum and in History modules. This is the current status, I had this discussion 2 weeks ago.

Please, before cancelling my statement try to make a multlilingual site with D9, with comments in nodes and in forum and try to filter the comments per language (you need to patch the core to do so). Also just try to preview or delete a comment in a translated page... Simple tests just try them and then report back here to state if it does or doesn't have major problems. I gave a specific link where all the issues with comments in multilingual sites are described. I would be happy if you read it.

[olstjos]

@cptX thanks for pointing me to that issue, I'll be following it.

[jenlampton]

I added a link to the Content translation improvements issue into the Original Post.

I still believe that we might be able to solve all the issues mentioned there (points pulled from this thread about why people chose entity translation over content translation) while still using content translation. It sounds like we have two major areas that need work: 1) a better user interface for translation and 2) more/better automatic detection of languages (which can likely be done with a core language context).

As people weigh in here with reasons they prefer (or were forced) to use entity translation, I would love if it we could add those reasons to that issue too -- since many of those improvements will be able to be included before 2025 :)

[olstjos]

I'll just drop this here:

entity_translation_unified_form

screenshot:

This module works for Drupal 7 and 8

I'm about to publish a new release for Drupal 8/9 with a minor bug fix.

[mazzech]

Yes, but for all Backdroppers with i18n sites in mind, let me emphasize... the Drupal 8/9 multilingual behaviour is based on a very different toolset (one node stores ALL language versions > Entity Translation) compared to Backdrop's translation system (separate ID for each translated article). In Drupal 7, both options were available and I did use them both pretty often back then. But in my last D7 projects, I switched back to node translation, because my projects rarely had a 1:1 relation among languages... meaning that e.g. the German section hat 100 pages, but only 50 of them have been translated to English, or some products simply were not sold to international customers. Node translation allowed a more flexible architecture for those scenarios.

I was also very pleased when TMGMT (translation management) module appeared for D7, allowing A/B input masks as seen in the screenshot above. But in reality, professional translators turned out to prefer their own systems, not working in the CMS at all. For the kind projects I did, an export/import function (XLIFF, HTML) is much more useful than having a state-of-the-art translation GUI – but I am very well aware that this could be different for other projects.

However, one thing that is missing in Backdrop i18n is more functions from D7's variable translation module.

[olstjos]

if you want to do something like entity_translation_unified_form for Backdrop using content_translation, you can probably try backporting this:

https://github.com/uottawa-wcms/dual_language_editor

otherwise, there's maybe other existing solutions on d.o that rely on content_translation (D6 style)