bibledit / cloud

Source code for Bibledit core library and Cloud binary
GNU General Public License v3.0
13 stars 3 forks source link

Google Translate resource request #803

Closed warpok closed 2 years ago

warpok commented 2 years ago

It turns out that Google Translate does a very interesting job of translating the Greek NT into Indonesian. Our translators and people visiting our Guest site often are hampered because of not having an interlinear that uses Indonesian glosses.

Would you be willing to make a resource that would take the Byzantine Text verse and run it through Google Translate?

Screenshot_20220716_091559 Screenshot_20220716_091317

It seems like this might be a resource other translators would appreciate, if it could be set up with variables: Greek Text: __ (one of the resources already in Bibledit) Target Language: (Instead of Indonesian, some might choose Chinese or French.)

teusbenschop commented 2 years ago

I think that if having glosses in their own language helps, and if google translate does a reliable job, then this would be a worthwhile effort to put in.

warpok commented 2 years ago

Yes! I'm certain it will help. The heading for this should say that it is from Google Translate, since people know that it often makes mistakes. They will be warned that the translation might not be perfect.

teusbenschop commented 2 years ago

It is already possible right now to translate a resource from one language to another language.

This can be done through creating a new USFM resource.

These are the steps to do it.

  1. Convert the desired original resource to a Bible. See for example here: http://bibledit.org:8090/bible/settings?bible=Sample. Click the option "Import text from a resource".
  2. Create a new Bible with the name of the target language, for example "Greek to Indonesian NT".
  3. Copy the USFM of each chapter of the original Bible into Google Translate.
  4. Paste the translated USFM from Google Translate into the sample "Greek to Indonesian NT" Bible.
  5. Do this for all the chapters required.
  6. Optionally correct and update the text as needed.
  7. Optionally convert this Bible to a Resource (but this step is not needed).

This way you may be able to display the "Greek to Indonesian NT" alongside the other resources.

This will assist the translators greatly.

teusbenschop commented 2 years ago

Version 5.0.985 has implemented, what it calls, the "Translated resource".

The Translated resource can take an original resource, and a language pair that indicates the source language and the target language, and then displays the original resource in the target language.

Here is how to define and use a Translated resource in Bibledit Cloud:

  1. Set up access to Google Translate via menu Settings / System / Google Translate, http://bibledit.org:8090/system/googletranslate.
  2. Create a Translated resource and define it via menu Settings / Resources / Translated, http://bibledit.org:8090/resource/translated9edit
  3. Display the Translated resource via menu Translate / Resources / Organise, http://bibledit.org:8090/resource/organize

Once the Resource is defined in Bibledit Cloud, it can then also be used via a Bibledit Client. The translated resource appears in the client among all the other resources.

Be aware that you need a Google account to set up access to Google Translate, and that Google requires you to setup billing. It's not that using Google Translate is expensive, actually it is really inexpensive. But even so they need you to set up billing for that.

teusbenschop commented 2 years ago

The newest Bibledit (Cloud) in the Ubuntu PPA now has this Google Translate access and provides the "Translated resource".

warpok commented 2 years ago

Teus, this doesn't work! Screenshot_20220902_104208 The user cannot choose the language to translate to, or the one to translate from.

teusbenschop commented 2 years ago

It is related to issue https://github.com/bibledit/cloud/issues/824. If access to Google Translate has not yet been set up, or is not yet working, then there's no language options to choose from. That is what you are seeing.

Here are the steps to set up Google Translate: https://github.com/bibledit/cloud/issues/803#issuecomment-1224306036

The first step points to a page to set it up. What does that page say in your situation?

warpok commented 2 years ago

Wow, Teus, Your instructions for doing the task of setting up the Google Translate service are truly daunting. Setting up a server for Bibledit-cloud is a piece of cake. What kind of super-humans are you expecting to be able to set that up?! (Clearly I fail.)

Although I thought I might have succeeded. The instructions are like a treasure hunt with clues. googletranslate.txt

Go to to the server to get the instructions, go five steps to the north and go to this site and do this procedure, then go to another site and find a key.

Does this look right? Screenshot_20220902_131318

warpok commented 2 years ago

Is this where I get the Json key? Is it the link on the right or the link on the left, or is it somewhere else? Screenshot_20220902_135527

Should I enable this? Screenshot_20220902_135817

The instructions for installing Google-cloud-c11 say that the default place to install is in the home subdirectory. But Bibledit-cloud can't find my installation. I think I should install it in the /var/bibledit folder instead. Do you agree?

Screenshot_20220902_140130

It seemed like I installed the Google-cloud-c11 properly.

Screenshot_20220902_140642

Actually, after that install didn't work, I tried to install in the /var/bibledit folder, but I never got that last screen. Probably due to not being able to finish the Shell Command Completion step.

warpok commented 2 years ago

The problem could be that I should use the Key menu choice found here: image

warpok commented 2 years ago

I thought I already had a key, so I didn't add another one.

I have clicked the link, and there is no response. The Red line stays red.

Screenshot_20220902_190737

Do I need Shell Command Completion? Screenshot_20220902_190436

So, bottom line:

warpok commented 2 years ago

Sorry for the tone of the first part of this post. If you were offended, please forgive. I hope you can read such things with the tone of a friendly jibe. If you can help me with this, I can try to write a step by step article, not for Map.Bloomfire, but something to post elsewhere. I did my best to install things properly, which I hope shows that I am greatly desiring the Google Translate feature to work for us.

One thing occurs to me: The automatic install will place things in the /var/bibledit folder. But you have said that Aranggi was more correct in placing the files in a Home/user folder. Now the gcloud CCI said that it would be best to install that in Home, but then it wasn't found by Bibledit. So, if occurs to me that it might be best for the sake of add-on programs working as expected with bibledit-cloud to make the automatic install go to a home/user area.

Actually, the place where I want the Google Translate feature to work properly is the Indonesian Guest instance which is still at Bibledit.org. If you feel you can install it there, please make copious screenshots that we can use for making a clear tutorial.

teusbenschop commented 2 years ago

[...] Your instructions for doing the task of setting up the Google Translate service are truly daunting. Setting up a server for Bibledit-cloud is a piece of cake. What kind of super-humans are you expecting to be able to set that up?! (Clearly I fail.)

I thought that giving the steps while referring to a guide written by Google should not have been to difficult. The steps in the file "googletranslate.txt" all refer to the Guide written by Google. I though that a big company like Google would be able much better than myself to write a guide that is helpful.

  • There are two editions of Google Translate. You don't tell us whether to choose basic or advanced.

That is correct, the file "googletranslate.txt" does not tell that. But when following the Guide written by Google, it never asks for that question. However the answer to your question is that Bibledit uses the basic edition of Google Translate. And the Guide does that too, albeit implicitly, without mentioning it. Thinking about the Guide, it might have been better if Google had left that remark about basic or advanced out entirely. The remark could lead to confusion. Anyway, the answer is: Basic edition.

  • You told me to get a json key, but I don't know what I am looking for. What will that key look like? Is it a string of about 24 numbers?

Normally the JSON key is a file. The contents of that JSON key looks like this:

{
  "type": "service_account",
  "project_id": "yourproject",
  "private_key_id": "<a long string of characters and number>",
  "private_key": "-----BEGIN PRIVATE KEY-----<a very long string of characters and numbers$
  "client_email": "test@project.iam.gserviceaccount.com",
  "client_id": "<a long number>",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test%40project.iam.gserviceaccount.com"
}

Does this look right? Screenshot_20220902_131318

This is the screen shot for creating a project, it looks right, it's one of the steps in the Google Guide.

teusbenschop commented 2 years ago

Is this where I get the Json key? Is it the link on the right or the link on the left, or is it somewhere else? Screenshot_20220902_135527

Yes, this is where you will be able to get the JSON key. The Google Guide tells you where to click exactly. The Guide says this:

  1. In the Google Cloud console, click the email address for the service account that you created.

Should I enable this? Screenshot_20220902_135817

Yes, you should enable this. Actually the Google Guide didn't mention whether to enable this, and that is a shortcoming in that guide.

The instructions for installing Google-cloud-c11 say that the default place to install is in the home subdirectory. But Bibledit-cloud can't find my installation. I think I should install it in the /var/bibledit folder instead. Do you agree?

It's not c11 but cli`. The wordcli``` is an abbreviation for "command line interface".

I didn't find the link to the place where the instructions tell that the default location to install is in the "home" subdirectory. Could you provide a link to those instructions?

It should install it in the default location where the gcloud-cli want to get installed, not specifically in /var/bibledit. The key for Bibledit Cloud to find the cloud-cli is to add the program to the PATH. It's one of the questions the installation process will ask you. And Google didn't say that in their guide.

Screenshot_20220902_140130

It seemed like I installed the Google-cloud-c11 properly.

Yes, it looks that the cloud-cli is now installed properly, looking at the screenshot.

Screenshot_20220902_140642

Actually, after that install didn't work, I tried to install in the /var/bibledit folder, but I never got that last screen. Probably due to not being able to finish the Shell Command Completion step.

There's no need to install it in /var/bibledit. Best is to let the installer decide its own location.

teusbenschop commented 2 years ago

The problem could be that I should use the Key menu choice found here: image

Yes, use that key menu choice.

The Google Guide says that too:

Create a service account key:

  1. In the Google Cloud console, click the email address for the service account that you created.
  2. Click Keys.
  3. Click Add key, and then click Create new key.
  4. Click Create. A JSON key file is downloaded to your computer.
  5. Click Close.
teusbenschop commented 2 years ago

I thought I already had a key, so I didn't add another one.

Did you store the key in the config folder of Bibledit?

This is what the file googletranslate.txt says:

Store the key in the Bibledit "config" folder as file "googletranslate.json".

I have clicked the link, and there is no response. The Red line stays red.

Screenshot_20220902_190737

The red line is self-explanatory. It means that Bibledit Cloud could not find the Google JSON key in the correct location.

Do I need Shell Command Completion? Screenshot_20220902_190436

The answer to that prompt will be Yes.

teusbenschop commented 2 years ago

Sorry for the tone of the first part of this post. If you were offended, please forgive.

No problem at all.

I hope you can read such things with the tone of a friendly jibe. If you can help me with this, I can try to write a step by step article, not for Map.Bloomfire, but something to post elsewhere. I did my best to install things properly, which I hope shows that I am greatly desiring the Google Translate feature to work for us.

Yes, that would be very helpful, if you were able to write an article with step by step information.

One thing occurs to me: The automatic install will place things in the /var/bibledit folder. But you have said that Aranggi was more correct in placing the files in a Home/user folder. Now the gcloud CCI said that it would be best to install that in Home, but then it wasn't found by Bibledit. So, if occurs to me that it might be best for the sake of add-on programs working as expected with bibledit-cloud to make the automatic install go to a home/user area.

There's two ways of doing things. If installing Bibledit Cloud from an Ubuntu package, then the most logical and official place is /var/bibledit.

What Aranggi did is different. He didn't install Bibledit Cloud from a package, but he installed it from source code. If doing it from source code then, yes, it's easier to do that in the home directory.

Actually, the place where I want the Google Translate feature to work properly is the Indonesian Guest instance which is still at Bibledit.org. If you feel you can install it there, please make copious screenshots that we can use for making a clear tutorial.

I can install it there but, but still, you will need to provide the JSON key, because that JSON key will be generated from your own google account, and will be charged to that account too. But once you have the JSON key, I can install it for you there.

warpok commented 2 years ago

Screenshot_20220903_183313 Screenshot_20220903_183326 Screenshot_20220903_183249

warpok commented 2 years ago

OK, with your information, one clicks on the link in the email column. Screenshot_20220903_183313

That opens this page, and one has to click the Keys menu item. Screenshot_20220903_183326

Then on this page, you have to click to add a key. The key is generated automatically and it was named, in my case, river-sunlight+longnumber.json. Screenshot_20220903_183249

This is then renamed as googletranslate.json and put in the bibledit-cloud/config folder.

Now, please tell me how to install the gcloud cli program so that it is on the path. What folder do I un-tar it and run it in? Or what command do I give to put something in the 'path' so that will be found by Bibledit-cloud?

teusbenschop commented 2 years ago

The Google Guide gives all the steps to install cloud. https://cloud.google.com/sdk/docs/install This guide is general, it covers many Linux flavours.

In case you'd like to install cloud on Ubuntu, then the steps are much simpler.

Here are the steps to take on Ubuntu:

sudo apt-get install apt-transport-https ca-certificates gnupg

echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

sudo apt update

sudo apt install google-cloud-cli

This automatically puts cloud into the PATH.

Once this is done, then Bibledit will be able to find the gcloud and then can start using it.

You will now be able to define a so called "Translated resource". (And then the translators will have access to a vastly expanded pool of resources...:)

warpok commented 2 years ago

Yay! It WORKS so well!

Yes, the 8084 one was for one team that is no longer working. Just to be absolutely sure, it would be good if you would send the tarball of the bibledit-cloud directory. Parlin will be working on the 8082 Indonesian guest site.

I am So Very Thrilled about the translated resources. We have been hindered for so long because of not having a translation of the source texts into Indonesian. And this works Great.

Phil

=============== Phil Fields

USA: 479-524-7005

Indonesia and Whatsapp: +6285 770 333 734

303 Ryan Ct., Siloam Springs, AR 72761

albata.info dailybiblereading.info Firman Allah adalah Kunci Kehidupan

Pada tanggal Min, 4 Sep 2022 pukul 04.57 Teus Benschop < @.***> menulis:

The Google Guide gives all the steps to install cloud. https://cloud.google.com/sdk/docs/install This guide is general, it covers many Linux flavours.

In case you'd like to install cloud on Ubuntu, then the steps are much simpler.

Here are the steps to take on Ubuntu:

sudo apt-get install apt-transport-https ca-certificates gnupg

echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

sudo apt update

sudo apt install google-cloud-cli

This automatically puts cloud into the PATH.

Once this is done, then Bibledit will be able to find the gcloud and then can start using it.

You will now be able to define a so called "Translated resource". (And then the translators will have access to a vastly expanded pool of resources...:)

— Reply to this email directly, view it on GitHub https://github.com/bibledit/cloud/issues/803#issuecomment-1236301878, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDSIJIHWKZMBMNXEE6LQWTV4RXBRANCNFSM53YFOHAA . You are receiving this because you authored the thread.Message ID: @.***>

teusbenschop commented 2 years ago

So happy it works well and is useful. Thanks for the feedback on this.

I'll send a WeTransfer with the content of the 8084 port and will take the port offline now.

warpok commented 2 years ago

Teus,

The Hebrew translation resource is not working now. It is returning nonsense or words in Hebrew characters now. I wonder if it will help if I change the Hebrew text.

Phil

=============== Phil Fields

USA: 479-524-7005

Indonesia and Whatsapp: +6285 770 333 734

303 Ryan Ct., Siloam Springs, AR 72761

albata.info dailybiblereading.info Firman Allah adalah Kunci Kehidupan

Pada tanggal Sen, 5 Sep 2022 pukul 13.23 Teus Benschop < @.***> menulis:

So happy it works well and is useful. Thanks for the feedback on this.

I'll send a WeTransfer with the content of the 8084 port and will take the port offline now.

— Reply to this email directly, view it on GitHub https://github.com/bibledit/cloud/issues/803#issuecomment-1237374089, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDSIJITHSNP4IC42SJS7ZLV4Y3ARANCNFSM53YFOHAA . You are receiving this because you authored the thread.Message ID: @.***>

teusbenschop commented 2 years ago

What is the current original resource in Hebrew? And the destination language is Indonesian? I could try to see how it behaves here.

warpok commented 2 years ago

I first tried the Westminster Hebrew text (one of the first options listed in the resource list). It seemed to work when I did it, but then didn't work. Then I tried the E-sword module entitled Masoretic text (or something like that). If you search for Masoretic in the resource list window, that is about the second one to be found. But that keeps not working. I will try another Hebrew text today, to see if that helps.

I wonder if the bug could be that there are TWO active Translated resources, and the working of one conflicts somehow with the other. Perhaps the order of the resources in our list makes a difference.

Thanks for helping me explore this. Phil

Phil Fields

USA: 479-524-7005

Indonesia and Whatsapp: +6285 770 333 734

303 Ryan Ct., Siloam Springs, AR 72761

albata.info dailybiblereading.info Firman Allah adalah Kunci Kehidupan

Pada tanggal Sel, 6 Sep 2022 pukul 11.44 Teus Benschop < @.***> menulis:

What is the current original resource in Hebrew? And the destination language is Indonesian? I could try to see how it behaves here.

— Reply to this email directly, view it on GitHub https://github.com/bibledit/cloud/issues/803#issuecomment-1238402642, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDSIJNSNNX7IJNWHE2QH6DV45YHJANCNFSM53YFOHAA . You are receiving this because you authored the thread.Message ID: @.***>

teusbenschop commented 2 years ago

I first tried the Westminster Hebrew text (one of the first options listed in the resource list). It seemed to work when I did it, but then didn't work.

It looks like Google is not doing a great job when it translates Biblical Hebrew, yet takes it as modern Hebrew, and also the vowels in the Hebrew seem to cause difficulties to Google.

Then I tried the E-sword module entitled Masoretic text (or something like that). If you search for Masoretic in the resource list window, that is about the second one to be found.

It's working well over here, but the translation is not too reliable.

I wonder if the bug could be that there are TWO active Translated resources, and the working of one conflicts somehow with the other.

I've got four Translated resources defined now, and they all work independently here. Not sure what's going on in your case.

warpok commented 2 years ago

Which four are working well for you?

Phil

=============== Phil Fields USA: 479-524-7005 Indonesia and Whatsapp: +6285 770 333 734

albata.info dailybiblereading.info

      Firman Allah adalah Kunci Kehidupan

===============

On Wed, Sep 7, 2022, 10:21 AM Teus Benschop @.***> wrote:

I first tried the Westminster Hebrew text (one of the first options listed in the resource list). It seemed to work when I did it, but then didn't work.

It looks like Google is not doing a great job when it translates Biblical Hebrew, yet takes it as modern Hebrew, and also the vowels in the Hebrew seem to cause difficulties to Google.

Then I tried the E-sword module entitled Masoretic text (or something like that). If you search for Masoretic in the resource list window, that is about the second one to be found.

It's working well over here, but the translation is not too reliable.

I wonder if the bug could be that there are TWO active Translated resources, and the working of one conflicts somehow with the other.

I've got four Translated resources defined now, and they all work independently here. Not sure what's going on in your case.

— Reply to this email directly, view it on GitHub https://github.com/bibledit/cloud/issues/803#issuecomment-1239533594, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDSIJP7XOG7TU5RG2Q4THDV5CXIBANCNFSM53YFOHAA . You are receiving this because you authored the thread.Message ID: @.***>

teusbenschop commented 2 years ago

Which four are working well for you?

It's not all four that are using Hebrew, but two of them do:

Translated translated [edit] Translated test [edit] Translated westminster [edit] Translated sword masoretic [edit]

The one using the Westminster text is having vowels and fails to translate properly. Westminster Hebrew וְהָי֤וּ לִמְאֹורֹת֙ בִּרְקִ֣יעַ הַשָּׁמַ֔יִם לְהָאִ֖יר עַל־הָאָ֑רֶץ וַֽיְהִי־כֵֽן׃ Translated westminster והי֤ו למאורות֙ בִּ֣יעַהָשִעִם לִ֖יר אל-ה֑֑רז וֽיְהי-օֽ׃

The second one using Hebrew is the Sword Masoretic text. Hebrew Masoretic OT והיו למאורת ברקיע השׁמים להאיר על־הארץ ו͏יהי־כן׃ Translated sword masoretic And let there be a light in the firmament of heaven to give light upon the earth, and let it be so.

The main issue is that the translation from Hebrew to any language appears not to be reliable. The reason seems to be that Google Translate assumes modern Hebrew, and that the Masoretic texts are Biblical Hebrew, which at times does not get translated too well.

The second issue I can see with Google Translate is that when the Masoretic text has the dots for the vowels, then Google Translate fails to translate it.

What may help in your situation is to have a kind of, say, literal translation in, say, English, and then translate that to Indonesian. I guess that may be more reliable than going from Biblical Hebrew to Indonesian. However these are just my thought at the moment.

warpok commented 2 years ago

Agreed. Thanks.

warpok commented 2 years ago

Wow!!!! I thought this was dropped. I didn’t have time to do what you said in the first response, so just left it in my inbox.

I am excited that you went ahead and created this resource. I’ll try it soon!

Thanks, Phil

Phil Fields USA: 479-524-7005 Indonesia/WA: +6285770333734 P.O. Box 701, Siloam Springs, AR 72761

http://albata.info http://dailybiblereading.info

One King to rule them all, One Son to find them, One blood to cleanse them all, And from the dark unbind them.

On Tue, Aug 23, 2022, 11:24 AM Teus Benschop @.***> wrote:

Version 5.0.985 has implemented, what it calls, the "Translated resource".

The Translated resource can take an original resource, and a language pair that indicates the source language and the target language, and then displays the original resource in the target language.

Here is how to define and use a Translated resource in Bibledit Cloud:

  1. Set up access to Google Translate via menu Settings / System / Google Translate, http://bibledit.org:8090/system/googletranslate.
  2. Create a Translated resource and define it via menu Settings / Resources / Translated, http://bibledit.org:8090/resource/translated9edit
  3. Display the Translated resource via menu Translate / Resources / Organise, http://bibledit.org:8090/resource/organize

Once the Resource is defined in Bibledit Cloud, it can then also be used via a Bibledit Client. The translated resource appears in the client among all the other resources.

Be aware that you need a Google account to set up access to Google Translated, and that Google requires you to setup billing. It's not that using Google Translate is expensive, actually it is really inexpensive. But even so they need you to set up billing for that.

— Reply to this email directly, view it on GitHub https://github.com/bibledit/cloud/issues/803#issuecomment-1224306036, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDSIJOM2O3O6IUYHUPLRW3V2T3L7ANCNFSM53YFOHAA . You are receiving this because you authored the thread.Message ID: @.***>