Biblehub missing in Cloud

[warpok]

Evidently something has changed again at Biblehub. We have a few members who prefer to use the Cloud, and Biblehub has stopped appearing for them. The same thing happens for me in the USA, and I noticed that about 2 weeks ago.

[teusbenschop]

Everything works okay here from this side.

I set up a new Cloud instance from the most recent code, and then opened the Biblehub interlinear Bible.

Here is the screenshot.

[warpok]

Maybe it would help to delete the resource in the cloud and reinstall it.

Phil

Phil Fields USA: 479-524-7005 Indonesia/WA: +6285770333734 P.O. Box 701, Siloam Springs, AR 72761

http://albata.info http://dailybiblereading.info

One King to rule them all, One Son to find them, One blood to cleanse them all, And from the dark unbind them.

On Sat, Feb 15, 2020, 8:09 AM Teus Benschop notifications@github.com wrote:

Everything works okay here from this side.

I set up a new Cloud instance from the most recent code, and then opened the Biblehub interlinear Bible.

Here is the screenshot.

[image: Screenshot 2020-02-15 at 15 08 09] https://user-images.githubusercontent.com/1310167/74589373-18d6a480-5005-11ea-8865-ef688270232f.png

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bibledit/cloud/issues/354?email_source=notifications&email_token=ABDSIJONZYT4KQTWROVLIADRC7ZRRA5CNFSM4KUTAL42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL3LLRY#issuecomment-586593735, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDSIJJRGAF223VP3IKWCPDRC7ZRRANCNFSM4KUTAL4Q .

[teusbenschop]

Maybe it would help to delete the resource in the cloud and reinstall it.

Yes, likely this will help and fix it.

[warpok]

It did NOT fix it. Please send me the address that Bibledit uses for Biblehub. I wonder if this is a server difference thing for different regions of the world.

[teusbenschop]

Here is the address that Bibledit uses:

https://biblehub.com/interlinear/genesis/1-1.htm

From this side of the globe, the website looks like this:

[warpok]

That link works for me in my browser. But not in my resources in the same browser using Bibledit Cloud.

[warpok]

Going to the extent of rediculousness, I added the url in a separate workspace:

But the window still doesn't update. Note, you sent me the link for Genesis 1:1. I now wonder if the thing causing the break has to do with the way Bibledit puts in the book, chapter, and verse.

[teusbenschop]

Note, you sent me the link for Genesis 1:1. I now wonder if the thing causing the break has to do with the way Bibledit puts in the book, chapter, and verse.

I checked on that too, opening Genesis 30:1, just as you have.

Here's how Bibledit renders Biblehub:

[teusbenschop]

Does the Journal give a clue about why it does not show Biblehub?

[teusbenschop]

On your server, if you do

wget http://biblehub.com/interlinear/genesis/1-1.htm

there's going to be output like this:

--2020-02-20 07:09:42--  http://biblehub.com/interlinear/genesis/1-1.htm
Resolving biblehub.com (biblehub.com)... 192.169.82.74
Connecting to biblehub.com (biblehub.com)|192.169.82.74|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://biblehub.com/interlinear/genesis/1-1.htm [following]
--2020-02-20 07:09:43--  https://biblehub.com/interlinear/genesis/1-1.htm
Connecting to biblehub.com (biblehub.com)|192.169.82.74|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17404 (17K) [text/html]
Saving to: ‘1-1.htm’

1-1.htm             100%[===================>]  17.00K  --.-KB/s    in 0.1s    

2020-02-20 07:09:43 (139 KB/s) - ‘1-1.htm’ saved [17404/17404]

There's also going to be a file called "1-1.htm".

Could you post:

1. The output of wget.
2. The file 1-1.htm.

Doing this from my side works okay. I just wonder if doing it from your side gives other output, that BibleHub gives another response. That's why the above information may help.

[warpok]

[warpok]

[warpok]

[warpok]

Inside of the 1-2.htm file, there doesn't seem to be any Interlinear data. I checked the ends of the lines for it.

[warpok]

Sorry, I didn't know the way to download the 1-2.htm file from the server.

[warpok]

I wonder if it would help to make it so that each Bibledit server always has a full copy of the Biblehub.com Interlinear. So that Biblehub Interlear is always is called from the Bibledit server rather than the Internet.

Another way of putting it is this: Since the Interlinear is a resource that is basic to the task of Bible translation (and since we frequently have a problem with Biblehub), wouldn't it be a good idea if every Bibledit server always kept a perfect copy of that resource? And it would be good if every client received a full copy of that resource right from the time of installation.

[teusbenschop]

Sorry, I didn't know the way to download the 1-2.htm file from the server.

I really need the full copy of that file for making a comparison.

[teusbenschop]

I wonder if it would help to make it so that each Bibledit server always has a full copy of the Biblehub.com Interlinear. So that Biblehub Interlear is always is called from the Bibledit server rather than the Internet.

Another way of putting it is this: Since the Interlinear is a resource that is basic to the task of Bible translation (and since we frequently have a problem with Biblehub), wouldn't it be a good idea if every Bibledit server always kept a perfect copy of that resource? And it would be good if every client received a full copy of that resource right from the time of installation.

It would be great, but due to copyright restrictions, I cannot include the content from that website in the Bibledit package. That's why Bibledit scrapes the web pages, to circumvent the copyright restrictions.

It has always worked well, the Cloud could scrape the biblehub.com website perfectly. If your problem can be resolved, then after that it will work well again.

That's why I really need that file.

Or you may mail me the credentials to access your server. Then I can go there and see it first-hand.

[teusbenschop]

There seems to be a permission error on the server:

/var/bibledit/bibledit-cloud/config$ ls -l
total 68
-rw-r--r-- 1 root     root      682 Feb  1 07:46 README
-rw-r--r-- 1 bibledit bibledit   28 Jun 25  2018 admin-email
-rw-r--r-- 1 bibledit bibledit   11 Jun 25  2018 admin-password
-rw-r--r-- 1 bibledit bibledit    7 Jun 25  2018 admin-username
-rw-r--r-- 1 root     root     1732 Apr 22  2019 alkitab_life_key.txt
-rw-r--r-- 1 bibledit bibledit    0 Jun 23  2018 badbrowswer.https
-rw-r--r-- 1 root     root        0 Nov 23 15:50 browser.https
-rw-r--r-- 1 root     root     1927 Feb  1 07:46 cert.pem
-rw-r--r-- 1 root     root     1647 Feb  1 07:46 chain.pem
-rw-r--r-- 1 bibledit bibledit 2590 Feb  8 22:25 config.txt
-rw-r--r-- 1 root     root     3574 Feb  1 07:46 fullchain.pem
-rw-r--r-- 1 bibledit bibledit 1478 Feb  8 22:25 ldap.txt
-rw-r--r-- 1 bibledit bibledit 1939 Feb  8 22:25 local.cert.pem
-rw-r--r-- 1 bibledit bibledit 2766 Feb  8 22:25 local.chain.pem
-rw-r--r-- 1 bibledit bibledit 1679 Feb  8 22:25 local.privkey.pem
-rw-r--r-- 1 bibledit bibledit    5 Nov 22 08:40 network-port
-rw-r--r-- 1 bibledit bibledit  487 Jun 25  2018 obfuscate.txt
-rw-r--r-- 1 root     root     1708 Feb  1 07:46 privkey.pem
-rw-r--r-- 1 bibledit bibledit   24 Feb  8 22:25 userfacingurl.txt

[teusbenschop]

The permissions have now been fixed, but the problem remained. I've also changed "browser.https" to "browser.https.off", so it won't crash anymore on a secure connection.

[teusbenschop]

The Biblehub pages now work again.

[teusbenschop]

There is definitely a problem with the secure connection on the website:

wget https://alkitab.live:2023 
--2020-02-21 19:41:50--  https://alkitab.live:2023/
Resolving alkitab.live (alkitab.live)... 178.128.28.19
Connecting to alkitab.live (alkitab.live)|178.128.28.19|:2023... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://alkitab.live:2023/index/index [following]
--2020-02-21 19:41:52--  https://alkitab.live:2023/index/index
Connecting to alkitab.live (alkitab.live)|178.128.28.19|:2023... connected.
GnuTLS: Error in the pull function.
Unable to establish SSL connection.

[teusbenschop]

Checking sometime later, the secure connection is fine:

wget https://alkitab.live:2023
--2020-02-21 20:01:44--  https://alkitab.live:2023/
Resolving alkitab.live (alkitab.live)... 178.128.28.19
Connecting to alkitab.live (alkitab.live)|178.128.28.19|:2023... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://alkitab.live:2023/index/index [following]
--2020-02-21 20:01:46--  https://alkitab.live:2023/index/index
Connecting to alkitab.live (alkitab.live)|178.128.28.19|:2023... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1174 (1.1K) [text/html]
Saving to: 'index.html'

index.html                             100%[============================================================================>]   1.15K  --.-KB/s    in 0s      

2020-02-21 20:01:48 (9.03 MB/s) - 'index.html' saved [1174/1174]

[warpok]

Thanks so much for helping me with that!!!! Was it perhaps the method that I used to renew the certificate the last time? Checking right now using the Cloud, I was not able to see Biblehub for Genesis 2:1 and 2:2, 2:3. I note that the address did not change to 2023 but stayed as http://alkitab.live:2022/resource/index

On the client, I see that reinstalling Biblehub as a resources is still downloading books way too fast. It has taken only 7 minutes to download the OT books for Biblehub on a cellular connection. I only wish that this could be the real speed! :-)

[warpok]

So do you understand what is happening?

[warpok]

Would it help to check the certificate with these? https://www.ssllabs.com/ssltest/analyze.html?d=alkitab.live https://www.ssllabs.com/ssltest/analyze.html?d=www.alkitab.live

The first of those failed just saying it was unable to connect to the server. It is amazing to me that Bibledit would keep working while there are problems like this going on!

[teusbenschop]

Since Genesis 2:1 and others do not display yet, and since the client downloads the resources way too fast, what will resolve it for you is to clear the web cache and the resources cache.

You may find that option right at the bottom of this page:

http://bibledit.org:8080/system/index

After clearing the caches, everything should be back to normal.

[teusbenschop]

Would it help to check the certificate with these? https://www.ssllabs.com/ssltest/analyze.html?d=alkitab.live https://www.ssllabs.com/ssltest/analyze.html?d=www.alkitab.live

That tests port 80, not port 2023.

[warpok]

There was an error in clearing the caches:

[warpok]

That tests port 80, not port 2023.

Would I be able to use those test commnds by adding :2022?

[warpok]

Note the misspelling in the journal. claaring > clearing

How long will it take for Biblehub to work after clearing the caches? So far it is not working. I haven’t told you before: Some other resources have not been working either. Net Bible has not been showing up for a while.

[warpok]

One day after clearing the caches, no change.

[teusbenschop]

Just now the Biblehub interlineair was not showing up when logging into the Cloud.

I noticed that the bibledit cloud server was stuck at 99% CPU usage.

After stopping and starting the bibledit cloud server, the Biblehub interlinear showed up again.

[teusbenschop]

I connected a Bibledit client to the Bibledit Cloud server at alkitab.live.

When displaying the BibleHub interlinear on the client, this is what it said:

Biblehub Interlinear Receiving: Connection reset by peer

Question: Why was the network connection reset by alkitab.live?

[teusbenschop]

There is a lot of connection failures from the client to the Cloud:

23 February 2020 20:50:28 | Files: Failure downloading file: Receiving: Connection reset by peer 23 February 2020 20:51:20 | Bibles: Failure getting total checksum: Receiving: Connection reset by peer 23 February 2020 20:51:20 | Receiving: Connection reset by peer 23 February 2020 20:51:20 | Notes: Failure to initiate connection 23 February 2020 20:51:29 | Files: Failure requesting directory files: alkitab.live:2022: Connection refused

[warpok]

I am surprised by 99% server usage. I will check if logging into Digital Ocean could give some insight. Is there anything you would suggest?

[teusbenschop]

Yes, perhaps Digital Ocean could give a clue.

There is also the connections that get reset regularly. Is there a firewall in Digital Ocean that is kicking in now and then?

[warpok]

There was an old firewall that was made when I was trying to make one in the Digital Ocean interface. But Firewalls only work in what they call 'Load Ballancers', and since I don't have one of those, I thought that it would have no effect. I have deleted that now.

The CPU was definitely stuck at 99+%.

[warpok]

[warpok]

7 day graph:

[warpok]

This is for 24 hours. The normal setting is for 6 hours. The disk usage seems normal. Why would there be sharp dips in CPU usage? Probably those are times that the automatic peer reset are happening.

[warpok]

Powering off the server didn't help. Setting the display back to the 6 hour setting, the server is still maxing out. Note the oposite Disk I/O. I wonder if the server could be attacked by a computer virus that is eating server time.

[warpok]

![Uploading Screenshot 2020-02-24 at 8.02.13 PM.png…]()

[warpok]

I decided to log into the administrator account for Bibledit, so I logged out of my manager account. I thought that I might be able to use the administrator account to delete the git repository backup for the KIA project. That project currently does not need to be in Bibledit.

NOW BIBLEDIT IS NOT ALLOWING ME TO LOG IN AS EITHER ADMINISTER OR MANAGER!

[warpok]

Now my Bibledit client also cannot access the server!

[warpok]

This is happening as I am in Indonesia, and this was the first day of our team workshop working intensively on key terms. I think it very clear that there is some spiritual opposition going on here!

[warpok]

The server now lets me log in, but it says System information disabled du to load higher than 1.0.

[warpok]

two screenshots from a team member:

[teusbenschop]

When did the problems start?

Was it after a server update?

[teusbenschop]

I wondered whether the server was under an attack.

$ sudo netstat -ntu

The output of this command did not show excessive connection attempts. So currently it does not look like there's a DDOS attack going on. Also the CPU usage is low at the moment.