search

selfire1 / BibleGateway-to-Obsidian

This script adapts jgclark's BibleGateway-to-Markdown script to export for use in Obsidian.
MIT License
204 stars 63 forks source link

Clarify copyright disclaimers #15

Closed AceDexter closed 1 year ago

AceDexter commented 2 years ago

Hello, I've attempted using this script to download the entire Bible from Bible Gateway but each .md file is the first chapter of the book being downloaded (e.g. Genesis Chapter 2 - 50 are all Genesis Chapter 1 even though the back links and file names are correct). This is true for every book of the Bible (Prov Chapter 2 - 31 are all Prov Chapter 1, etc.).

I've tried downloading, NASB1995, ESV, and WEB translations but I'm getting the same results.

I'm not getting any errors in the console while the script is running and I'm pretty confident that I installed Ruby and the required gems correctly.

I am using ruby version 3.0.3p157 (2021-11-24 revision 3fb7d2cadc) [x64-mingw32]

Update:

I tried using ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x64-mingw32] on another machine and am getting the same results.

selfire1 commented 2 years ago

Hi! I ran the script again on my machine and didn't have any issues. I am running ruby 3.0.3p157 (2021-11-24 revision 3fb7d2cadc).

Which OS are you on? Would you mind running gem list to check if the required gems have been installed correctly?

AceDexter commented 2 years ago

Thank you for the reply!

I'm using Windows 10.

Below is my gem list:

*** LOCAL GEMS ***

benchmark (default: 0.1.0)
bigdecimal (default: 2.0.0)
bundler (default: 2.1.4)
cgi (default: 0.1.0.1)
clipboard (1.3.6)
colorize (0.8.1)
csv (default: 3.1.2)
date (default: 3.0.3)
dbm (default: 1.1.0)
delegate (default: 0.1.0)
did_you_mean (default: 1.4.0)
etc (default: 1.1.0)
fcntl (default: 1.0.0)
ffi (1.15.5 x64-mingw32)
fiddle (default: 1.0.0)
fileutils (default: 1.4.1)
forwardable (default: 1.3.1)
gdbm (default: 2.1.0)
getoptlong (default: 0.1.0)
io-console (default: 0.5.6)
io-wait (0.2.1)
ipaddr (default: 1.2.2)
irb (default: 1.2.6)
json (default: 2.3.0)
logger (default: 1.4.2)
matrix (default: 0.2.0)
minitest (5.13.0)
mutex_m (default: 0.1.0)
net-http (0.2.0)
net-pop (default: 0.1.0)
net-protocol (0.1.2)
net-smtp (default: 0.1.0)
net-telnet (0.2.0)
observer (default: 0.1.0)
open3 (default: 0.1.0)
openssl (default: 2.1.3)
optparse (0.2.0)
ostruct (default: 0.2.0)
power_assert (1.1.7)
prime (default: 0.1.1)
pstore (default: 0.1.0)
psych (default: 3.1.0)
racc (default: 1.4.16)
rake (13.0.1)
rdoc (default: 6.2.1.1)
readline (default: 0.0.2)
reline (default: 0.1.5)
rexml (default: 3.2.3.1)
rss (default: 0.2.8)
sdbm (default: 1.0.0)
singleton (default: 0.1.0)
stringio (default: 0.1.0)
strscan (default: 1.0.3)
test-unit (3.3.4)
timeout (default: 0.1.0)
tracer (default: 0.1.0)
uri (default: 0.10.0)
webrick (default: 1.6.1)
xmlrpc (0.3.0)
yaml (default: 0.1.0)
zlib (default: 1.1.0)
selfire1 commented 2 years ago

I am sorry, but I am at a loss as to what the issue could be. It seems like all gems are properly installed.

Maybe try deinstalling and reinstalling ruby again? Since I am on MacOS I can't really debug Windows.

AceDexter commented 2 years ago

No worries. I thought it might end up being an OS issue so I ended up implementing my own web scrapper using Python.

Markdian commented 2 years ago

I had the same problem and I solved it with the old version of the jgclark script.

selfire1 commented 2 years ago

@Markdian Which version did you end up using?

Markdian commented 2 years ago

v1.4.3

dperdue commented 2 years ago

I had the same issue on Mac with Ruby 2.6.3p62 (2019-04-16 revision 67580). Getting 1.43 (d693e85) also fixed my issue.

selfire1 commented 2 years ago

Thanks for letting me know! Seems like there is some problem with the jgclark script then. I'll raise an issue in a bit.

jgclark commented 2 years ago

@selfire1 I am concerned that people's use of your script as reported in this thread is very clearly breaking copyright for some of the translations mentioned. You should not be facilitating scraping a whole Bible text which isn't in the public domain.

For example from https://www.esv.org/resources/esv-global-study-bible/copyright-page/:

"The ESV text may be quoted (in written, visual, or electronic form) up to and inclusive of five hundred (500) consecutive verses without express written permission of the publisher, provided that the verses quoted do not amount to more than one-half of any one book of the Bible or its equivalent measured in bytes and provided that the verses quoted do not account for twenty-five percent (25%) or more of the total text of the work in which they are quoted."

My script clearly says it is for use in looking up short passages for study.

selfire1 commented 2 years ago

Hi @jgclark, thanks for commenting your concern. I also care about copyright and mentioned clearly in the disclaimers of the Readme, that the default WEB scraping may only be changed as long as copyright standards are honoured.

I understood you as giving your general thumbs up for usage of your script in Usage and adaptation of your script · Issue #3. However, I am aware that you didn't intend your script to be used for copyright violations.

In the past, I handled it as the user's responsibility. I clearly stated that the user must be aware of the copyright. I don't know the copyright details of every version, and wouldn't want to restrict it simply to the WEB. Rather I wanted every believer who uses the script to act justly. But I am open to suggestions/confrontation on how to handle that better.

Of course, as soon as you withdraw permission to use your script, I'll pull it. Thanks for taking the time to express your concern and I'm genuinely open to hear how your suggestions on how to navigate this better.

jgclark commented 2 years ago

Thanks, @selfire1. I thought we'd be on the same page on this, and it sounds like we are.

Now I've checked what your script is designed to do -- scrape the whole Bible in a single operation -- I'd like to make two suggestions:

  1. You incorporate the text in the disclaimer in the first section. (Incidentally, it's 'discretion' not 'disgression').
  2. Include a question at the start of your script saying "I confirm that I have checked and understand the copyright/license conditions for and wish to continue downloading it in its entirety. <Yes/No>"

How does that sound?

northern-arizona commented 2 years ago

I wanted to add to the discussion on Bible copyrights. The NET version has a generous copyright policy (netbible.com/copyright). Looks like it is fine to download the whole thing for free for your personal study. When quoting it in print (online or otherwise), you do need to mark it as NET. See their policy for more specifics.

selfire1 commented 2 years ago

Those two suggestions sound great. I'll implement them. To clarify, you would suggest adding the ESV disclaimer in the readme, did I get that right?

jgclark commented 2 years ago

My item 1 was about changing the order of material in your README. It wasn't about ESV specifically: the point is that each version has different copyright and use terms, and its impossible for us to programmatically understand what they are. Hence pushing the responsibility to users (item 2). On item 2 I think now you should also include in the question a reference to the particular version that's going to be scraped.

jgclark commented 2 years ago

@northern-arizona wrote:

I wanted to add to the discussion on Bible copyrights. The NET version has a generous copyright policy (netbible.com/copyright). Looks like it is fine to download the whole thing for free for your personal study. When quoting it in print (online or otherwise), you do need to mark it as NET. See their policy for more specifics.

This would be a good comment to add to your README (if there isn't already), as an example of one that does have a very generous copyright policy.