search

BCcampus / pressbooks-openstax-import

[UNMAINTAINED] Pressbooks Plugin for OpenStax Textbook Import
GNU General Public License v3.0
5 stars 0 forks source link

OpenStax Import #50

Closed lucwrite closed 6 years ago

lucwrite commented 6 years ago

Description

OpenStax Import plugin times out (error HTTP 500) when trying to import

Expected behaviour A book can be imported using the link to the zip file from OpenStax. Here is the link address used https://cnx.org/exports/0889907c-f0ef-496a-bcb8-2a5bb121717f@2.49.zip/elementary-algebra-2.49.zip

[What you expected to happen] The book to be imported into the following Pressbook https://pressbooks.bccampus.ca/capilanosandbox/

Actual behaviour The process timed-out and we received the following error HTTP ERROR 500

Steps to reproduce the problem

  1. Go to Elementary Algebra on CNX https://cnx.org/contents/CImQfPDv@2.49:GfaWl1GG@4/Introduction. Note this has also been tested with two other books
  2. Copy the Zip file URL by right clicking
  3. Select CNX Import of a Zip file in the target Pressbook
  4. Select Import from URL and past in the URL for the zip https://cnx.org/exports/0889907c-f0ef-496a-bcb8-2a5bb121717f@2.49.zip/elementary-algebra-2.49.zip
  5. Click begin import

[First Step, Second Step, etc]

System Information

System Information

Book Info

Book ID: 349 Book URL: https://pressbooks.bccampus.ca/capilanosandbox/ Book Privacy: Public

Browser

Platform: OS X Browser Name: Chrome Browser Version: 65.0.3325.162 User Agent String: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.162 Safari/537.36

WordPress Configuration

Network URL: http://pressbooks.bccampus.ca/ Network Type: Subdirectory Version: 4.9.4 Language: en_US WP_ENV: Not set WP_DEBUG: Enabled Memory Limit: 64M

Pressbooks Configuration

Version: 5.1.0 Book Theme: Open Textbooks Book Theme Version: 2.1.1 Root Theme: Aldine Root Theme Version: 1.1.0

Pressbooks Dependencies

Epubcheck: Installed Kindlegen: Installed xmllint: Installed PrinceXML: Installed Saxon-HE: Installed

Must-Use Plugins

hm-autoloader.php: n/a

Network Active Plugins

BC Post-Secondary Validator: 1.0.0 CC Export for Pressbooks: 0.2.1 f5 Force SSL: 1.0.0 iThemes Security: 6.9.2 Openstax Import for Pressbooks: 1.0.1 Pressbooks: 5.1.0 Pressbooks Stats: 1.4.0 Textbooks for Pressbooks: 4.0.2 WP-Piwik: 1.0.19

Book Active Plugins

Inactive Plugins

Akismet Anti-Spam: 4.0.3 BuddyPress: 2.9.3 H5P: 1.10.1 mPDF for Pressbooks: 3.1.1 WP QuickLaTeX: 3.8.4

Server Configuration

PHP Version: 7.1.15 MySQL Version: 5.5.5 Webserver Info: Apache

PHP Configuration

Safe Mode: Disabled Memory Limit: 512M Upload Max Size: 100M Post Max Size: 100M Upload Max Filesize: 100M Time Limit: 60 Max Input Vars: 1000 URL-aware fopen: On (1) Display Errors: N/A

PHP Extensions

OPcache: Disabled XDebug: Disabled cURL: Supported cURL Version: 7.19.7 imagick: Not Installed xsl: Installed

bdolor commented 6 years ago

Thanks for reporting Lucas — I could recreate this once, then when I tried again it worked. The file is only 11MB so it shouldn't be that file size is an issue. It may have been something on cnx.org side. Would you mind trying again?

image

lucwrite commented 6 years ago

Hi Brad both Rajiv and myself are still receiving timeout errors on for this. Here are the details.

Book (CNX) https://cnx.org/exports/0889907c-f0ef-496a-bcb8-2a5bb121717f@2.49.zip/elementary-algebra-2.49.zip

Target Book: https://opentextbc.ca/algebratest/

Error: HTTP Error 500

System Information

System Information

Book Info

Book ID: 230 Book URL: https://opentextbc.ca/algebratest/ Book Privacy: Public

Browser

Platform: OS X Browser Name: Chrome Browser Version: 65.0.3325.162 User Agent String: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.162 Safari/537.36

WordPress Configuration

Network URL: http://opentextbc.ca/ Network Type: Subdirectory Version: 4.9.4 Language: en_US WP_ENV: Not set WP_DEBUG: Disabled Memory Limit: 64M

Pressbooks Configuration

Version: 5.1.0 Book Theme: Open Textbooks Book Theme Version: 2.1.1 Root Theme: Aldine Root Theme Version: 1.1.0

Pressbooks Dependencies

Epubcheck: Installed Kindlegen: Installed xmllint: Installed PrinceXML: Installed Saxon-HE: Installed

Must-Use Plugins

hm-autoloader.php: n/a

Network Active Plugins

Akismet Anti-Spam: 4.0.3 Better WordPress Google XML Sitemaps: 1.4.1 CC Export for Pressbooks: 0.2.1 f5 Force SSL: 1.0.0 iThemes Security: 6.9.2 Openstax Import for Pressbooks: 1.0.1 Pressbooks: 5.1.0 Pressbooks Stats: 1.4.0 Textbooks for Pressbooks: 4.0.2 User Activation Keys: 4.6 WP-Piwik: 1.0.19

Book Active Plugins

Inactive Plugins

BuddyPress: 2.9.3 H5P: 1.10.1 mPDF for Pressbooks: 3.1.1 Relevanssi: 4.0.4 WP QuickLaTeX: 3.8.4

Server Configuration

PHP Version: 7.1.15 MySQL Version: 5.5.5 Webserver Info: Apache

PHP Configuration

Safe Mode: Disabled Memory Limit: 512M Upload Max Size: 100M Post Max Size: 100M Upload Max Filesize: 100M Time Limit: 60 Max Input Vars: 1000 URL-aware fopen: On (1) Display Errors: N/A

PHP Extensions

OPcache: Disabled XDebug: Disabled cURL: Supported cURL Version: 7.19.7 imagick: Not Installed xsl: Installed

bdolor commented 6 years ago

@lucwrite - the link to the book you've provided isn't the same one that I've been using.

Can you confirm, this is the book/place where we should both be pulling from? https://cnx.org/contents/e9XCtyLF@3.9:uUfJZx98@4/Preface

lucwrite commented 6 years ago

Different books. We have also had failed imports on the other books linked in the issue. Just to clarify are you thinking that the issue is not with the OpenStax import but rather with Individual CNX zip files?

thatpsychprof commented 6 years ago

Do you think it is a timeout issue? I also tried uploading the zip file from my computer but it appeared to timeout. If so we could try breaking up the 2GB zip file into smaller ones for each chapter and try uploading one of those?

bdolor commented 6 years ago

If it doesn't time out first, it's maxing out the memory_limit on the server during a curl operation.

lucwrite commented 6 years ago

@bdolor what further tests do we need to do so that we can provide you and the community more infor so that we start importing OS books again?

bdolor commented 6 years ago

The issue is that this is a 2 GB book. OpenStax textbooks can still be imported, just not this one book because of its size. No further tests are necessary. I've tried both increasing php's memory limit to 2GB and extending the time limit for http_requests to 1.5 hours. We're bumping up against the limits of normal server configuration and expected functionality.

thatpsychprof commented 6 years ago

Apologies for the limits of my technical knowledge but can we work around this by unzipping and rezipping two (or more) smaller files? My faculty member has been waiting for the ability to modify the book and if I can do something to help I would like to.

Rajiv Jhangiani, Ph.D. Special Advisor to the Provost on Open Education & Psychology Instructor, Kwantlen Polytechnic University Associate Editor, Psychology Learning and Teaching Ambassador, Center for Open Science

On Thu, Apr 5, 2018 at 2:21 PM -0400, "Brad" notifications@github.com<mailto:notifications@github.com> wrote:

The issue is that this is a 2 GB book. OpenStax textbooks can still be imported, just not this one book because of its size. No further tests are necessary. I've tried both increasing php's memory limit to 2GB and extending the time limit for http_requests to 1.5 hours. We're bumping up against the limits of normal server configuration and expected functionality.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/BCcampus/pressbooks-openstax-import/issues/50#issuecomment-379031077, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AWD9ayXUqVjD0J89BpMQ16mIMvBKhLihks5tlmC5gaJpZM4SulDI.

bdolor commented 6 years ago

I've fixed the ability to upload zip files in https://github.com/BCcampus/pressbooks-openstax-import/commit/ff36e538dca19d1622a62d61db2f6a3d25c01588 and temporarily gave PHP ridiculous amounts of memory on the server. Huge image sizes are the source of the bloated zip file and perhaps because of how much memory the upload process consumes, also causes some problems importing.

image

But the formulas and content seem intact. https://pressbooksdev.bccampus.ca/elementaryalgebra/ Hopefully this is something that you can work with @thatpsychprof with the help of @lucwrite while the bigger issues are given some time to get ironed out properly.

lucwrite commented 6 years ago

@bdolor thanks for the temporary fix. Would you suggest that we reach out to the OpenStax team to discuss the size of their images and Zip files. This does not seem sustainable for interoperability.

bdolor commented 6 years ago

If you have the ability to persuade them to optimize their images, it would take a lot off our plate. For example...the worst offender in that book in the chapter 'Add and Subtract Polynomials' which has 272 images (mostly small pictures of equations) each of which are about one half of a MB.

image

Unless their use case is to have these images available for print, which I suspect it is, there is plenty of room to reduce the size of these images without sacrificing quality. The size of that one chapter (m60158), unzipped, is 154MB.

image

bdolor commented 6 years ago

this is being addressed in smaller, more actionable issues. #52 #53 #54 #55