search

standardebooks / web

The source code for the Standard Ebooks website.
https://standardebooks.org
Creative Commons Zero v1.0 Universal
235 stars 64 forks source link

Initial code changes and script to insert/update Ebook records #351

Closed colagrosso closed 5 months ago

colagrosso commented 5 months ago

Building on the initial schema we committed in this branch, here are the code changes and script to actually populate the database tables. The deploy-ebook-to-www script will now call a new script, scripts/update-ebook-database, that will create an Ebook PHP instance from ebookWwwFilesystemPath and create or update a record in the database.

Here is a gist that shows sample records from each table:

https://gist.github.com/colagrosso/f8896c6af84886a70b2c0ec15c41108f

You can see the IndexableText is now written to each Ebook record.

Still so much to do, but it's a nice milestone that the code can populate all the existing ebooks and update them as they are rebuilt. I've been testing backfilling the existing ebooks with variants of:

for BOOK in $(find /standardebooks.org/ebooks -maxdepth 1 -type d)
do
  tsp nice /standardebooks.org/web/scripts/deploy-ebook-to-www --verbose --no-images --no-epubcheck --no-feeds --no-bulk-downloads "$BOOK"
done
colagrosso commented 5 months ago

I pushed some small changes to address your feedback. When we're done you can squash them all together if you'd prefer.

colagrosso commented 5 months ago

Once we wrap up ebook validation, are able to pull from the database for /ebook, and do some extensive testing, we can try this branch in production to see how things go.

Yep, I plan to spin up a DigitalOcean droplet with this branch to help with testing.

acabal commented 5 months ago

Great, just a few more comments I thought of as I looked at this again.

I can spin up a droplet for you on SE's dime, just let me know when you're ready to test.

colagrosso commented 5 months ago

Great, just a few more comments I thought of as I looked at this again.

Thanks for pointing out current and future refactors.

Thanks, too, for the changes in 5b3f8f7b7732d98f2eecc6ac7b2fd8481af2d9e0. The reflection to get the enums with $type::from() when appropriate is cool.

In the future, I'll put enums like EbookSourceType in their own file. Sometime soon, I'll rebase this branch onto master and fix the conflict to keep this branch current with master.

I can spin up a droplet for you on SE's dime, just let me know when you're ready to test.

Thanks!