LibrariesHacked / openlibrary-search

Searching Open Library by keywords to return ISBNs
147 stars 26 forks source link

Communicative load process #7

Closed Chloe-Meinshausen closed 1 year ago

Chloe-Meinshausen commented 1 year ago

Dear Dave, Here is my PR. Feel free not to accept it! I don't mean to write over any of your work!

I am the kind of person who has-to-watch-the-pot-boil, unless I have a timer giving me permission to let the pot boil by itself. Especially if the pot is my laptop. haha

I was going a little nuts by the time the works file started being processed -It took my laptop 10+ hrs to load. (that was before you moved indexing around so it probably wouldn't take that long now) so I decided to create a new py file based off of yours and break the larger files into minifiles/chunks, then I created a load script that can cycle through the mini files and load them sequentially, while also giving updates in the terminal. I had done all that before you sent me a link to your new version but when I was testing the new version I was still clawing my eyes out waiting for files to load so I decided to integrate it into yours.
Its fully automated and doesn't take any more steps than before.
I added some sample files with the .txt removed so you could run it quickly if you wanted to test it.

image image