Refactor/loaders

[collindutter]

• [x] I have read and agree to the contributing guidelines for submitting new pull requests.

Describe your changes

Added

• BaseFileLoader for Loaders that load from a path.
• BaseLoader.fetch() method for fetching data from a source.
• BaseLoader.parse() method for parsing fetched data.
• BaseFileManager.encoding to specify the encoding when loading and saving files.
• BaseWebScraperDriver.extract_page() method for extracting data from an already scraped web page.
• TextLoaderRetrievalRagModule.chunker for specifying the chunking strategy.
• file_utils.get_mime_type utility for getting the MIME type of a file.

Changed

• BREAKING: Removed BaseFileManager.default_loader and BaseFileManager.loaders.
• BREAKING: Loaders no longer chunk data, use a Chunker to chunk the data.
• BREAKING: Removed fileutils.load_file and fileutils.load_files.
• BREAKING: Removed loaders-dataframe and loaders-audio extras as they are no longer needed.
• BREKING: TextLoader, PdfLoader, ImageLoader, and AudioLoader now take a str | PathLike instead of bytes.
• BREAKING: Removed DataframeLoader.
• LocalFileManagerDriver.workdir is now optional.
• filetype is now a core dependency.
• FileManagerTool now uses filetype for more accurate file type detection.
• BaseFileLoader.load_file() will now either return a TextArtifact or a BlobArtifact depending on whether BaseFileManager.encoding is set.

The purpose of this PR was to clean up the Loader interface, and their define purpose.

Loaders fetch data from a source, and parse it into Artifacts. Loaders do not chunk data, that is the role of Chunkers.

We provide 4 top level Loaders that provide from a variety of sources:

• BaseFileLoader: Loads data from a file.
• WebLoader: Loads data from a web page.
• SqlLoader: Loads data from a SQL database.
• EmailLoader: Loads data from an email inbox.

BaseFileLoader then has subclasses that provide file-type specific parsing logic:

• TextLoader
• ImageLoader
• AudioLoader
• CsvLoader
• BlobLoader In a future PR, I'd like this parsing logic to live in a new class of Driver, File Parser Driver(?), instead of Loaders.

  Issue ticket number and link

  Closes https://github.com/griptape-ai/griptape/issues/1102

---

📚 Documentation preview 📚: https://griptape--1116.org.readthedocs.build//1116/

[collindutter]

Note that this PR's base branch is refactor/artifacts so no tests have run yet.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 95.58824% with 9 lines in your changes missing coverage. Please review.	Files with missing lines	Patch %	Lines
griptape/loaders/text_loader.py	71.42%	1 Missing and 1 partial :warning:
griptape/utils/file_utils.py	80.00%	1 Missing and 1 partial :warning:
.../drivers/file_manager/local_file_manager_driver.py	75.00%	0 Missing and 1 partial :warning:
griptape/loaders/blob_loader.py	80.00%	1 Missing :warning:
griptape/tasks/base_image_generation_task.py	0.00%	1 Missing :warning:
griptape/tools/file_manager/tool.py	87.50%	0 Missing and 1 partial :warning:
griptape/tools/image_query/tool.py	0.00%	1 Missing :warning:

:loudspeaker: Thoughts on this report? Let us know!