adithya-s-k / omniparse

Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks
https://docs.cognitivelab.in
GNU General Public License v3.0
4.37k stars 350 forks source link

Added an async client, table extraction from markdown to list of list with headers #2

Closed AswanthManoj closed 3 weeks ago

AswanthManoj commented 3 weeks ago

PR: Add AsyncOmniParse Client, Table Extraction, and Pydantic Models

This PR introduces several enhancements to our OmniParse project:

  1. AsyncOmniParse Client:

    • Implemented a new asynchronous client class AsyncOmniParse for interacting with the OmniParse server.
    • Added methods for parsing various types of documents (PDF, PPT, DOCX), images, videos, audio, and websites.
    • Included error handling for unsupported file types and API request failures.
    • Added comprehensive docstrings with usage examples for better developer experience.
  2. Table Extraction from Markdown:

    • Introduced functions to extract tables from markdown content:
      • extract_markdown_tables: Extracts raw markdown tables from a string.
      • markdown_to_tables: Converts markdown tables to a list of TableObj instances.
    • Tables are now represented as list of lists with separate headers, improving data structure for further processing.
  3. Pydantic Models:

    • Added several Pydantic models to ensure type safety and data validation:
      • ImageObj: Represents extracted images with name, binary data, and MIME type.
      • TableObj: Represents extracted tables with headers and data.
      • MetaData: Stores metadata about parsed documents.
      • ParsedDocument: Represents the complete parsed document with markdown content, images, tables, and metadata.

These additions provide a robust, type-safe, and asynchronous interface for interacting with OmniParse.

adithya-s-k commented 3 weeks ago

@AswanthManoj Thank you very much ⭐for contributing

will review the PR ,merge it and use it as a base for the the SDK