Azure-Samples / document-intelligence-code-samples

Sample site for Document Intelligence code samples and associated media.
MIT License
85 stars 48 forks source link

page_type: sample languages:

Azure Document Intelligence code samples repo

Note: on July 2023, the Azure Cognitive Services Form Recognizer service was renamed to Azure AI Document Intelligence. Any mentions of Form Recognizer or Document Intelligence in documentation refer to the same Azure service.

Welcome! Azure AI Document Intelligence is a cloud service that uses machine learning to analyze text and structured data from your documents.

Python .NET Java JavaScript

Table of Contents

Features

Azure AI Document Intelligence is a cloud-based Azure AI service that enables you to build intelligent document processing solutions. Massive amounts of data, spanning a wide variety of data types, are stored in forms and documents. Document Intelligence enables you to effectively manage the velocity at which data is collected and processed and is key to improved operations, informed data-driven decisions, and enlightened innovation.

Prerequisites

Setup

  1. Open a terminal window in your local environment and install the Azure AI Document Intelligence client library for Python with pip:
pip install azure-ai-documentintelligence --pre
  1. Clone or download this sample repository
  2. Open the sample folder in Visual Studio Code or your IDE of choice.

Run the samples

  1. Open a terminal window and cd to the directory that the samples are saved in.
  2. Set the environment variables specified in the sample file you wish to run.
  3. If necessary, select Data to get your document.
  4. Below are some sample code guidelines so that you can choose the sample according to your needs.
    Note: For more samples, see Azure SDK for Python and Async Samples.

Common samples

Select the link of the model name to reach the corresponding topic page for more details. Select v3.1 (2023-07-31-GA) to view earlier versions.

Read model: Extract printed and handwritten text.

sample_analyze_read.py

Layout model: Extract and anlayze text, tables, and document structure.

sample_analyze_layout.py

Prebuilt models: Add intelligent document processing to your apps and flows without having to train and build your own models.

sample_analyze_invoices.py — Extract and analyze customer and vendor details from invoice documents.
sample_analyze_identity_documents.py — Extract and analyze identification and verification details identification documents.
sample_analyze_receipts.py — Extract and analyze sales transaction details from receipt documents.
sample_analyze_tax_us_w2.py — Extract and analyze taxable compensation details from US tax W-2 forms.

Add-on capabilities: Extend the extracted results from documents with add-on capabilities.

sample_analyze_addon_barcodes.py - Extract barcode properties documents using this add-on capability.
sample_analyze_addon_fonts.py - Extract font properties from documents using this add-on capability.
sample_analyze_addon_formulas.py - Extract formulas from documents using this add-on capability.
sample_analyze_addon_highres.py - Extract high resolution properties from documents using this add-on capability.
sample_analyze_addon_languages.py - Detact language from documents using this add-on capability.
sample_analyze_addon_query_fields.py - Specify and extract query fields from documents using this add-on capability.

Custom model: Train your own models to extract data from structured and unstructured documents.

sample_analyze_custom_documents.py - Analyze a document with a custom built model. The document must be of the same type as the documents the custom model was trained on.
sample_classify_document.py - Classify a document using a trained document classifier.
sample_compose_model.py - This is useful when you have built different models and want to aggregate a group of them into a single model that you (or a user) could use to analyze a document.
sample_copy_model_to.py - Copy a custom model from a source Form Recognizer resource to a target Form Recognizer resource.
sample_copy_model_to.py - Copy a custom model from a source Document Intelligence resource to a target Document Intelligence resource.
sample_manage_classifiers.py - Manage the classifiers on your account.
sample_manage_models.py - Manage the models on your account.

Additional samples

sample_convert_to_dict.py - Convert a model returned from an analyze operation to a JSON serializable dictionary.

Retrieval Augmented Generation (RAG) samples

The Layout model provides various building blocks like tables, paragraphs, section headings, etc. that can enable different semantic chunking strategies of the document. With semantic chunking in Retrieval Augmented Generation (RAG), it will be more efficient in storage and retrieval, together with the benefits of improved relevance and enhanced interpretability. The following samples show how to use the Layout model to do semantic chunking and use the chunks to do RAG.
Note:Only available for v4.0 (2024-02-29-preview) .

sample_rag_langchain.ipynb
Sample RAG notebook using Azure AI Document Intelligence as document loader, MarkdownHeaderSplitter and Azure AI Search as retriever in Langchain.

sample_figure_understanding.ipynb
Sample notebook showcasing how to crop the figures and send figure content (with its caption) to Azure OpenAI GPT-4V model to understand the semantics. The figure description will be used to update the markdown output, which can be further used for semantic chunking.

sample_identify_and_merge_cross_page_tables.ipynb and sample_identify_and_merge_cross_page_tables.py
Sample postprocessing script to identify and merge cross-page tables based on business rules.

Pre/post processing samples

There are usually some pre/post processing steps that are needed to get the best results from the Document Intelligence models. These steps are not part of the Document Intelligence service, but are common steps that are needed to get the best results. The following samples show how to do these steps.
Note:Applies to all versions.

sample_disambiguate_similar_characters.ipynb and sample_disambiguate_similar_characters.py: Sample postprocessing script to disambiguate similar characters based on business rules.

sample_identify_cross_page_tables.ipynb and sample_identify_cross_page_tables.py:Sample postprocessing script to identify cross-page tables based on business rules.

Next steps

To learn more about Azure Document Intelligence features and capabilities, see the client library API reference documentation.

Current document analysis models

Document analysis models enable text extraction from forms and documents and return structured business-ready content ready for your organization's action, use, or development.

Current prebuilt models

Prebuilt models enable you to add intelligent document processing to your apps and flows without having to train and build your own models.

Current custom models

Current add-on capabilities

✔️ ocr.highResolution ✔️ ocr.formula ✔️ ocr.font ✔️ ocr.barcode ✔️ queryFields