Closed thucpn closed 3 weeks ago
Latest commit: 228ae883199d1af9aed4c2a51a4930304b605318
The changes in this PR will be included in the next version bump.
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
This update enhances the LlamaCloud pipeline for data ingestion in TypeScript applications, focusing on improved document handling, private file uploads, and modularity. Key changes include refined file upload processes, extended metadata management, and the introduction of new dependencies to facilitate these improvements. Overall, these modifications streamline operations, bolster security, and elevate the functionality and usability of the system.
Files | Change Summary |
---|---|
.changeset/violet-islands-collect.md | Introduces "create-llama" patch focused on private file uploads and data ingestion improvements. |
helpers/env-variables.ts | Updates documentation for LLAMA_CLOUD_ORGANIZATION_ID , removing "Python only" restriction. |
templates/components/llamaindex/typescript/documents/helper.ts | Renames loadDocuments to storeAndParseFile , adding logic for private document handling and modifying metadata. |
templates/components/llamaindex/typescript/documents/pipeline.ts | Simplifies runPipeline function by removing conditional logic for LlamaCloudIndex , enforcing a single processing path. |
templates/components/llamaindex/typescript/documents/upload.ts | Modifies uploadDocument to include filename parameter, enabling direct uploads to LlamaCloud for relevant indexes. |
templates/components/llamaindex/typescript/streaming/events.ts | Introduces downloadFilesFromNodes function for improved modularity while removing direct usage of LLamaCloudFileService . |
templates/components/vectordbs/typescript/llamacloud/generate.ts | Replaces LlamaCloudIndex with LLamaCloudFileService in loadAndIndex , shifting to document-by-document processing. |
templates/types/streaming/express/package.json | Adds @llamaindex/cloud dependency to enhance project capabilities. |
templates/types/streaming/express/src/controllers/chat-config.controller.ts | Introduces validation for LLAMA_CLOUD_API_KEY in the configuration function, enhancing robustness. |
templates/types/streaming/nextjs/app/api/chat/config/llamacloud/route.ts | Adds validation for LLAMA_CLOUD_API_KEY in the GET function to ensure proper configuration before execution. |
templates/types/streaming/nextjs/package.json | Adds @llamaindex/cloud dependency for the Next.js project to extend functionality. |
sequenceDiagram
participant User
participant UploadService
participant LlamaCloudFileService
User->>UploadService: Upload file (filename, raw data)
UploadService->>LlamaCloudFileService: Upload file to LlamaCloud
LlamaCloudFileService-->>UploadService: Confirmation of upload
UploadService-->>User: Upload successful message
🐰 In the garden of code, we hop and play,
With LlamaCloud's magic, we brighten the day!
Files float in safely, metadata in tow,
A swift little dance as we watch the data grow!
Hooray for the changes, a cheer from us all,
For every new feature, we hop and we crawl! 🌼✨
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Note: upgrade LITS to use LlamaCloudFileService. Update LITS will also fix E2E is running fail in express
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Chores