dosubot[bot]
commented
7 months ago 🤖
The error you're encountering is likely due to the fact that Deno does not support Node.js built-in modules like 'fs', 'path', and 'url'. Deno has its own set of standard libraries and does not use Node.js's 'require' system.
To resolve this issue, you should use Deno's built-in functions to replace the Node.js modules. Here's how you can modify your code:
import { test, expect } from "@jest/globals";
import { PDFLoader } from "../fs/pdf.js";
import { readFileSync } from "https://deno.land/std/fs/mod.ts";
import { resolve, dirname } from "https://deno.land/std/path/mod.ts";
import { fromFileUrl } from "https://deno.land/std/path/mod.ts";
test("Test PDF loader from blob", async () => {
const filePath = resolve(
dirname(fromFileUrl(import.meta.url)),
"./example_data/1706.03762.pdf"
);
const loader = new PDFLoader(
new Blob([readFileSync(filePath)], {
type: "application/pdf",
})
);
const docs = await loader.load();
expect(docs.length).toBe(15);
expect(docs[0].pageContent).toContain("Attention Is All You Need");
expect(docs[0].metadata).toMatchInlineSnapshot(`
{
"blobType": "application/pdf",
"loc": {
"pageNumber": 1,
},
"pdf": {
"info": {
"Author": "",
"CreationDate": "D:20171207010315Z",
"Creator": "LaTeX with hyperref package",
"IsAcroFormPresent": false,
"IsXFAPresent": false,
"Keywords": "",
"ModDate": "D:20171207010315Z",
"PDFFormatVersion": "1.5",
"Producer": "pdfTeX-1.40.17",
"Subject": "",
"Title": "",
"Trapped": {
"name": "False",
},
},
"metadata": null,
"totalPages": 15,
"version": "1.10.100",
},
"source": "blob",
}
`);
});In this code, I've replaced the Node.js 'fs', 'path', and 'url' modules with equivalent functions from Deno's standard library. The 'readFileSync' function is used instead of 'fs.readFile', and 'resolve' and 'dirname' functions are used instead of 'path.resolve' and 'path.dirname'. The 'fromFileUrl' function is used to convert the URL to a file path.
You can find more information about this in the langchain/src/document_loaders/tests/pdf-blob.test.ts file in the langchainjs repository.
As for your question about whether the langchainjs framework supports DenoJS, it does. The evidence for this is in the provided context, specifically in the deno.json file. This file is used to map imports in Deno, and it includes several langchain modules, such as "@langchain/community/", "@langchain/openai", "@langchain/core/", and "@langchain/pinecone". This indicates that these modules can be imported and used in a Deno environment. You can find more information about this in the deno.json file in the langchainjs repository.
jacoblee93
mcgear
I am trying to use PDFLoader from a test in DenoJS, to start getting a feel for it, but am stuck almos immediately.
Getting the following error when running the test
Test is here: https://github.com/fathym-deno/everything-as-code/blob/feature/thinky/tests/workshop/bench.ts
Dependencies defined here: https://github.com/fathym-deno/everything-as-code/blob/feature/thinky/tests/test.deps.ts
Trying to import pdf-parse instead of npm install (it fails even if i run
npm install pdf-parse): import "npm:pdf-parse";How can i get this working in deno?