TypeError: Cannot read properties of undefined (reading 'transcriptBodyRenderer') using YoutubeLoader

Annahcj commented 3 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain.js documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain.js rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

const loader = YoutubeLoader.createFromUrl(url, {
  language: 'en',
  addVideoInfo: true,
});

return loader.load();

Error Message and Stack Trace (if applicable)

Error: Failed to get YouTube video transcription: [YoutubeTranscript] 🚨 TypeError: Cannot read properties of undefined (reading 'transcriptBodyRenderer')
    at YoutubeLoader.load (/Users/annaan/Documents/JS/langchain2/server/node_modules/langchain/dist/document_loaders/web/youtube.cjs:100:19)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

Description

I'm using the YoutubeLoader from LangChain to load Youtube transcripts into documents. Getting this error with any youtube URL. This issue comes from the library youtube-transcript which is used under the hood - https://github.com/Kakulukian/youtube-transcript/issues/19 I believe Youtube's API has changed

System Info

Platform: Mac Node version: 18.18.2 npm version: 9.8.1 langchain: ^0.1.31 youtube-transcript: ^1.1.0 youtubei.js: ^9.2.0

dosubot[bot] commented 3 months ago

Hey there @Annahcj! Nice to see you back. Hope all is well on your end.

Thank you for providing detailed information about the issue you're encountering. The error message you're seeing is typically indicative of a change in the structure of the response from YouTube's API. This could mean that the transcriptBodyRenderer property, which the YoutubeLoader feature in LangChain expects to find, is no longer present in the same location within the response object.

Given that the error is likely within the dependencies (youtube-transcript or youtubei.js), here are some steps you can take to troubleshoot and potentially resolve the issue:

Check for Updates: Make sure that all dependencies are up to date. If there have been changes to YouTube's API, it's possible that these libraries have already released updates to accommodate these changes.
Review Dependency Documentation: Check the documentation for youtube-transcript and youtubei.js for any recent changes or updates that might be relevant to your issue. If YouTube's API has changed, these libraries may have updated their documentation to reflect these changes.
Inspect API Responses: If possible, add some logging to your code to inspect the raw API responses from YouTube. This can help you identify any changes in the structure of the response that might be causing your issue.
Adjust Code According to API Changes: If you identify changes in the API response structure, you may need to adjust your code to correctly handle these changes. This could involve changing how you access certain properties or handle the response data.
Contact Dependency Maintainers: If the issue persists and you're unable to resolve it based on the available documentation, consider reaching out to the maintainers of youtube-transcript and youtubei.js. They may be able to provide further guidance or updates on the issue.
Fallback to Alternative Methods: If the current dependencies do not quickly adapt to the API changes, consider using alternative methods or libraries for fetching YouTube transcripts and metadata as a temporary workaround.

I hope these steps help you in resolving the issue. If you have any further questions or need more assistance, feel free to ask.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

tonymanh-dev commented 3 months ago

Same issue here. youtube-transcript or youtubei.js just released a new version and they might change something in their API structure.

siddharthd0 commented 3 months ago

having the same issue

pjsofts commented 3 months ago

Same here

olefirenko commented 3 months ago

Same here

alejandrochacinortega commented 2 months ago

Having the same issue here as well.

sinedied commented 2 months ago

For a temporary workaround until this is fixed upstream, you can:

npm i https://github.com/sinedied/youtube-transcript\#a10a073ac325b3b88018f321fa1bc5d62fa69b1c

This will use my fork that use a compatible drop-in code replacement from https://github.com/Kakulukian/youtube-transcript/issues/19, all the code credits goes to the folks there.

When the issue is fixed upstream, you can simply:

npm rm youtube-transcript

and it will return to the upstream version.

prash-murali commented 2 months ago

Same issue here with youtube-transcript

SahilPulikal commented 2 months ago

For a temporary workaround until this is fixed upstream, you can:
npm i https://github.com/sinedied/youtube-transcript\#a10a073ac325b3b88018f321fa1bc5d62fa69b1c
This will use my fork that use a compatible drop-in code replacement from Kakulukian/youtube-transcript#19, all the code credits goes to the folks there.

When the issue is fixed upstream, you can simply:
npm rm youtube-transcript
and it will return to the upstream version.

Thank you so much @sinedied , it worked.

ArsenKakasyan commented 2 months ago

For a temporary workaround until this is fixed upstream, you can:
npm i https://github.com/sinedied/youtube-transcript\#a10a073ac325b3b88018f321fa1bc5d62fa69b1c
This will use my fork that use a compatible drop-in code replacement from Kakulukian/youtube-transcript#19, all the code credits goes to the folks there.

When the issue is fixed upstream, you can simply:
npm rm youtube-transcript
and it will return to the upstream version.

Thank you.

langchain-ai / langchainjs