alamshafil / auto-shorts

AutoShorts can generate short videos with the help of AI. It can generate popular types of video seen on YouTube Shorts and TikTok.
https://alamshafil.github.io/auto-shorts/
MIT License
88 stars 8 forks source link

Invalid Video Type Error in AutoShorts (Using OllamaAIGen) #2

Open Tobe2d opened 2 months ago

Tobe2d commented 2 months ago

Hello,

I encountered an issue when using AutoShorts with the OllamaAIGen AI type. While generating a video script for a "news short" on sport cars, everything works well up until the video generation step, where I receive an error:

F:\AI_Projects3\auto-shorts\node_modules\auto-shorts\dist\index.js:170
            throw Error("Invalid video type!");
                  ^

Error: Invalid video type!
    at genVideo (F:\auto-shorts\node_modules\auto-shorts\dist\index.js:170:19)
    at genVideoWithAI (F:\auto-shorts\node_modules\auto-shorts\dist\index.js:90:18)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async cli (F:\auto-shorts\node_modules\auto-shorts\dist\cli.js:429:18)

Node.js v20.8.1

And when i change to GPTAIGen iget this error:


Invalid JSON data: Unexpected token 'T', "TODO" is not valid JSON
F:\auto-shorts\node_modules\auto-shorts\dist\index.js:140
        throw new Error("Invalid JSON data!");
              ^

Error: Invalid JSON data!
    at genVideo (F:\auto-shorts\node_modules\auto-shorts\dist\index.js:140:15)
    at genVideoWithAI (F:\auto-shorts\node_modules\auto-shorts\dist\index.js:90:18)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async cli (F:\auto-shorts\node_modules\auto-shorts\dist\cli.js:429:18)

I would appreciate any assistance or guidance on how to resolve this issue.

Thanks in advance!

alamshafil commented 2 months ago

Before I start, I would like to apologize for the late message. I have been working on a new version (tentatively v0.2.0) with solving your issue in mind. This is a known issue and I have experienced it myself.

First of all, the current way the AI works in my code is it expects a specific structured output; however, AI models (LLMs) tend to hallucinate and provide incorrect structured output. I need to improve the way this is handled and I am reworking the AI script generation in v0.2.0 which should, at least, attempt to fix this issue.

What I believe has happened in your case is the LLM model has provided an incorrect type with the incorrect fields for the types. It may have some correct info in the script but the code may not have seen due to the incorrect type (hence the Invalid video type error). I am reworking the way the codes asks AI for the script and reworking how the code handles to invalid JSON.

I know this may not be the answer you were expected. If you want to try the video generation capabilities, you can copy the JSON output of the AI response from the console and put into a separate JSON file. Here you can fix the JSON file with a valid video type. It should be type: 'topic'

You can run this command to read a JSON file instead of asking AI for script: npx auto-shorts --jsonFile [file]

Finally, GPTAIGen failed since I have not implemented the OpenAI API in v0.1.0. I am adding OpenAI API support to v0.2.0.

Thank for your time and interest in this project. Once again I apologize for the late message.

Tobe2d commented 2 months ago

Thanks for the reple @alamshafil

I cant wait for v0.2.0.

I've tested npx auto-shorts --jsonFile [file] and got another error:

Error: Error: JSON data is missing required "text" field!
    at TopicVideo.checkJson (F:\auto-shorts\node_modules\auto-shorts\dist\types\topicVid.js:21:19)
    at TopicVideo.generateVideo (F:\auto-shorts\node_modules\auto-shorts\dist\types\topicVid.js:28:14)
    at genVideo (F:\auto-shorts\node_modules\auto-shorts\dist\index.js:151:17)
    at genVideoWithJson (F:\auto-shorts\node_modules\auto-shorts\dist\index.js:120:18)
    at cli (F:\auto-shorts\node_modules\auto-shorts\dist\cli.js:423:52)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.8.1