relimS
commented
2 years ago Disclaimer: I am not a developer on this project so I might be completely wrong. If you are a developer then correct me. What does 5 FPS mean? 5 frames per second so each frame is 1/5 of a second (200 milliseconds) The AI is trained to predict the motion of an object. Let's say an object is moving a 10 centimeter per second the framerate is 10 FPS, for each frame, that object moved 1 centimeter. With this information, the AI can predict how far the object will move in 1/20 or 1/30 of a second and draw them essentially turning a 10 FPS video to 20 or 30 FPS. For a 10 FPS video, the AI only needs to predict the motion in between each frame (1/10 of a second or 100 milliseconds) With 5 FPS and large movements, it is not enough information for the AI to draw extra frames therefore less accurate. There is not much that can be done to resolve this except for capturing at a higher framerate. Training the AI may help a bit but the core problem is still there. The only way I can see it working is to (hire an animator to) animate movement from each frame to another as a human is more informed than a bot then try the software again. In a nutshell, the only thing you can do is to animate the 5 FPS and make it to a 10 FPS (it will take days even for a professional and hiring is going to cost a lot)
urix
I find your model behave bad on low rate video with large movement (50->30fps), I wonder how to fix this? Maybe Train on low rate videos can help? Thanks a lot.