search

sipeed / MaixPy-v1

MicroPython for K210 RISC-V, let's play with edge AI easier
https://wiki.sipeed.com/maixpy
Other
1.68k stars 439 forks source link

AVI audio support, get frames from avi, AVI video recording without sound and can't be played #246

Open easy-and-simple opened 4 years ago

easy-and-simple commented 4 years ago

Hello, AVI video is recorded without sound and video recorded using video recording demo script have no index and can't be played I tried to assign MIC0 pins to I2S device 0 channel 0 but this did nothing Also documentation is awful ant there is no any example how to use microphone. Speach recognition is not working of course because firmware does not contain such module as SpeechRecognizer

anoken commented 4 years ago

Unable to play in demo_video_play.py the file recorded by demo_video_record.py.

demo_video_play.py OS Error :6

demo_video_record.py https://github.com/sipeed/MaixPy_scripts/blob/master/multimedia/video/demo_video_record.py

demo_video_play.py https://github.com/sipeed/MaixPy_scripts/blob/master/multimedia/video/demo_video_play.py

Lastest MaixPy is a bug. No header information in demo_video_record module.

This cause is 2d307ae47bc8c274cd6cb4c55e26c090abdfe6ba. There was no problem before this commit. https://github.com/sipeed/MaixPy/commit/2d307ae47bc8c274cd6cb4c55e26c090abdfe6ba

It is better to revert to the previous change.

junhuanchen commented 4 years ago

Unable to play in demo_video_play.py the file recorded by demo_video_record.py.

demo_video_play.py OS Error :6

demo_video_record.py https://github.com/sipeed/MaixPy_scripts/blob/master/multimedia/video/demo_video_record.py

demo_video_play.py https://github.com/sipeed/MaixPy_scripts/blob/master/multimedia/video/demo_video_play.py

Lastest MaixPy is a bug. No header information in demo_video_record module.

This cause is 2d307ae. There was no problem before this commit. 2d307ae

It is better to revert to the previous change.

Thank you very much for your question.

This 2d307ae fix is a change to make the normal player play normally. If you need to restore the previous function, it will be configured to allow recording.

Avi -> record_audio = args_parsed [ARG_audio] u_bool;

So if you want to be able to be played by record to play as before, you need to set it

V = video.open ("/sd/capture.avi", record=1, audio = True, interval=200000, quality=50)

There is a bug here, because there is no audio header, so it cannot be played by normal players.

To solve this problem, we should further fix the case where the play parse does not have audio headers.

junhuanchen commented 4 years ago

Hello, AVI video is recorded without sound and video recorded using video recording demo script have no index and can't be played I tried to assign MIC0 pins to I2S device 0 channel 0 but this did nothing Also documentation is awful ant there is no any example how to use microphone. Speach recognition is not working of course because firmware does not contain such module as SpeechRecognizer

  1. Speechrecognizer is being tested internally. It does not incorporate implementation and provide python test cases. Where did you know about this module.

  2. AVI does not realize audio recording function, so as you said, avi video recording without sound and can't be played, but it can already be played (normal video player, but no sound).

andreireanu commented 4 years ago

@junhuanchen Can I play a recorded video and treat it as a camera capture in order to run machine learning models on it?

easy-and-simple commented 4 years ago

anoken Thank you for info about header issue. I will not start to fix it because I have no idea what they are doing and why, but this doesn't make your replay less useful.

junhuanchen speachrecognizer is available as not working example in maixpy demo scripts that are available for download here on github If avi without sound is dumb but acceptable case, lack of microphone example is not acceptable at all. Why boards have microphone if they can't be used. I hope buggy speachrecognizer example is not burned my mic because it connects mic data line to I2S output data line instead to input data line. Also there is no option to append avi video that is very necessary option

AndreiUngureanu101 To analyze video it is necessary to have function that return single frame, but it look that there is no such function, so you can't analyze avi on maixpy boards. Also I dont understand from where you will get avi with resolution 320x240, and what is the sense to analyze it on maixpy if you can do that on pc or phone

andreireanu commented 4 years ago

@easy-and-simple Well the idea was to run everything on the micropython environment on the device without compiling anything for the pc. You could do workarounds but this feature would be helpful.

junhuanchen commented 4 years ago

anoken Thank you for info about header issue. I will not start to fix it because I have no idea what they are doing and why, but this doesn't make your replay less useful.

junhuanchen speachrecognizer is available as not working example in maixpy demo scripts that are available for download here on github If avi without sound is dumb but acceptable case, lack of microphone example is not acceptable at all. Why boards have microphone if they can't be used. I hope buggy speachrecognizer example is not burned my mic because it connects mic data line to I2S output data line instead to input data line. Also there is no option to append avi video that is very necessary option

AndreiUngureanu101 To analyze video it is necessary to have function that return single frame, but it look that there is no such function, so you can't analyze avi on maixpy boards. Also I dont understand from where you will get avi with resolution 320x240, and what is the sense to analyze it on maixpy if you can do that on pc or phone

No, I mean you can put the recorded avi on a PC to determine the problem, and make the sample public so that others can help you analyze and solve the problem.

Speachrecognizer has not yet merged this code in the firmware, and I think there is a misunderstanding.

Your focus should be on solving problems, not finding meaning.

For example, this is what I recorded.

https://streamja.com/0PZga

Change the suffix to avi after downloading.

andreireanu commented 4 years ago

@junhuanchen what about loading the video frame by frame? is it possible?

junhuanchen commented 4 years ago

@junhuanchen what about loading the video frame by frame? is it possible?

That's true.

see this https://stackoverflow.com/questions/885160/what-is-the-specifications-for-motion-jpeg

Neutree commented 4 years ago
PJPAC commented 3 years ago

Hi, The issue #246 is unclear to me as it seems to have diverged along the way into 2 distinct and interesting issues.

246 starts with sound not being recorded in the avi video file (overlapping with #187 @Matias159753 which was closed). About 10 months later @easy-and-simple 's post, I face the same issue (and frustration) as despite having a microphone and a camera on the same platform, the 2 do not seem to be associated (yet ?). Is there any plan to address this limitation ? Have audio headers/data be added to the avi files produced on Sipeed products ?

Then over the course of the discussion of issue #246, a new and different issue was brought up by @AndreiUngureanu101 relating to treating video recordings as sets of camera captures in order to run machine learning models on them. I have to admit that I am also interested in this separate issue. Has video.get_frame() been implemented ? Can we analyse (for instance) every other frame to ease the image analysis ? Why would we be limited to analysing avi files with a 320x240 resolution, when the Sipeed built-in cameras (in MaixAmigo for instance) have a native resolution of 640x480 ? Ideally I would like to analyse videos of different resolutions (1280x720, 640x480 and 1920 × 1080) and formats (.avi, .mov and .mkv). It seems to me that in terms of visualisation the screen part of MaixPy IDE should not be limited by the resolution of the components (camera & display) built-in a Sipeed device (Maix Amigo). Any advice and suggestions would be welcome.

Thanks