[X] I have searched open or closed issues and found nothing related to my idea.
[X] I have read the Documentation and it doesn't mention anything about my idea.
[X] To my best knowledge, my idea wouldn't break something for other users.
Describe your Idea
This idea targets two problem that currently DeFFcode APIs fails to address:
Discarding default FFmpeg parameters: Certain default FFmpeg parameters in DeFFcode API such as -framerate, -pix_fmt, -size cannot be removed/discarded from Decoding Pipeline in any way possible, and thereby present in every Pipeline that DeFFcode API has ever created. These parameters were thought to be extremely necessary for Pipeline to work but my recent tests shown that even if we remove these parameters the Decoding pipeline (with some changes) works totally fine. And instead, FFmpeg itself handles these parameter in best possible manner based on source and decoder provided by the user.
Using FFmpeg Filters: Currently DeFFcode APIs fails to recognize values from filters like fps, format, scale which tends to break the pipeline with artifacts appearing on the output frames. This is due to fact that -framerate, -pix_fmt, -size like parameter are given advantage over filters values by the DeFFCode APIs and was primarily used by them to decode frames. For example if we scale=320:240 filter as follows, the output will be improper(with artifacts) since API tries to use FFmpeg parameter -s i.e. equal to source video frame size(for e.g. 1920x1080) which is way higher than require 320x240 as defined by the filter:
# import the necessary packages
from deffcode import FFdecoder
import cv2
# define params and custom filters
ffparams = {
"-vf": "scale=320:240", # framerate=60fps
}
# initialize and formulate the decoder with params and custom filters
decoder = FFdecoder(
"t.mp4", frame_format="bgr24", verbose=True, **ffparams
).formulate()
# grab the BGR24 frames from decoder
for frame in decoder.generateFrame():
# check if frame is None
if frame is None:
break
# {do something with the frame here}
# Show output window
cv2.imshow("Invalid Output", frame)
# check for 'q' key if pressed
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
break
# close output window
cv2.destroyAllWindows()
# terminate the decoder
decoder.terminate()
Use Cases
Both Discarding default FFmpeg parameters and using FFmpeg Filters, are absolute necessity for decoding/transcoding video frames in GPU, as with FFmpeg commands like ffmpeg -y -vsync 0 -hwaccel_output_format cuda -hwaccel cuda -c:v h264_cuvid -i input.mp4 -vf "scale_npp=format=yuv420p,hwdownload,format=yuv420p,fps=25.0" -f rawvideo - fails to work with FFmpeg parameters -framerate, -pix_fmt, -size and throws Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scale_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented error. Therefore, implementing both these features will resolves issue with decoding/transcoding with DeFFcode APIs automatically (thus resolving #30).
Issue guidelines
Issue Checklist
Describe your Idea
This idea targets two problem that currently DeFFcode APIs fails to address:
-framerate
,-pix_fmt
,-size
cannot be removed/discarded from Decoding Pipeline in any way possible, and thereby present in every Pipeline that DeFFcode API has ever created. These parameters were thought to be extremely necessary for Pipeline to work but my recent tests shown that even if we remove these parameters the Decoding pipeline (with some changes) works totally fine. And instead, FFmpeg itself handles these parameter in best possible manner based on source and decoder provided by the user.Using FFmpeg Filters: Currently DeFFcode APIs fails to recognize values from filters like
fps
,format
,scale
which tends to break the pipeline with artifacts appearing on the output frames. This is due to fact that-framerate
,-pix_fmt
,-size
like parameter are given advantage over filters values by the DeFFCode APIs and was primarily used by them to decode frames. For example if wescale=320:240
filter as follows, the output will be improper(with artifacts) since API tries to use FFmpeg parameter-s
i.e. equal to source video frame size(for e.g.1920x1080
) which is way higher than require320x240
as defined by the filter:Use Cases
Both Discarding default FFmpeg parameters and using FFmpeg Filters, are absolute necessity for decoding/transcoding video frames in GPU, as with FFmpeg commands like
ffmpeg -y -vsync 0 -hwaccel_output_format cuda -hwaccel cuda -c:v h264_cuvid -i input.mp4 -vf "scale_npp=format=yuv420p,hwdownload,format=yuv420p,fps=25.0" -f rawvideo -
fails to work with FFmpeg parameters-framerate
,-pix_fmt
,-size
and throwsImpossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scale_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented
error. Therefore, implementing both these features will resolves issue with decoding/transcoding with DeFFcode APIs automatically (thus resolving #30).Any other Relevant Information?
No response