Rotate and crop exported video

[JamesXNelson]

Sorry if this is already answered in chinese, but I am a lowly monolingual user. :-)

I was trying to figure out how I can apply crop and rotate (for example, to remove the 90 degree camera rotation) when saving output video.

I'm also trying to figure out how to map color matrix effects into the filter DSL used in this project, but... one step at a time!

Any help is surely appreciated.

[JamesXNelson]

If I can't figure out the rotation aspect, I was thinking to extract the input rotation metadata, and then re-apply it to the output. All I know is, right now, all my exported videos come out 90 degree rotated (the input is rotated with rotation metadata, so it still plays right side up).

As for cropping, I'm digging around cgeVideoEncoder.cpp, but my C++ skills are much lower than my java skills. I'm not afraid to get my hands dirty, but some pointers on where to look / what to do would be appreciated.

[wysaid]

They did it by saving the rotation metadata, and write it back again. You can also do this: Use ffmpeg to read the rotation metadata, save the video with the right metadata. I will add this fix and the crop function into the demo later.

[JamesXNelson]

Many thanks!

To use ffmpeg directly, should I just write my own JNI bindings, or is there a better way to do so?

[JamesXNelson]

In case anyone else stumbles upon this, looking to replicate mainstream filters like sepia, I used the cge-tools repository to find the color matrix generator.

Sepia comes out as: @colormul mat 0.3588 0.7044 0.1368 0.299 0.587 0.114 0.2392 0.4696 0.0912

will play around a little more to see if I can do 4x4 matrices. If you like, I can add any common examples I transcribe for "common" filters like sepia to the list of demo filters. I think having examples of the cookie-cutter things people commonly look for in a filter library would help increase adoption. :-)

Great work by the way, this library has been, by far, the very best android tool I've found.

[wysaid]

@JamesXNelson

• You can just modify the offscreen-video processing function, and add the rotation feature. (with c++)

• That's cool! Just make a merge request for your newly created filters~😋

[JamesXNelson]

ok. hate to keep bothering you, but is that in cgeVideoEncoder?

My guess is I need to set an option on the context somewhere in

bool CGEVideoEncoderMP4::_openVideo(/*AVFormatContext *oc, AVCodec *codec, AVStream *st*/)
{
    int ret;
    AVCodecContext *c = m_context->pVideoStream->codec;
    AVFormatContext *oc = m_context->pFormatCtx;
    AVCodec *codec = m_context->pVideoCodec;

But I'm not sure what the options would be, or where to set it. I assume using the source of ffmpeg would help me figure it out, but if you know offhand what documentation to look at, it would certainly save me some time :-)

[JamesXNelson]

Looks like I need to use something like https://ffmpeg.org/doxygen/3.2/vf__rotate_8c.html

All it seems to be doing is flipping around the memory in each texture location; would I have to add rotate.h to have access to that, or should I just grab the memory flipping code and use it directly as an option to the save/encode function?

Seems like it could / should just be a custom filter that can be arbitrarily used.

I mean, save/encode should be able to read in rotation metadata and fix when saving, but if it was supported as a filter, I assume save/encode could actually add it, if missing, to the filter config string. Not sure if you would prefer to keep those matters separate, but it seems like a developer using this might already be tracking and fixing rotation (if it was a valid custom filter) manually.

As for a new parameter on the save video JNI call, perhaps -1 (default) could mean "figure out rotation and fix via a filter", 0 for "do nothing", and 1-4 as valid manual rotations to apply.

I'm not well versed in the architecture for filters to know if this even belongs there or not, so having some clear instructions from you would be nice. Thanks.

[wysaid]

Maybe you're looking for this: #118 Someone do it with ffmpeg command: String command = "-i " + file + " -c copy -metadata:s:v:0 rotate=90 " + ImageUtil.getPath() + "/rec_ChangeOrientation.mp4";

To use this command, you may need to recompile ffmpeg with the ffmpeg-command feature. See here

You may change the output video's metadata instead of change the real video data frames. You can do it in the cgeGenerateVideoWithFilter function. Just read the metadata of the input video, and write it into the output one. (Both of them have the same metadata at last)

[wysaid]

Fixing the video data directly can be a better choice. You can try your method. 😋 I will do that later. (I'm not so good at using FFMPEG, too😂)

[wysaid]

Hi, this commit may solve your problem

[JamesXNelson]

Ohhhh! So that pulls out the metadata, and then stuffs it back into the output?

That's really great.

I got pulled into a different project for another week or so, but will be happy to take this for a spin.

It also sounds like we might not be cropping to square anymore, so I might not even need to dig into ffmpeg. If I do wind up having to dig into that, are the compiled binaries easy to update? I.E. can I just add more to the header to access crop options, or do I need to recompile the full ffmpeg project?

If you want to close this issue, I'll be happy to test within the week.

[wysaid]

This commit will crash sometimes, I'm trying to find the problem.

[gzgang]

大师兄这个问题有解吗，我看你代码里面关闭了 写视频头信息内容的逻辑。

[wysaid]

@gzgang 是的， 有时候会崩溃， 暂时没找到原因， 如果你有时间就看看能不能找到原因吧