search

OpenVisualCloud / SVT-HEVC

SVT HEVC encoder. Scalable Video Technology (SVT) is a software-based video coding technology that is highly optimized for Intel® Xeon® processors. Using the open source SVT-HEVC encoder, it is possible to spread video encoding processing across multiple Intel® Xeon® processors to achieve a real advantage of processing efficiency.
Other
519 stars 169 forks source link

[macOS]Could not allocate enough memory for channel 1 #495

Closed yurikoles closed 4 years ago

yurikoles commented 4 years ago

I'm trying to encode akiyo_qcif.yuv sample on macOS, but I see the following error:

$ ../Bin/Debug/SvtHevcEncApp -- -w 176 -h 144 -i /tmp/akiyo_qcif.yuv -b /tmp/akiyo.mp4
<...>
Error at sem_open: Too many open files
allocate memory failed, at ../Source/Lib/Codec/EbSystemResourceManager.c, L30
Encoding          
Could not allocate enough memory for channel 1
Encoder finished
Full output 
$ ../Bin/Debug/SvtHevcEncApp -w 176 -h 144 -i /usr/local/Cellar/svt-hevc/1.4.3/share/svt-hevc/akiyo_qcif.yuv -b /tmp/akiyo.mp4
-------------------------------------------
SVT-HEVC Encoder
SVT [version]:  SVT-HEVC Encoder Lib v1.4.3
SVT [build]  :  GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.33.17)  64 bit
LIB Build date: Mar 15 2020 11:10:02
-------------------------------------------
Number of logical cores available: 6
Number of PPCS 107
------------------------------------------- 
SVT [config]: Main Profile  Tier (auto) Level (auto)    
SVT [config]: EncoderMode / Tune                            : 7 / 1 
SVT [config]: EncoderBitDepth / CompressedTenBitFormat / EncoderColorFormat         : 8 / 0 / 1
SVT [config]: SourceWidth / SourceHeight / InterlacedVideo              : 176 / 144 / 0
SVT [config]: FrameRate / Gop Size                          : 60 / 64 
SVT [config]: HierarchicalLevels / BaseLayerSwitchMode / PredStructure          : 3 / 0 / 2 
SVT [config]: BRC Mode / QP / LookaheadDistance / SceneChange               : CQP / 32 / 17 / 1 
SVT [config]: BitRateReduction / ImproveSharpness                   : 0 / 0 
SVT [config]: tileColumnCount / tileRowCount / tileSliceMode / Constraint MV        : 1 / 1 / 0 / 0
SVT [config]: De-blocking Filter / SAO Filter                       : 1 / 1 
SVT [config]: HME / UseDefaultHME                           : 1 / 1 
SVT [config]: MV Search Area Width / Height                         : 16 / 7 
SVT [config]: HRD / VBV MaxRate / BufSize / BufInit                 : 0 / 0 / 0 / 90
SVT [config]: More configurations for debugging:
SVT [config]: Channel ID / ActiveChannelCount                       : 0 / 1
SVT [config]: Number of Logical Processors / Target Socket              : 0 / -1
SVT [config]: Threads To RT / Thread Count / ASM Type                   : 1 / 0 / 1
SVT [config]: Speed Control / Injector Frame Rate                   : 0 / 60
SVT [config]: MaxCLL / MaxFALL / Output Reconstructed YUV               : 0 / 0 / 0
SVT [config]: MasterDisplayColorVolume / DolbyVisionProfile             : 0 / 0
SVT [config]: DisplayPrimaryX[0], DisplayPrimaryX[1], DisplayPrimaryX[2]        : 0 / 0 / 0
SVT [config]: DisplayPrimaryY[0], DisplayPrimaryY[1], DisplayPrimaryY[2]        : 0 / 0 / 0
SVT [config]: WhitePointX (0, 0) / DisplayMasteringLuminance Range [0 ~ 0]          
SVT [config]: Constrained Intra / HDR / Code VPS SPS PPS / Code EOS         : 0 / 0 / 1 / 0
SVT [config]: Sending VUI / Temporal ID / VPS Timing Info               : 0 / 1 / 1
SVT [config]: SEI Message:
SVT [config]: AccessUnitDelimiter / BufferingPeriod / PictureTiming         : 0 / 0 / 0
SVT [config]: RegisteredUserData / UnregisteredUserData / RecoveryPoint         : 0 / 0 / 0
------------------------------------------- 
Error at sem_open: Too many open files
allocate memory failed, at ../Source/Lib/Codec/EbSystemResourceManager.c, L30
Encoding          
Could not allocate enough memory for channel 1
Encoder finished

What I had tried:

Output of successful run from VSCode 
@"\r\n"
@"Channel 1\r\n"
@"Average Speed:\t\t301.73 fps\r\n"
@"Total Encoding Time:\t3977 ms\r\n"
@"Total Execution Time:\t4244 ms\r\n"
@"Average Latency:\t422 ms\r\n"
@"Max Latency:\t\t474 ms\r\n"
@"SVT: you have no memory leak\r\r\n"
@"Encoder finished\r\n"
The program '/Users/yurikoles/work/SVT-HEVC/Bin/Debug/SvtHevcEncApp' has exited with code 0 (0x00000000).
1480c1 commented 4 years ago

https://github.com/OpenVisualCloud/SVT-AV1/blob/master/README.md#linux-operating-systems-64-bit

Try this

tianjunwork commented 4 years ago

Hi @yurikoles , does above instruction work for you? Do you need further assist?

yurikoles commented 4 years ago

Hi @1480c1 and @tianjunwork,

Sorry for delay, I don't wanted to reboot my system. I already had some limits bumped and I bumped even more settings. Those limits were applied only after reboot. And it works!

tianjunwork commented 4 years ago

Thanks for the update @yurikoles.

Selur commented 4 years ago

Sorry, but I don't get what the solution for this is. Calling:

selur@Selurs-Mac MacOS % ./ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "/Users/selur/Desktop/test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | ./SvtHevcEncApp -i stdin -fps-num 25 -fps-denom 1 -n 429 -profile 2  -b "/Users/selur/temp/test_2020-08-30@10_50_25_1810_01.265"

I get:

-------------------------------------------
SVT-HEVC Encoder
SVT [version]:  SVT-HEVC Encoder Lib v1.5.0
SVT [build]  :  GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1103.0.32.62)  64 bit
LIB Build date: Aug 29 2020 17:37:08
-------------------------------------------
Number of logical cores available: 16
Number of PPCS 54
------------------------------------------- 
SVT [config]: Main10 Profile    Tier (auto) Level (auto)    
SVT [config]: EncoderMode / Tune                            : 7 / 1 
SVT [config]: EncoderBitDepth / CompressedTenBitFormat / EncoderColorFormat         : 10 / 0 / 1
SVT [config]: SourceWidth / SourceHeight / InterlacedVideo              : 640 / 352 / 0
SVT [config]: Fps_Numerator / Fps_Denominator / Gop Size / IntraRefreshType         : 25 / 1 / 24 / -1
SVT [config]: HierarchicalLevels / BaseLayerSwitchMode / PredStructure          : 3 / 0 / 2 
SVT [config]: BRC Mode / QP / LookaheadDistance / SceneChange               : CQP / 32 / 17 / 1 
SVT [config]: BitRateReduction / ImproveSharpness                   : 0 / 0 
SVT [config]: tileColumnCount / tileRowCount / tileSliceMode / Constraint MV        : 1 / 1 / 0 / 0
SVT [config]: De-blocking Filter / SAO Filter                       : 1 / 1 
SVT [config]: HME / UseDefaultHME                           : 1 / 1 
SVT [config]: MV Search Area Width / Height                         : 16 / 7 
SVT [config]: HRD / VBV MaxRate / BufSize / BufInit                 : 0 / 0 / 0 / 90
------------------------------------------- 
Error at sem_open: Too many open files
allocate memory failed, at /Users/selur/workspace/SVT-HEVC/Source/Lib/Codec/EbSystemResourceManager.c, L30
Encoding          
Could not allocate enough memory for channel 1
Encoder finished

Should I add some additional command line parameter or do I need to add some additional parameter when compiling SvtHevcEncApp?

1480c1 commented 4 years ago

It seems one of the lines related to macOS semaphores was removed from the readme

https://github.com/OpenVisualCloud/SVT-AV1/issues/178#issuecomment-481452141

you can try setting ulimit -n 1000 to set to limit higher temporarily

Selur commented 4 years ago

Okay, good and bad news. :) good: that seems to help with that problem,...

Selurs-Mac:MacOS selur$ ulimit -n 1000
Selurs-Mac:MacOS selur$ ./ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "/Users/selur/Desktop/test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0  -f yuv4mpegpipe - | ./SvtHevcEncApp -i stdin -fps-num 25 -fps-denom 1 -n 429 -profile 2  -b "/Users/selur/temp/test_2020-08-30@11_32_57_1810_01.265"
-------------------------------------------
SVT-HEVC Encoder
SVT [version]:  SVT-HEVC Encoder Lib v1.5.0
SVT [build]  :  GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1103.0.32.62)  64 bit
LIB Build date: Aug 29 2020 17:37:08
-------------------------------------------
Number of logical cores available: 16
Number of PPCS 54
------------------------------------------- 
SVT [config]: Main10 Profile    Tier (auto) Level (auto)    
SVT [config]: EncoderMode / Tune                            : 7 / 1 
SVT [config]: EncoderBitDepth / CompressedTenBitFormat / EncoderColorFormat         : 10 / 0 / 1
SVT [config]: SourceWidth / SourceHeight / InterlacedVideo              : 640 / 352 / 0
SVT [config]: Fps_Numerator / Fps_Denominator / Gop Size / IntraRefreshType         : 25 / 1 / 24 / -1
SVT [config]: HierarchicalLevels / BaseLayerSwitchMode / PredStructure          : 3 / 0 / 2 
SVT [config]: BRC Mode / QP / LookaheadDistance / SceneChange               : CQP / 32 / 17 / 1 
SVT [config]: BitRateReduction / ImproveSharpness                   : 0 / 0 
SVT [config]: tileColumnCount / tileRowCount / tileSliceMode / Constraint MV        : 1 / 1 / 0 / 0
SVT [config]: De-blocking Filter / SAO Filter                       : 1 / 1 
SVT [config]: HME / UseDefaultHME                           : 1 / 1 
SVT [config]: MV Search Area Width / Height                         : 16 / 7 
SVT [config]: HRD / VBV MaxRate / BufSize / BufInit                 : 0 / 0 / 0 / 90
------------------------------------------- 
Encoding          Illegal instruction: 4

but now I get the 'Illegal instruction: 4', adding '-asm 0' the encoding works.

But my guess is that this is thanks to the fact that I'm running MacOS in a VMWare on a system using Intel(R) Xeon(R) CPU E5640 @ 2.67GHz which simply is too old. :)

-> thanks for the help with the ulimit

yurikoles commented 4 years ago

Issue should be reopened since instructions on limits for macOS was removed from readme

tianjunwork commented 4 years ago

Hi @yurikoles , macOS is never officially claimed to be supported and maintained. Thanks to @1480c1 from the community, SVT-HEVC can be built and run on macOS. Hi @Selur, I wonder if you can share your latest recipe to use SVT-HEVC on macOS in this page.

Selur commented 4 years ago

@tianjunwork: Since I managed to get MacOS running in a VM I'm currently working on getting the tool I develop Hybrid running on latest MacOS again. What I do in Hybrid is use either ffmpeg or a Vapoursynth script and vspipe to decode filter and pipe content to SVT-HEVC. :) So not really much to tell about atm. but I'm working on it and try frequently update wip builds of Hybrid.