Big size of video files

[miolion]

Xiboside could not get huge size of video files from CMS, so I had to changes code in xthread.py as follows.

            try:
                with open(file_path, 'wb') as f:
                    for offset in range(0, int(float(entry.size)), 1024*1024*2):
                        param.chuckSize = str(1024*1024*2)
                        param.chunkOffset = str(offset)
                        resp = cl.send_request('GetFile', param)
                        if resp:
                            f.write(resp.content)
                            f.flush()
                            os.fsync(f.fileno())
                            downloaded = True
            except IOError:
                self.log.error('Download failed: %s' % file_path)

[ajiwo]

How big is the file size? does that patch also work for small video filesize? This could be the fix, but I also see there is an option in CMS to send big file not via xmds but over feature called sendfile, which is still not implemented by xiboside.

[miolion]

Hi Amin,

In my case, it was about up to 6MB on a small PC with very limited memory. If I run Xiboside on another virtual machine, I could see the following error message.

ERROR:xmds:Server raised fault: 'Allowed memory size of 134217728 bytes exhausted (tried to allocate 276134948 bytes)' ERROR:xmds:Server raised fault: 'Allowed memory size of 134217728 bytes exhausted (tried to allocate 134294353 bytes)' Terminated

After I applied the changes to Xiboside, it worked fine so that I just see the working with mplayer today. Now I am able to play the content at http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 for my test.

Regards, SJ

[ajiwo]

ERROR:xmds:Server raised fault: 'Allowed memory size of 134217728 bytes exhausted (tried to allocate 276134948 bytes)'

I think there are recommended memory values for php.ini in xibo-cms, it's showed somewhere on xibo-cms installation page.

[miolion]

Hi,

I think that the error is not related to php.ini and Xibo CMS server side. And my PHP settings allowed the file post and upload size are about more than 8GB and up to 16GB.

The PHP settings are about the max file size, but this error is related to allocation on player side to get the whole file in a memory buffer.

You can test with the exact video file at http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 that try to allocate the memory on the client side according to the amount of the file size to transfer at once.

So we need to chop the file as a chunk of file sending several times, and my fixed code is attached for your review, xthread.py.zip

Best regards, SJ

P.S. Not only that error message from CMS, I could also see an error on client side when getting huge files if it is short in Player's memory, out of memory in client side before CMS raised the shortage of memory at server side.

[ajiwo]

So we need to chop the file as a chunk of file sending several times

Correct. Your patch could be an enhancement, it reduces memory usage on base64 encoding on server side. What is the value of memory_limit on your php.ini?

The PHP settings are about the max file size, but this error is related to allocation on player side to get the file.

So it should be around this line xmds.py:309

[miolion]

I didn't change it as it was given from PHP default setting.

memory_limit = 128M

What I changed are these three things.

max_execution_time = 180 ;30
post_max_size = 8G ;8M
upload_max_filesize = 8G ;2M

Here is my settings of PHP, php.ini.zip

Regarding xmds.py:309, thank you for your comments on xmds,py.

I couldn't yet see the xmds implementation as I am very new to any signage project, I am just going to start studying on XMDS,,,

[miolion]

https://github.com/ajiwo/xiboside/compare/master@%7B1day%7D...miolion:patch-1?diff=split&name=patch-1

Since I was very new to Github, I have been digging into how to use it as a beginner....