globocom / hlsclient

Python HLS Client
Other
105 stars 21 forks source link

Avoid downloading entire segments to memory #26

Closed mahendra closed 12 years ago

mahendra commented 12 years ago

Currently chunks are downloaded to memory and written to disk. When you are downloading a large number of HD m3u8 streamins in parallel, this can lead to a memory crunch. By using file objects and making use of AES cipher's ability to encrypt/decrypt in chunks, we can reduce this memory utilization.

To do this, two classes Encrypt and Decrypt have been created in crypto.py, which are subclassed from StringIO. This handles the chunked (in multiple of 16 bytes) encryption and decryption.

In consumer.py, shutil.copyfileobj is used to copy the contents from the urllib2 request object.

In pkcs7.py, the statement amount_to_pad = self.block_size - (text_length % self.block_size)

will never result in amount_to_pad = 0, so the next lines are not necessary

if amount_to_pad == 0: amount_to_pad = self.block_size