Currently chunks are downloaded to memory and written to disk. When you are downloading a large number of HD m3u8 streamins in parallel, this can lead to a memory crunch. By using file objects and making use of AES cipher's ability to encrypt/decrypt in chunks, we can reduce this memory utilization.
To do this, two classes Encrypt and Decrypt have been created in crypto.py, which are subclassed from StringIO. This handles the chunked (in multiple of 16 bytes) encryption and decryption.
In consumer.py, shutil.copyfileobj is used to copy the contents from the urllib2 request object.
In pkcs7.py, the statement
amount_to_pad = self.block_size - (text_length % self.block_size)
will never result in amount_to_pad = 0, so the next lines are not necessary
if amount_to_pad == 0: amount_to_pad = self.block_size
Currently chunks are downloaded to memory and written to disk. When you are downloading a large number of HD m3u8 streamins in parallel, this can lead to a memory crunch. By using file objects and making use of AES cipher's ability to encrypt/decrypt in chunks, we can reduce this memory utilization.
To do this, two classes Encrypt and Decrypt have been created in crypto.py, which are subclassed from StringIO. This handles the chunked (in multiple of 16 bytes) encryption and decryption.
In consumer.py, shutil.copyfileobj is used to copy the contents from the urllib2 request object.
In pkcs7.py, the statement
amount_to_pad = self.block_size - (text_length % self.block_size)will never result in amount_to_pad = 0, so the next lines are not necessary
if amount_to_pad == 0: amount_to_pad = self.block_size