globocom / m3u8

Python m3u8 Parser for HTTP Live Streaming (HLS) Transmissions
Other
2.03k stars 471 forks source link

feat: add version_matching module #341

Closed davigps closed 4 months ago

davigps commented 11 months ago

As described at #220

I implemented an initial version of a version_matching module.

davigps commented 5 months ago

All errors are now raised when we finish to validate a file in strict mode, example:

Traceback (most recent call last):
  File "/home/..././invalid.py", line 4, in <module>
    content = m3u8.parse(invalid_versioned_playlists.M3U8_RULE_IV, strict=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../m3u8/m3u8/parser.py", line 82, in parse
    raise Exception(found_errors)
Exception: [VersionMatchingError(line_number=2, line='#EXT-X-KEY: METHOD=AES-128, IV=0x123456789ABCDEF0123456789ABCDEF0, URI="https://example.com/key.bin"', how_to_fix='Change the protocol version to 2 or higher.', description='You must use at least protocol version 2 if you have IV in EXT-X-KEY.'), VersionMatchingError(line_number=4, line='#EXTINF: 10.0,', how_to_fix='Change the protocol version to 3 or higher.', description='You must use at least protocol version 3 if you have floating point EXTINF duration values.')]

I thought about printing them, but I didn't see any other prints in Parse, so I just raised

leandromoreira commented 5 months ago

All errors are now raised when we finish to validate a file in strict mode, example:

Traceback (most recent call last):
  File "/home/..././invalid.py", line 4, in <module>
    content = m3u8.parse(invalid_versioned_playlists.M3U8_RULE_IV, strict=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../m3u8/m3u8/parser.py", line 82, in parse
    raise Exception(found_errors)
Exception: [VersionMatchingError(line_number=2, line='#EXT-X-KEY: METHOD=AES-128, IV=0x123456789ABCDEF0123456789ABCDEF0, URI="https://example.com/key.bin"', how_to_fix='Change the protocol version to 2 or higher.', description='You must use at least protocol version 2 if you have IV in EXT-X-KEY.'), VersionMatchingError(line_number=4, line='#EXTINF: 10.0,', how_to_fix='Change the protocol version to 3 or higher.', description='You must use at least protocol version 3 if you have floating point EXTINF duration values.')]

I thought about printing them, but I didn't see any other prints in Parse, so I just raised

I think raising it's the best we can do, I suggested we use the group exception so the natural print will be easier to read.

davigps commented 5 months ago

I suggested we use the group exception

As I said, I wanted to use ExceptionGroup, but it is from Python 3.11 and the project requires only Python3.7+, I dont know if we can use it here. 😕

leandromoreira commented 5 months ago

I suggested we use the group exception

As I said, I wanted to use ExceptionGroup, but it is from Python 3.11 and the project requires only Python3.7+, I dont know if we can use it here. 😕

that's okay, thanks for the efforts

leandromoreira commented 5 months ago

A minor feature we can add, if the verification list keeps growing, is a map/dict like structure to avoid N*N:


if hash(line) in verifications:
  verifications[verification_hash(line)].valid?

def verification_hash(line):
  if line.contains("version"):
    return "version"
mauricioabreu commented 5 months ago

Awesome work, @davigsousa

mauricioabreu commented 5 months ago

Could you check why it is failing for 3.7 version @davigsousa ?

leandromoreira commented 5 months ago

It looks like related to:

available_rules: List[type[VersionMatchRuleBase]] = []

Maybe replacing type[ by typing.Type or type(.

https://docs.python.org/3.7/library/typing.html?highlight=type#typing.Type

tests/test_invalid_versioned_playlists.py:9: in <module>
    import m3u8
m3u8/__init__.py:9: in <module>
    from m3u8.model import (
m3u8/model.py:8: in <module>
    from m3u8.parser import format_date_time, parse
m3u8/parser.py:17: in <module>
    from m3u8 import protocol, version_matching
m3u8/version_matching.py:4: in <module>
    from m3u8.version_matching_rules import VersionMatchingError, available_rules
m3u8/version_matching_rules.py:108: in <module>
    ValidEXTXBYTERANGEOrEXTXIFRAMESONLY,
E   TypeError: 'type' object is not subscriptable
mauricioabreu commented 4 months ago

Hello @davigsousa, are you still working on these awesome changes? If you don't have time to complete the task, I can take over the PR from here and handle the finalization and merging.

davigps commented 4 months ago

Hi @mauricioabreu , unfortunately I have very little time for open source lately, sorry for that! I wanted to finish the work here, but thank you for your help, I'll accept it! We could speed up the merge here.

You can take over, thank you very much!