dtcooper / python-fitparse

Python library to parse ANT/Garmin .FIT files
http://pythonhosted.org/fitparse/
MIT License
738 stars 183 forks source link

Allow parsing files without storing all the data in memory #120

Closed pR0Ps closed 3 years ago

pR0Ps commented 3 years ago

Renames the FitFile class to BaseFitFile and removes the message cache from it. Adds a new class called FitFile that adds the cache back in. This preserves backwards compatibility while allowing users to parse files without storing all the contents in memory.

@xmedeko: This was built on your original #61 . Can you give it a quick review?

Obsoletes #61 Fixes #59 Closes: #72

xmedeko commented 3 years ago

@pR0Ps I work neither with FIT nor Python now. Just by a quick overview it seems technically OK 👍 , just in the get_messages() is copy&paste code

 names = None
        if name is not None:
            if is_iterable(name):
                names = set(name)
            else:
                names = set((name,))

which could be refactored to some (static) method like _get_names().

If a class is called BaseXyz then it's usually meant to be subclassed, while this BaseFitFile is a standalone, working class. That's why I have suggested the name FitFileDecoder in #72.

Also, #72 is not about memory caching only, it's about to strip more functionality from BaseFitFile. But close is as you like, as I have written, I no longer work with FIT.

pR0Ps commented 3 years ago

@xmedeko Gotcha. I've pushed a new commit that should be more in line with what you were going for.

xmedeko commented 3 years ago

Yep, that looks great! Even the fitdump should consume just a little memory now.