Closed sanketsharma411 closed 7 years ago
@sanketsharma411 that is a valid point. I've been thinking about how to do better error handling and debugging in that function.
My main concern was not to sweep under the rug major problems (e.g. if the downloaded file is corrupt, if the file format has changed), and also to avoid cascading errors . Perhaps a threshold could be set on how many records to ignore before quitting.
@hadiasghari yeah, that makes sense. But I am not entirely sure about how we plan to identify corrupt file, or different file format problems. Maybe we can add a boolean to mrtx.parse_mrt_file(..., skip_record_on_error = False)
that allows users to skip records with some problems. Making it clear that with this argument set, pyasn will try its best to parse the file, and simply ignore errors.
@sanketsharma411, could you do a pull request again? thanks
@hadiasghari #38
Merged.
So, right now, in
mrtx.parse_mrt_file(..)
, if a single record has an error, the whole file gets skipped. I guess it would help to modify the function such that a single line error does not skip the rest of the lines.The issue can be re-created as