optimized delay between downloads

manusimidt / py-xbrl

Python-based parser for parsing XBRL and iXBRL files

https://py-xbrl.readthedocs.io/en/latest/

GNU General Public License v3.0

100 stars 37 forks source link

optimized delay between downloads #65

Closed mrx23dot closed 2 years ago

mrx23dot commented 2 years ago

Delay between 2 downloads still exists. But only executes if needed.

eg.

download file1,
do something else for long time,
no delay for file2 since delay already elapsed.

Also no delay if only downloading 1file.

Very useful if downloading many fillings.

//find_entry_file might have slipped in, github ui is horrible, might need removing

manusimidt commented 2 years ago

@mrx23dot Thank you for your contributions! I am sorry that I am currently so slow with answering the issues and pull requests. Since I've been on vacation for two weeks and am currently moving to another city (and even country) for my Master's degree, I unfortunately have very little time at the moment. I will look at your pull requests within the next 14 days and implement/comment.

Greetings, Manu

manusimidt commented 2 years ago

@mrx23dot The optimized delay works great! Will definetly save much time in comparison to the stupid delay that was implemented before. Thanks! 😊

mrx23dot commented 2 years ago

Good to hear, aws still gave me hard time with SEC, saying I'm exceeding rate, turns out it uses shared IP. Also it helps a lot when parsing takes more time than the backoff time.