sockysec / Telerecon

A reconnaissance framework for researching and investigating Telegram.
828 stars 115 forks source link

Make the scrape process handle more entity #13

Closed NeonBubbles closed 8 months ago

NeonBubbles commented 9 months ago
  1. Use subprocess to handle the event loop of every script in launcher.py
  2. The entity of post.sender in userscraper.py might be a Channel, and original_message.sender might be a "NoneType" To avoid the error interrupting the scrape process, use the isinstance function to identify the entity and use the try-except statement to handle the "NoneType" object.
sockysec commented 8 months ago

@NeonBubbles, thanks a bunch for putting work into this, these changes look great. Out of curiosity, have you done much testing of this? Particularly whether the launcher subprocess changes impact any of the other scripts (especially the return to launcher function after completion)? Additionally, whether the improvements to userscraper.py could/should also be applied to usermultiscraper.py ?

NeonBubbles commented 8 months ago

After using the subprocess, I manually tested the userscrape.py (option 3); it worked fine. However, I have yet to do any tests on other scripts. I will try to carry out the tests tonight.

NeonBubbles commented 8 months ago

The subprocess method is not an excellent way to handle the event loop in your program; the importlib is better but requires more code change. Maybe we can fix them in another PR so I keep the launcher.py unchanged. Also, I made the exact change to the usermultiscraper.py as I did to the userscraper.py. And add some private files to .gitignore. Last, I made some optimizations to setup.py so that it can be used without needing the detailsshell.py file.

I am unfamiliar with Git, so the commit numbers I made might be annoying. If you mind, I can reopen a PR. @sockysec

sockysec commented 8 months ago

@NeonBubbles thanks for this, this method appears to correctly solve the event looping issue. Did some testing myself and this all looks great. Will merge the request now. Thanks again