search

twitter / the-algorithm

Source code for Twitter's Recommendation Algorithm
https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm
GNU Affero General Public License v3.0
62.51k stars 12.16k forks source link

Raise exception when a module is not found #885

Open MurphyAdam opened 1 year ago

MurphyAdam commented 1 year ago

https://github.com/twitter/the-algorithm/blob/ec83d01dcaebf369444d75ed04b3625a0a645eb9/trust_and_safety_models/toxicity/load_model.py#L19

As an example above, you could see that we are wrapping an import in a try-except block, but we only use the except for printing that the module doesn't exists..which seems a little redundant since that's what you'd get anyways if a module doesn't exist. In the same file, we are using such import as if they're guaranteed to be imported? What's the logic here? Either have defaults, or let the the python import do it's job.

impoverishedowl commented 1 year ago

Hello, could you perhaps create a pull request for this? We don't have enough time to implement this as we are working on boosting engagement for Elon Musk's posts, we hope you understand our situation.

Thank you, have a great day!

--- edit from admins: this user is impersonating a Twitter employee.

nobodyindustries commented 1 year ago

Is this happening in the Ligma engine?

dagelf commented 1 year ago

Maybe this helps somehow in their CI getting it in stdout rather than stderr?

MurphyAdam commented 1 year ago

@nobodyindustries who the hell is Steve Jobs?

MurphyAdam commented 1 year ago

@impoverishedowl totally understandable, I will rewrite it in rust and create a PR. :+1:

MurphyAdam commented 1 year ago

@dagelf I couldn't find any tests for it, I hope they just excluded them and not that they never existed, that still would be a bad idea as if they want people to refactor stuff, then they need to have tests.

jdm-twtr commented 1 year ago

@impoverishedowl is not a Twitter employee.