Limit too high returning 502 Server Error

Hello,

I'm trying to automatically extract IOCs from malware families. However it looks like the limit parameter is preventing successful returns when it's higher than the number of IOCs over the last month, which I assume is the max search_scope available.

These are the tests i did with threatfox_query_malware.py :

python3 threatfox_query_malware.py CobaltStrike 50 Works fine, returns 50 results in 6 hours scope.

python3 threatfox_query_malware.py ClearFake 50

<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>502 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body></html>

python3 threatfox_query_malware.py ClearFake 20 Works fine, returns 20 results in 1 month scope.

By reducing the limit, I get results that I should also get with a higher limit. From what I've observed, the number of results cannot be different from the max result (limit), which is a problem for malware with little activity like ClearFake in this example which doesn't have 50 IOCs over the last month. The limit doesn't seem to be a limit but a fixed number of expected results which sometimes cannot be achieved and ends up with the previous error. I tried the same thing with other malware families and observed the same thing. Reducing the limit gives me results and a high limit returns an error.

Is this limitation intentional, and if so, is there a way to find the max limit without trial and error?

Also, is it possible to extend the search_scope to more than a month?

abusech / ThreatFox

Limit too high returning 502 Server Error #3