TypeError: unsupported operand type(s) for -=: 'set' and 'list'

[c0den]

Processing source URLs Completed source processing Downloading samples, check log for details Traceback (most recent call last): File "./maltrieve.py", line 352, in main() File "./maltrieve.py", line 326, in main malware_urls -= past_urls TypeError: unsupported operand type(s) for -=: 'set' and 'list'

[ghost]

Current implementation is working fine with Python version 2.7.9 Ensure you have installed the requirements

pip install -r requirements.txt

┌─[10:33:52]─[alienone@binary] └──> maltrieve $ >> /home/alienone/anaconda/bin/python maltrieve.py Processing source URLs Completed source processing Downloading samples, check log for details <type 'set'>

[c0den]

@alienone It runs fine on the first launch but after it creates the urls.json file it won't run again and I get that error. Just to verify I pulled it down in a sandbox did a pip install -r requirements.txt and am still getting the same error after the second run.

[ghost]

Running all the way through - will see if I can recreate your problem set via running 2nd round after 1st round completion.

[ghost]

Just for the record the type(malware_urls) on line 309 on 1st round execution comes back as expected type set. c0den since you can run 2nd round - put a print statement at line 309

print(type(malware_urls))
malware_urls -= past_urls

[c0den]

Have you pulled the recent changes since yesterday? Line 309 within my maltrieve.py contains " 'http://support.clean-mx.de/clean-mx/rss?scope=viruses&limit=0%2C64': process_xml_list_title, "

[ghost]

Hmm odd - I just forked the entire project - let me do a git pull - So add at line 325 print statement so looks like this for lines 325 and 326

print(type(malware_urls))
malware_urls -= past_urls

[ghost]

pulled from main repo so we are on same sheet of music now - line 326

[c0den]

I get the following after adding:

print(type(malware_urls))
print(type(past_urls))

This is first run: -> % ./maltrieve.py
Processing source URLs Completed source processing Downloading samples, check log for details <type 'set'> <type 'set'>

This is second run: -> % ./maltrieve.py
Processing source URLs Completed source processing Downloading samples, check log for details <type 'set'> <type 'list'> Traceback (most recent call last): File "./maltrieve.py", line 353, in main() File "./maltrieve.py", line 327, in main malware_urls -= past_urls TypeError: unsupported operand type(s) for -=: 'set' and 'list'

I think it is line 344 causing the type change:

341   if past_urls:
342        logging.info('Dumping past URLs to file')
343       with open('urls.json', 'w') as urlfile:
344            json.dump(list(past_urls), urlfile)

[c0den]

Sorry This is first run: -> % ./maltrieve.py
Processing source URLs Completed source processing Downloading samples, check log for details type 'set' type 'set'

This is second run: -> % ./maltrieve.py
Processing source URLs Completed source processing Downloading samples, check log for details type 'set' type 'list' Traceback (most recent call last): File "./maltrieve.py", line 353, in main() File "./maltrieve.py", line 327, in main malware_urls -= past_urls TypeError: unsupported operand type(s) for -=: 'set' and 'list'

[ghost]

Correction - Line 295 needs to output a set data structure and not a list data structure

a_set = {1, 2, 3, 4}
a_list = [5, 6, 7, 8]
b_set = {3, 4, 7, 8}
a_set -= b_set
print(a_set)
a_set -= a_list

set([1, 2])

---

TypeError Traceback (most recent call last)

in () 4 a_set -= b_set 5 print(a_set) ----> 6 a_set -= a_list 7 TypeError: unsupported operand type(s) for -=: 'set' and 'list' ``` ```

[krmaxwell]

Can somebody try that?

[ghost]

There we go - testing

[c0den]

Confirmed working now.