Closed osamahamad closed 2 years ago
osamahamad
commented
2 years ago 
axiom-scan mysubs.out -m nuclei -nuclei-templates nuclei-templates-custom/ -anew outputfile.txt
I got the following error: File "
Note that axiom-exec id will give similar error :
root@localhost:~# docker exec -it 9b4f794e8007 /usr/bin/bash
root@9b4f794e8007:/# 9b4f794e8007^C
root@9b4f794e8007:/# axiom-exec id
Traceback (most recent call last):
File "/usr/local/bin/interlace", line 33, in <module>
sys.exit(load_entry_point('Interlace==1.9.6', 'console_scripts', 'interlace')())
File "/usr/local/bin/interlace", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.10/dist-packages/Interlace-1.9.6-py3.10.egg/Interlace/interlace.py", line 6, in <module>
File "/usr/local/lib/python3.10/dist-packages/Interlace-1.9.6-py3.10.egg/Interlace/lib/core/output.py", line 4, in <module>
File "/usr/local/lib/python3.10/dist-packages/colorclass-2.2.0-py3.10.egg/colorclass/__init__.py", line 11, in <module>
File "/usr/local/lib/python3.10/dist-packages/colorclass-2.2.0-py3.10.egg/colorclass/codes.py", line 4, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)
tail: cannot open '/root/.axiom/tmp/exec/axiom-exec+1653141752/logs/*' for reading: No such file or directory
tail: no files remaining
Traceback (most recent call last):
File "/usr/local/bin/interlace", line 33, in <module>
sys.exit(load_entry_point('Interlace==1.9.6', 'console_scripts', 'interlace')())
File "/usr/local/bin/interlace", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.10/dist-packages/Interlace-1.9.6-py3.10.egg/Interlace/interlace.py", line 6, in <module>
File "/usr/local/lib/python3.10/dist-packages/Interlace-1.9.6-py3.10.egg/Interlace/lib/core/output.py", line 4, in <module>
File "/usr/local/lib/python3.10/dist-packages/colorclass-2.2.0-py3.10.egg/colorclass/__init__.py", line 11, in <module>
File "/usr/local/lib/python3.10/dist-packages/colorclass-2.2.0-py3.10.egg/colorclass/codes.py", line 4, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)
Traceback (most recent call last):
File "/usr/local/bin/interlace", line 33, in <module>
sys.exit(load_entry_point('Interlace==1.9.6', 'console_scripts', 'interlace')())
File "/usr/local/bin/interlace", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.10/dist-packages/Interlace-1.9.6-py3.10.egg/Interlace/interlace.py", line 6, in <module>
File "/usr/local/lib/python3.10/dist-packages/Interlace-1.9.6-py3.10.egg/Interlace/lib/core/output.py", line 4, in <module>
File "/usr/local/lib/python3.10/dist-packages/colorclass-2.2.0-py3.10.egg/colorclass/__init__.py", line 11, in <module>
File "/usr/local/lib/python3.10/dist-packages/colorclass-2.2.0-py3.10.egg/colorclass/codes.py", line 4, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)For the first issue, it seems that you need to create another ticket to linode support to increase the limit of instances/Linodes, it seems that it is limited to 9.
For the second one, this occurs because Interlace has some problem with python 3.10 which is installed in the docker container while trying to follow axiom install steps via docker. So the solution will be either changing python version inside the container to a version that works fine with Interlace or just adding the following line
from collections.abc import Mapping
in
/usr/lib/python3.10/collections/init.py
/usr/lib/python3.10/collections/__init__.py
0xtavian
commented
2 years ago @osamahamad just seeing this. Ty for troubleshooting. I’ll try to look into this sometime today, it seems like it could be related to this issue https://github.com/codingo/Interlace/issues/163 About the rate limits, aside from making a support request the increase in instances/linodes, you’ll need to spin up in batches of 10 ( to be safe ).
osamahamad
commented
2 years ago @0xtavian Hi @0xtavian , May I ask your advice on which provider I should use that does not require me to wait so much time to be able to spin up 100 instance easily?
The reason because when I contacted Linode about increasing the limit, they replayed the following :
What happened here is that you hit an account limit that we place on new accounts to help prevent fraudulent activity on our platform. We've reviewed your account and we are unable to increase your account limits at this time.
If you would like to request this again in the future, we ask that you wait until you have at least 3 months of positive billing history on this account.
If you have any additional questions or if you need anything else please let us know.
Regards,
Linode Support Team
and 3-months period is so much ... The account is created using referral. Any other provider I could easily spin up 100 instances with without having a similar condition ?
0xtavian
commented
2 years ago @osamahamad im not sure tbh. The providers have limits on new accounts to prevent fraud, this is pretty consistent with all providers.
Jesse415
commented
2 years ago For the first issue, it seems that you need to create another ticket to linode support to increase the limit of instances/Linodes, it seems that it is limited to 9.
For the second one, this occurs because Interlace has some problem with python 3.10 which is installed in the docker container while trying to follow axiom install steps via docker. So the solution will be either changing python version inside the container to a version that works fine with Interlace or just adding the following line
from collections.abc import Mappingin/usr/lib/python3.10/collections/init.py /usr/lib/python3.10/collections/__init__.py
How can I add this to AWS AMI so when I lunch axiom-fleet I don't have to ssh in individually can add these?
0xtavian
commented
2 years ago For the first issue, it seems that you need to create another ticket to linode support to increase the limit of instances/Linodes, it seems that it is limited to 9. For the second one, this occurs because Interlace has some problem with python 3.10 which is installed in the docker container while trying to follow axiom install steps via docker. So the solution will be either changing python version inside the container to a version that works fine with Interlace or just adding the following line
from collections.abc import Mappingin/usr/lib/python3.10/collections/init.py /usr/lib/python3.10/collections/__init__.pyHow can I add this to AWS AMI so when I lunch axiom-fleet I don't have to ssh in individually can add these?
@Jesse415 i have not gotten the chance to test AWS yet, but I see the issue you are facing. https://github.com/pry0cc/axiom/blob/bef03d0624f22402522f18e0626f24093057e77a/images/builders/aws.json#L20
For all other providers we use Ubuntu 20.04, you can see for AWS we are using ubuntu-jammy-22.04, which uses the latest python by default. If you switch this to use Ubuntu 20.04, you shouldn’t have this issue as it’s not using the latest python. I’ll try to get around to testing today but if you can a chance before me, lmk if using Ubuntu 20.04 fixes this ( for all other providers we are using Ubuntu 20.04 for instances so we don’t run into this issue ).
@pry0cc any reason for using 22.04 for AWS by chance?
pry0cc
commented
2 years ago About interlace, we should probably PR them and fix it for the latest python version. I also have it on my todo to use packer 1.8.0 in axiom-configure. Both of these are upstream issues that we can fix.
For AWS, I'm using 22.04 just because it was readily available. It seemed to build just fine and I don't see any issue with it unless you do?
0xtavian
commented
2 years ago Yep they are already aware of the issue and have identified the root cause https://github.com/codingo/Interlace/issues/163, once this is fixed we shouldn’t have this issue anymore. I do worry about drift between the Ubuntu versions, which could make it somewhat difficult to reuse the same provisioners. I think we should keep consistent and only use Ubuntu 20.04 for instances to avoid these types of issues.
I haven’t gotten a chance to pay with AWS but if I get a chance today, I’ll see if using 20.04 is a reliable option for AWS, unless of course there is some concern. Thanks all!
Jesse415
commented
2 years ago For the first issue, it seems that you need to create another ticket to linode support to increase the limit of instances/Linodes, it seems that it is limited to 9. For the second one, this occurs because Interlace has some problem with python 3.10 which is installed in the docker container while trying to follow axiom install steps via docker. So the solution will be either changing python version inside the container to a version that works fine with Interlace or just adding the following line
from collections.abc import Mappingin/usr/lib/python3.10/collections/init.py /usr/lib/python3.10/collections/__init__.pyHow can I add this to AWS AMI so when I lunch axiom-fleet I don't have to ssh in individually can add these?
@Jesse415 i have not gotten the chance to test AWS yet, but I see the issue you are facing.
For all other providers we use Ubuntu 20.04, you can see for AWS we are using
ubuntu-jammy-22.04, which uses the latest python by default. If you switch this to use Ubuntu 20.04, you shouldn’t have this issue as it’s not using the latest python. I’ll try to get around to testing today but if you can a chance before me, lmk if using Ubuntu 20.04 fixes this ( for all other providers we are using Ubuntu 20.04 for instances so we don’t run into this issue ).@pry0cc any reason for using 22.04 for AWS by chance?
I can confirm that changing line 20 to "name": "ubuntu/images/*ubuntu-focal-20.04-amd64-server-*" in .axiom/images/axiom.json and .axiom/imagines/builders/aws.json fixed the issues that I was having with python by using Ubuntu 20.04, Thank you
sanjay493
commented
2 years ago For the first issue, it seems that you need to create another ticket to linode support to increase the limit of instances/Linodes, it seems that it is limited to 9.
For the second one, this occurs because Interlace has some problem with python 3.10 which is installed in the docker container while trying to follow axiom install steps via docker. So the solution will be either changing python version inside the container to a version that works fine with Interlace or just adding the following line
from collections.abc import Mappingin/usr/lib/python3.10/collections/init.py /usr/lib/python3.10/collections/__init__.py
and issue with interlace resolved. Thanks
sanjay493
commented
2 years ago ┌──(kali㉿DESKTOP-6K4VUA2)-[/usr/lib/python3.10/collections] └─$ sudo nano init.py [sudo] password for kali:
add the code written below in init.py file which is /usr/lib/python3.10/collections folder. from collections.abc import Mapping
0xtavian
commented
2 years ago @sanjay493 @Jesse415 @osamahamad Thanks everyone. Interlace merged a PR fixing this issue a few days ago. Also, with the latest changes, Linode can spin up fleets of 100+ without timeouts https://github.com/pry0cc/axiom/issues/564#issuecomment-1242877719. Closing this issue. Please let me know if you disagree.
I ran the following
This is what I got:
I'm using Linode. I'm using axiom with docker latest version. Why this is happening, is there any solution for it?