httpx module not getting downloaded

[jerinjose61]

Describe the bug The httpx module is not getting downloaded and installed. I get the below error:

fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "attributes": null,
            "copy": true,
            "creates": null,
            "decrypt": true,
            "dest": "/root/.bbot/tools",
            "exclude": [],
            "extra_opts": [],
            "group": null,
            "include": [
                "httpx"
            ],
            "io_buffer_size": 65536,
            "keep_newer": false,
            "list_files": false,
            "mode": null,
            "owner": null,
            "remote_src": true,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "src": "https://github.com/projectdiscovery/httpx/releases/download/v1.2.5/httpx_1.2.5_linux_arm64.zip",
            "unsafe_writes": false,
            "validate_certs": true
        }
    },
    "msg": "Failure downloading https://github.com/projectdiscovery/httpx/releases/download/v1.2.5/httpx_1.2.5_linux_arm64.zip, An unknown error occurred: HTTPSConnection.__init__() got an unexpected keyword argument 'cert_file'"
}

The last error message seems to point to an issue where ansible is not able to connect and download httpx. A google search lead me to the below issue in ansible:

https://github.com/void-linux/void-packages/issues/47483

The solution provided is to upgrade ansible version. I did the same and now it works. httpx gets downloaded and installed.

However, after upgrading ansible, I did get the below warning:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
bbot 1.1.6 requires ansible<8.0.0,>=7.3.0, but you have ansible 9.5.1 which is incompatible.

I am a bit worried about this error as I don't want things to break because of the version mismatch.

Is there another correct way of resolving this issue which may not lead to errors later on?

Expected behavior httpx module should get download and installed.

BBOT Command Example: bbot -m httpx -t evilcorp.com

OS, BBOT Installation Method + Version OS: Ubuntu Server 24.04 method: pip, BBOT version: 1.1.6

[TheTechromancer]

@jerinjose61 thanks for reporting this. What is your python version?

[jerinjose61]

I tested with both Python 3.12 and 3.10. In both cases I encountered the same issue.

[TheTechromancer]

Hmm okay, I have an ubuntu 22.04 box and I've tried with python 3.10 and 3.12 and I can't reproduce. I'd say there's probably an issue between bbot's pip packages and the ones you already have installed on your box.

Can you try installing with pipx? This will make sure bbot gets its own isolated python environment.

[jerinjose61]

@TheTechromancer as per your suggestion, i installed bbot using pipx and ran a scan. I am still facing the same issue. Wonder why. Anyway, I am going to try this out on a Ubuntu 22.04 server as well to see if that makes any difference.

[jerinjose61]

Apologies for the confusion. I faced the issue on Ubuntu Server 24.04 and not 22.04 as mentioned in my original post. Sorry again for the mistake. I will now try once on 22.04 and see if the issue persists.

[jerinjose61]

@TheTechromancer I tested this on Ubuntu Server 22.04 (Hetzner Cloud) with Python 3.10.12 and it works fine there. httpx gets downloaded and installed without any issue.

Will it be possible for you to check once on an Ubuntu 24.04 box and see if you can reproduce the bug?

[TheTechromancer]

Hmm I tried on 24.04 with pip and pipx and I couldn't reproduce.

[TheTechromancer]

Closing for inactivity. Please reopen if it's still an issue.