search

locustio / locust

Write scalable load tests in plain Python 🚗💨
https://locust.cloud
MIT License
25.12k stars 3k forks source link

Decide if ipv6 can work #2923

Closed nc-marco closed 1 month ago

nc-marco commented 1 month ago

Summary of changes:

Previously IPV6 was enabled if the kernel supported the IPV6 stack.  This was an issue in the distributed scenario when there is a network component between the master and worker nodes that does not support IPV6.  This is the case for example in AWS EKS cluster where the service resource does not support IPV6 even if the pods in the cluster do.   To avoid attempting to communicate over IPv6 when it is not possible, some basic logic was added  which does the following:

  1. Determine as before if the kernel supports IPv6, but uses the socket method, has_dualstack_ipv6().  This does not have the limitations of has_ipv6() of the socket library (and I imagine the reason why it was avoided) since it actually does a syscall to the kernel which will fail if the kernel does not support IPv6.  We chose to switch to this method (from that of urllib3) since we needed other methods from the basic python socket library.
  2. Verify if the master hostname can be resolved as an IPv6 address.  This was deemed as a reasonable way to determine if the network environment supports IPv6 (outside of the OS where locust is running).  No admin would configure AAAA DNS records if the network isn't completely IPv6 ready.
  3. If IP addresses are used instead of host names (for example in the bind on the server side), then determine if the address is a valid IPv6 address or not. 

The same checks are done independently on the server and client during socket creation and will require no changes in the use of locust.  For example, currently the bind address of '*' will cause the master node to listen on both IPv6 and IPv4 addresses.  This behaves the same as before while setting master_bind_host to an IPv4 address, such as '0.0.0.0', will prevent zmq from enabling IPv6 on the master since it should only listen on an IPv4 interface.

cyberw commented 1 month ago

Fix looks reasonable but we'd need to investigate what's wrong with macos/windows.

nc-marco commented 1 month ago

That's very kind of you. I will attempt to debug it on my own a bit. I just haven't had the time. Hopefully by the end of the week. I have a more experienced programmer that might give me a hand too. I have unfortunately more of an infrastructure background.

On Wed, Oct 9, 2024 at 5:12 PM Lars Holmberg @.***> wrote:

Fix looks reasonable but we'd need to investigate what's wrong with macos/windows.

— Reply to this email directly, view it on GitHub https://github.com/locustio/locust/pull/2923#issuecomment-2402619049, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2B4QIXYDO4S4LGTEKGGEL3Z2VBV5AVCNFSM6AAAAABPKUAEAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBSGYYTSMBUHE . You are receiving this because you authored the thread.Message ID: @.***>

nc-marco commented 1 month ago

So, thanks to my friend's mac (I run linux of course), we were able to see what the issue was. It was raising and resolver error. I need to dig probably deeper to understand why it happens on macOS and windows and not Linux, but putting in a try/except block and now it passes on these OS's. I verified it still does what I want it to do in EKS, so I'm happy with it. I'll leave it up to you to decide if it is OK.

cyberw commented 1 month ago

I think it looks OK! Any specific reason you renamed the socked module?

import socket as csocket
nc-marco commented 1 month ago

No strong feelings about that. With the addition of gaierror, the list after import ... was getting rather long I felt. csocket, referring to the traditional c socket, I thought would prevent the confusion with zmq socket.

But as I said, I have no strong feelings. If you prefer it as it was before, or with a different alias, that's fine too.

On Fri, Oct 11, 2024 at 7:37 PM Lars Holmberg @.***> wrote:

I think it looks OK! Any specific reason you renamed the socked module?

import socket as csocket

— Reply to this email directly, view it on GitHub https://github.com/locustio/locust/pull/2923#issuecomment-2407859411, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2B4QIQVDTAKUSH2TAGQVJDZ3AEGJAVCNFSM6AAAAABPKUAEAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBXHA2TSNBRGE . You are receiving this because you authored the thread.Message ID: @.***>

cyberw commented 1 month ago

maybe from socket import gaierror, has_dualstack_ipv6?

I dont really have a strong opinion either, but renames always rub me the wrong way :)

nc-marco commented 1 month ago

Sure. Will do.

On Fri, Oct 11, 2024 at 7:56 PM Lars Holmberg @.***> wrote:

maybe from socket import gaierror, has_dualstack_ipv6?

— Reply to this email directly, view it on GitHub https://github.com/locustio/locust/pull/2923#issuecomment-2407886128, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2B4QIRUZ65KU5T2O22IDODZ3AGOXAVCNFSM6AAAAABPKUAEAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBXHA4DMMJSHA . You are receiving this because you authored the thread.Message ID: @.***>

nc-marco commented 1 month ago

Thx!

On Fri, Oct 11, 2024 at 11:24 PM Lars Holmberg @.***> wrote:

Merged #2923 https://github.com/locustio/locust/pull/2923 into master.

— Reply to this email directly, view it on GitHub https://github.com/locustio/locust/pull/2923#event-14614197067, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2B4QIWJCTEQQ2TKVX2TO4DZ3A6YJAVCNFSM6AAAAABPKUAEAGVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJUGYYTIMJZG4YDMNY . You are receiving this because you authored the thread.Message ID: @.***>