Add support for multiple services/ipv6ranges, static typing, some general improvements.

[WardPearce]

Improvements

• Support for multiple "services"/ipv6 ranges
• Refactoring the code to be more readable
• Static typing

How to test

1. git clone https://github.com/WardPearce/smart-ipv6-rotator
2. cd smart-ipv6-rotator
3. sudo python smart-ipv6-rotator.py run --ipv6range 2407:7000:9827:4100::/64 --services google

Usage

# Rotate IPV6 for google
sudo python smart-ipv6-rotator.py run --ipv6range 2407:7000:9827:4100::/64 --services google

# Or just
sudo python smart-ipv6-rotator.py run --ipv6range 2407:7000:9827:4100::/64

# Cleanup
sudo python smart-ipv6-rotator.py clean

# Clean one
sudo python smart-ipv6-rotator.py clean-one --services google

# Manual IPV6 assigning
sudo python smart-ipv6-rotator.py run --ipv6range 2407:7000:9827:4100::/64 --external-ipv6-ranges 2001:4860:4000::/36,2404:6800:4000::/36,2607:f8b0:4000::/36

# Maybe you want to use a service & manually assign, thats fine too!
sudo python smart-ipv6-rotator.py run --ipv6range 2407:7000:9827:4100::/64 --services google --external-ipv6-ranges 2001:4860:4000::/36,2404:6800:4000::/36,2607:f8b0:4000::/36

Design choices

• Use Expections instead sys.exit, more "pythonic"
• The use of ranges.py, so other developers can add other services in the future for easy use.

TODO

• [x] Update README
• [x] Update Dockerfile

[unixfox]

Very cool but I still want to keep the requirements.txt. The point is to be able to use the script by only relying on packages from the linux distribution.

Poetry install is outside of most Linux distribution packages.

Apart from that very cool to be able to specify new ipv6 ranges. Maybe something like being able to tweak the testing url would be cool too (ipv6.icanhazip.com).

Also "services" should be for google by default because this script was made for escaping youtube blockage.

[WardPearce]

Very cool but I still want to keep the requirements.txt. The point is to be able to use the script by only relying on packages from the linux distribution.

Poetry install is outside of most Linux distribution packages.

Apart from that very cool to be able to specify new ipv6 ranges. Maybe something like being able to tweak the testing url would be cool too (ipv6.icanhazip.com).

Also "services" should be for google by default because this script was made for escaping youtube blockage.

yup good points!

[WardPearce]

Very cool but I still want to keep the requirements.txt. The point is to be able to use the script by only relying on packages from the linux distribution.

Poetry install is outside of most Linux distribution packages.

Apart from that very cool to be able to specify new ipv6 ranges. Maybe something like being able to tweak the testing url would be cool too (ipv6.icanhazip.com).

Also "services" should be for google by default because this script was made for escaping youtube blockage.

Slight issue with having a default service, is that you'd have to explicitly disable the service if you are using manual ipv6 ranges in the cli. I think it would be easier that the documentation for https://docs.invidious.io/ipv6-rotator would just have the --service google flag

[unixfox]

Slight issue with having a default service, is that you'd have to explicitly disable the service if you are using manual ipv6 ranges in the cli.

It's not possible to disable all services when manual ipv6 ranges are specified?

[WardPearce]

Slight issue with having a default service, is that you'd have to explicitly disable the service if you are using manual ipv6 ranges in the cli.

It's not possible to disable all services when manual ipv6 ranges are specified?

Yes, but that then means people can add a service + other custom ipv6 ranges for whatever reason

[unixfox]

Slight issue with having a default service, is that you'd have to explicitly disable the service if you are using manual ipv6 ranges in the cli.

It's not possible to disable all services when manual ipv6 ranges are specified?

Yes, but that then means people can add a service + other custom ipv6 ranges for whatever reason

I think it's a valid case. Maybe one want to rotate for google and other service too.

What about having the ability to disable service by passing a blank value to service like --service "" or a parameter called --no-service?

[WardPearce]

Slight issue with having a default service, is that you'd have to explicitly disable the service if you are using manual ipv6 ranges in the cli.

It's not possible to disable all services when manual ipv6 ranges are specified?

Yes, but that then means people can add a service + other custom ipv6 ranges for whatever reason

I think it's a valid case. Maybe one want to rotate for google and other service too.

What about having the ability to disable service by passing a blank value to service like --service "" or a parameter called --no-service?

I'm fine with adding support for blank services, I do think it makes it overly complex & easy for someone to mess up tho. Personally I think just including the --service google flag by default in docs would solve the issue.

[WardPearce]

maybe you should be able to pass multiple services? e.g. --services google,twitter for example?

[unixfox]

I'm sorry, I'm against adding new dependencies that you can't find in the popular Linux distributions. I want this script to be as easy as possible to use and with very little dependencies.

---

I'm fine with adding support for blank services, I do think it makes it overly complex & easy for someone to mess up tho.

Then let's have a parameter called --no-service, this way it disables all the default services.

maybe you should be able to pass multiple services? e.g. --services google,twitter for example?

If we were to do that, then google will remain the only one activated by default. But being able to pass multiple services is a good idea.

[WardPearce]

I'm sorry, I'm against adding new dependencies that you can't find in the popular Linux distributions. I want this script to be as easy as possible to use and with very little dependencies.

I'm fine with adding support for blank services, I do think it makes it overly complex & easy for someone to mess up tho.

Then let's have a parameter called --no-service, this way it disables all the default services.

maybe you should be able to pass multiple services? e.g. --services google,twitter for example?

If we were to do that, then google will remain the only one activated by default. But being able to pass multiple services is a good idea.

sounds good

[TheFrenchGhosty]

So, here's my opinion (follow up to the discussion we had on Matrix):

• Poetry shouldn't be implemented, let's not depend on something non standard
• Instead of Poetry, proper support for pipx install git+<URL> should be implemented

About the new dependencies, this is basically a non-issue (other than for the developers of it, so @unixfox will decide if they're used) since pipx trivialize the dependencies installation

[WardPearce]

So, here's my opinion (follow up to the discussion we had on Matrix):

* Poetry shouldn't be implemented, let's not depend on something non standard

* Instead of Poetry, proper support for `pipx install git+<URL>` should be implemented

About the new dependencies, this is basically a non-issue (other than for the developers of it, so @unixfox will decide if they're used) since pipx trivialize the dependencies installation

Yup, am working on removing poetry as a dependency

[unixfox]

Summarying what I said on matrix.

• I want to have as little dependencies as possible, I'm ok with adding like 1-2 more but these should only be installable from the OS package manager.
• Poetry is fine to me as long as it's not the default way to install the script. In fact, any other advertised ways to install the script is fine as long as the default install method is installing packages from the OS then that's it.
• New features are fine but let's not make this script too complicated. Let's keep in mind the initial goal, rotate the ipv6 address for google ipv6 subnet ranges.

[WardPearce]

Summarying what I said on matrix.

* I want to have as little dependencies as possible, I'm ok with adding like 1-2 more but these should only be installable from the OS package manager.

* Poetry is fine to me as long as it's not the default way to install the script. In fact, any other advertised ways to install the script is fine as long as the default install method is installing packages from the OS then that's it.

* New features are fine but let's not make this script too complicated. Let's keep in mind the initial goal, rotate the ipv6 address for google ipv6 subnet ranges.

Personally I think it makes sense to make use of the available (well tested & documented) third-party modules available via the pypi repo instead of having to re-implement something like tinydb. And as @TheFrenchGhosty said pipx trivialize the dependencies installation.

I think also allowing different "services"/ipv6 ranges should be the only new future added, as it seems genuinely useful for other FOSS projects & doesn't add too much overhead (personally).

If these changes are out of scope for this project, I'm happy to maintain my own fork for people who wish to use it.

[unixfox]

I think also allowing different "services"/ipv6 ranges should be the only new future added, as it seems genuinely useful for other FOSS projects & doesn't add too much overhead (personally).

Sure ok! I'm ok with that.

Personally I think it makes sense to make use of the available (well tested & documented) third-party modules available via the pypi repo instead of having to re-implement something like tinydb. And as @TheFrenchGhosty said pipx trivialize the dependencies installation.

Well we do not need a library database, the existing way to read from a file is fine. If you really want to migrate to a JSON file then I'm ok with it but let's use system libraries like "json" (https://www.geeksforgeeks.org/read-json-file-using-python/).

About the lib "click", while I understand that argparse is a bit of a pain to work with. You can still reach your goals with it.

[WardPearce]

I think also allowing different "services"/ipv6 ranges should be the only new future added, as it seems genuinely useful for other FOSS projects & doesn't add too much overhead (personally).

Sure ok! I'm ok with that.

Personally I think it makes sense to make use of the available (well tested & documented) third-party modules available via the pypi repo instead of having to re-implement something like tinydb. And as @TheFrenchGhosty said pipx trivialize the dependencies installation.

Well we do not need a library database, the existing way to read from a file is fine. If you really want to migrate to a JSON file then I'm ok with it but let's use system libraries like "json" (https://www.geeksforgeeks.org/read-json-file-using-python/).

About the lib "click", while I understand that argparse is a bit of a pain to work with. You can still reach your goals with it.

Yea I'm happy to port back to argparse & can write a small implementation of a json db myself.

[WardPearce]

Have remove tinydb & click :woozy_face:

[WardPearce]

Is now backwards compatible with previous implementation :confetti_ball:

[WardPearce]

A bit of a pointless question, what "version" should be consider this?

[unixfox]

A bit of a pointless question, what "version" should be consider this?

1.0.0 is good for me. it means the script works fine for mostly everyone

[unixfox]

Maybe a unit test would have picked the issue, but I try to make it work at minimum python 3.11. That's the latest version of python3 for debian 12: https://packages.debian.org/bookworm/python3

root@rotatetest:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py 
Traceback (most recent call last):
  File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 4, in <module>
    main()
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 245, in main
    args.func(args)
    ^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'func'
root@rotatetest:~/smart-ipv6-rotator# python3 --version
Python 3.11.2

It seems like you are using a feature of argparse that does not exist in python 3.11.

EDIT: Ha no I think it's the error that you are getting when no arg is passed to the script like:

python3 smart-ipv6-rotator.py 

Same for python3 smart-ipv6-rotator.py run

[WardPearce]

Maybe a unit test would have picked the issue, but I try to make it work at minimum python 3.11. That's the latest version of python3 for debian 12: https://packages.debian.org/bookworm/python3

root@rotatetest:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py 
Traceback (most recent call last):
  File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 4, in <module>
    main()
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 245, in main
    args.func(args)
    ^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'func'
root@rotatetest:~/smart-ipv6-rotator# python3 --version
Python 3.11.2

It seems like you are using a feature of argparse that does not exist in python 3.11.

Odd, I'll check it out in the morning. Was probably because im on 3.12. What minimum python version should we support? 3.10 onwards?

[WardPearce]

Maybe a unit test would have picked the issue, but I try to make it work at minimum python 3.11. That's the latest version of python3 for debian 12: https://packages.debian.org/bookworm/python3

root@rotatetest:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py 
Traceback (most recent call last):
  File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 4, in <module>
    main()
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 245, in main
    args.func(args)
    ^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'func'
root@rotatetest:~/smart-ipv6-rotator# python3 --version
Python 3.11.2

It seems like you are using a feature of argparse that does not exist in python 3.11.

EDIT: Ha no I think it's the error that you are getting when no arg is passed to the script like:

python3 smart-ipv6-rotator.py 

Same for python3 smart-ipv6-rotator.py run

aah okay, will fix that in the morning

[WardPearce]

Is this TODO section up to date? If so, I'm happy to implement them myself.

https://github.com/iv-org/smart-ipv6-rotator?tab=readme-ov-file#todo-priority

[unixfox]

Is this TODO section up to date? If so, I'm happy to implement them myself.

iv-org/smart-ipv6-rotator#todo-priority

Yes it is, feel free to do so! Maybe in another PR because this one is already big.

[WardPearce]

Is this TODO section up to date? If so, I'm happy to implement them myself. iv-org/smart-ipv6-rotator#todo-priority

Yes it is, feel free to do so! Maybe in another PR because this one is already big.

Yea probably good to get this merged 1st

[unixfox]

I just tested a couple of options.

If you configure a custom external ipv6 range then try to cleanup, it say that there is no cleanup.
Same if you try to configure an external custom ipv6 range then try to configure wihout an external custom ipv6 range

root@rotatetest:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py run --ipv6range 2a03:b0c0:3:d0::1999:c000/124 --external-ipv6-ranges 2606:4700::/64
[INFO] You have IPv6 connectivity. Continuing.
[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
[DEBUG] Debug info:
random_ipv6_address --> 2a03:b0c0:3:d0::1999:c005
random_ipv6_address_mask --> 124
gateway --> 2a03:b0c0:3:d0::1
interface_index --> 2
interface_name --> eth0
ipv6_subnet --> 2a03:b0c0:3:d0::1999:c000/124
[INFO] Correctly using the new random IPv6 address, continuing.
[INFO] Correctly configured the IPv6 routes for IPv6 ranges ['2001:4860:4000::/36', '2a00:1450:4000::/36', '2c0f:fb50:4000::/36', '2800:3f0:4000::/36', '2404:6800:4000::/36', '2606:4700::/64', '2607:f8b0:4000::/36'].
[INFO] Successful setup. Waiting for the propagation in the Linux kernel.
root@rotatetest:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py clean
[INFO] No cleanup of previous setup needed.
root@rotatetest:~/smart-ipv6-rotator# ls -l /tmp
total 16
-rw-r--r-- 1 root root  382 Apr 21 18:16 smart-ipv6-rotator.json

[WardPearce]

I just tested a couple of options.

If you configure a custom external ipv6 range then try to cleanup, it say that there is no cleanup. Same if you try to configure an external custom ipv6 range then try to configure wihout an external custom ipv6 range

root@rotatetest:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py run --ipv6range 2a03:b0c0:3:d0::1999:c000/124 --external-ipv6-ranges 2606:4700::/64
[INFO] You have IPv6 connectivity. Continuing.
[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
[DEBUG] Debug info:
random_ipv6_address --> 2a03:b0c0:3:d0::1999:c005
random_ipv6_address_mask --> 124
gateway --> 2a03:b0c0:3:d0::1
interface_index --> 2
interface_name --> eth0
ipv6_subnet --> 2a03:b0c0:3:d0::1999:c000/124
[INFO] Correctly using the new random IPv6 address, continuing.
[INFO] Correctly configured the IPv6 routes for IPv6 ranges ['2001:4860:4000::/36', '2a00:1450:4000::/36', '2c0f:fb50:4000::/36', '2800:3f0:4000::/36', '2404:6800:4000::/36', '2606:4700::/64', '2607:f8b0:4000::/36'].
[INFO] Successful setup. Waiting for the propagation in the Linux kernel.
root@rotatetest:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py clean
[INFO] No cleanup of previous setup needed.
root@rotatetest:~/smart-ipv6-rotator# ls -l /tmp
total 16
-rw-r--r-- 1 root root  382 Apr 21 18:16 smart-ipv6-rotator.json

This is expected behaviour I believe as clean will use google as default. So in your case it should be python3 smart-ipv6-rotator.py clean --external-ipv6-ranges 2606:4700::/64

[unixfox]

Mmh I would have tought that the expected behavior of clean is to remove all modifications done by the script. This is to "reset" the state before running the script.

That's how I imaginated the option to work.

[WardPearce]

Mmh I would have tought that the expected behavior of clean is to remove all modifications done by the script. This is to "reset" the state before running the script.

That's how I imaginated the option to work.

Yea maybe I should move the current clean command to clean-one & make clean reset all modifications

[WardPearce]

Have fixed all reported bugs & feature changes. Should be ready for merging now or additional feedback.

[WardPearce]

Added a section in the README around upgrading (https://github.com/WardPearce/smart-ipv6-rotator/tree/master?tab=readme-ov-file#upgrading). I think it should be considered best practice to clean before upgrading always to avoid issues already around how interfaces & ipv6 addresses are stored and/or formatted.

Also considering on changing the json file location to ${HOME}/.config/smart-ipv6-rotator to avoid any issues with anything automatically clearing temp files after X amount of time. Let me know if you think this is a good idea & what the appropriate path should be.

[unixfox]

Sorry for the very long wait! I forgot about this PR... You should remind me on matrix if there is no response for too long :).

There are a couple of things left to tackle:

• If we switch to JSON then we should have a safeguard if the script detects an old database format then stop, do nothing and print an error stating to run "clean" in the old script. And also redirect to the documentation.
• If I run the script then do clean I get a strange error that was not on the existing version:

  [Error] Failed to remove the test IPv6 subnet.
          May be expected if the route were not yet configured and that was a cleanup due to an error.

  Either you keep the test ipv6 subnet range or you skip the clean on manual cleaning.

• If the IPv6 range didn't work there is too many error message:

  
  root@ipv6-rot-invidious:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py run --ipv6range 2a03:b0c0:2:d0::1432:400/124 --external-ipv6-ranges 2606:4700::/64
  [INFO] You have IPv6 connectivity. Continuing.
  [Error] Failed to remove the test IPv6 subnet.
          May be expected if the route were not yet configured and that was a cleanup due to an error.

[INFO] Finished cleaning up previous setup. [INFO] Waiting for the propagation in the Linux kernel. [DEBUG] Debug info: ranges --> ['2606:4700::/64', '2404:6800:4000::/36', '2607:f8b0:4000::/36', '2800:3f0:4000::/36', '2001:4860:4000::/36', '2a00:1450:4000::/36', '2c0f:fb50:4000::/36'] random_ipv6_address --> 2a03:b0c0:2:d0::1432:40a gateway --> 2a03:b0c0:2:d0::1 interface_index --> 2 interface_name --> eth0 ipv6_subnet --> 2a03:b0c0:2:d0::1432:400/124 random_ipv6_address_mask --> 124 [Error] Failed to remove the configured IPv6 subnets 2606:4700::/64,2404:6800:4000::/36,2607:f8b0:4000::/36,2800:3f0:4000::/36,2001:4860:4000::/36,2a00:1450:4000::/36,2c0f:fb50:4000::/36 May be expected if the route were not yet configured and that was a cleanup due to an error

[INFO] Finished cleaning up previous setup. [INFO] Waiting for the propagation in the Linux kernel. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 174, in _new_conn conn = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection raise err File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection sock.connect(sa) TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 704, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 399, in _make_request conn.request(method, url, **httplib_request_kw) File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 239, in request super(HTTPConnection, self).request(method, url, body=body, headers=headers) File "/usr/lib/python3.11/http/client.py", line 1282, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.11/http/client.py", line 1328, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.11/http/client.py", line 1037, in _send_output self.send(msg) File "/usr/lib/python3.11/http/client.py", line 975, in send self.connect() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 205, in connect conn = self._new_conn() ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 179, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f3717cad490>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 489, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 788, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='2606:4700::6812:7261', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f3717cad490>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/root/smart-ipv6-rotator/smart_ipv6_rotator/init.py", line 152, in run check_new_ipv6_address = requests.get( ^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/api.py", line 73, in get return request("get", url, params=params, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/sessions.py", line 587, in request resp = self.send(prep, send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/sessions.py", line 701, in send r = adapter.send(request, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/requests/adapters.py", line 553, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='2606:4700::6812:7261', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f3717cad490>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 4, in main() File "/root/smart-ipv6-rotator/smart_ipv6_rotator/init.py", line 268, in main args.func(args) File "/root/smart-ipv6-rotator/smart_ipv6_rotator/init.py", line 63, in _parse_args return func(**params) ^^^^^^^^^^^^^^ File "/root/smart-ipv6-rotator/smart_ipv6_rotator/init.py", line 159, in run raise Exception( Exception: [ERROR] Failed to send the request for checking the new IPv6 address! The setup did not work! Your provider probably does not allow setting any arbitrary IPv6 address. Or did you correctly configure the IPv6 subnet to use? Exception: HTTPConnectionPool(host='2606:4700::6812:7261', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f3717cad490>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)'))

   The user won't understand what to do. Could you please only show the error message `[ERROR] Failed to send the request for checking the new IPv6 address! The setup did not work!` (with the rest of the text)?
   And maybe keep the most useful error message `TimeoutError: timed out`? This way you also get useful info for example when you enter the wrong `--external-ipv6-ranges`:

root@ipv6-rot-invidious:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py run --ipv6range 2a03:b0c0:2:d0::1432:4000/124 --external-ipv6-ranges 2606:4700:/64 [INFO] You have IPv6 connectivity. Continuing. [INFO] No cleanup of previous setup needed. [DEBUG] Debug info: ranges --> ['2800:3f0:4000::/36', '2404:6800:4000::/36', '2c0f:fb50:4000::/36', '2a00:1450:4000::/36', '2606:4700:/64', '2001:4860:4000::/36', '2607:f8b0:4000::/36'] random_ipv6_address --> 2a03:b0c0:2:d0::1432:400e gateway --> 2a03:b0c0:2:d0::1 interface_index --> 2 interface_name --> eth0 ipv6_subnet --> 2a03:b0c0:2:d0::1432:4000/124 random_ipv6_address_mask --> 124 [INFO] Correctly using the new random IPv6 address, continuing. [Error] Failed to remove the test IPv6 subnet. May be expected if the route were not yet configured and that was a cleanup due to an error.

[Error] Failed to remove the configured IPv6 subnets 2800:3f0:4000::/36,2404:6800:4000::/36,2c0f:fb50:4000::/36,2a00:1450:4000::/36,2606:4700:/64,2001:4860:4000::/36,2607:f8b0:4000::/36 May be expected if the route were not yet configured and that was a cleanup due to an error

[INFO] Finished cleaning up previous setup. [INFO] Waiting for the propagation in the Linux kernel. Traceback (most recent call last): File "/root/smart-ipv6-rotator/smart_ipv6_rotator/init.py", line 189, in run IPROUTE.route( File "/usr/lib/python3/dist-packages/pyroute2/iproute/linux.py", line 2172, in route .apply_filter(RouteFieldFilter()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pyroute2/requests/main.py", line 48, in apply_filter self.update(self) File "/usr/lib/python3/dist-packages/pyroute2/requests/main.py", line 40, in update self[key] = value

  File "/usr/lib/python3/dist-packages/pyroute2/requests/main.py", line 23, in __setitem__
    for nkey, nvalue in self.filter(key, value).items():
                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/requests/main.py", line 34, in filter
    return getattr(
           ^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/requests/common.py", line 115, in set_dst
    return self.parse_target('dst', context, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/requests/common.py", line 100, in parse_target
    ret[key] = ipaddress.ip_address(value).compressed
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ipaddress.py", line 54, in ip_address
    raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
ValueError: '2606:4700:' does not appear to be an IPv4 or IPv6 address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 4, in <module>
    main()
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 268, in main
    args.func(args)
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 63, in _parse_args
    return func(**params)
           ^^^^^^^^^^^^^^
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 199, in run
    raise Exception(
Exception: [Error] Failed to configure the test IPv6 route. The setup did not work!
        Exception:
'2606:4700:' does not appear to be an IPv4 or IPv6 address
```

[WardPearce]

Sorry for the very long wait! I forgot about this PR... You should remind me on matrix if there is no response for too long :).

There are a couple of things left to tackle:

* If we switch to JSON then we should have a safeguard if the script detects an old database format then stop, do nothing and print an error stating to run "clean" in the old script.
  And also redirect to the documentation.

* If I run the script then do `clean` I get a strange error that was not on the existing version:
  ```
  [Error] Failed to remove the test IPv6 subnet.
           May be expected if the route were not yet configured and that was a cleanup due to an error.
  ```

  Either you keep the test ipv6 subnet range or you skip the clean on manual cleaning.

* If the IPv6 range didn't work there is too many error message:

  root@ipv6-rot-invidious:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py run --ipv6range 2a03:b0c0:2:d0::1432:400/124 --external-ipv6-ranges 2606:4700::/64
[INFO] You have IPv6 connectivity. Continuing.
[Error] Failed to remove the test IPv6 subnet.
            May be expected if the route were not yet configured and that was a cleanup due to an error.

[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
[DEBUG] Debug info:
ranges --> ['2606:4700::/64', '2404:6800:4000::/36', '2607:f8b0:4000::/36', '2800:3f0:4000::/36', '2001:4860:4000::/36', '2a00:1450:4000::/36', '2c0f:fb50:4000::/36']
random_ipv6_address --> 2a03:b0c0:2:d0::1432:40a
gateway --> 2a03:b0c0:2:d0::1
interface_index --> 2
interface_name --> eth0
ipv6_subnet --> 2a03:b0c0:2:d0::1432:400/124
random_ipv6_address_mask --> 124
[Error]  Failed to remove the configured IPv6 subnets 2606:4700::/64,2404:6800:4000::/36,2607:f8b0:4000::/36,2800:3f0:4000::/36,2001:4860:4000::/36,2a00:1450:4000::/36,2c0f:fb50:4000::/36
            May be expected if the route were not yet configured and that was a cleanup due to an error

[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 704, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 399, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.11/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/lib/python3.11/http/client.py", line 975, in send
    self.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 179, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f3717cad490>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 788, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='2606:4700::6812:7261', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f3717cad490>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 152, in run
    check_new_ipv6_address = requests.get(
                             ^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 553, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='2606:4700::6812:7261', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f3717cad490>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 4, in <module>
    main()
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 268, in main
    args.func(args)
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 63, in _parse_args
    return func(**params)
           ^^^^^^^^^^^^^^
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 159, in run
    raise Exception(
Exception: [ERROR] Failed to send the request for checking the new IPv6 address! The setup did not work!
        Your provider probably does not allow setting any arbitrary IPv6 address.
        Or did you correctly configure the IPv6 subnet to use?
       Exception:
HTTPConnectionPool(host='2606:4700::6812:7261', port=80): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f3717cad490>, 'Connection to 2606:4700::6812:7261 timed out. (connect timeout=5)'))

The user won't understand what to do. Could you please only show the error message [ERROR] Failed to send the request for checking the new IPv6 address! The setup did not work! (with the rest of the text)? And maybe keep the most useful error message TimeoutError: timed out? This way you also get useful info for example when you enter the wrong --external-ipv6-ranges:

root@ipv6-rot-invidious:~/smart-ipv6-rotator# python3 smart-ipv6-rotator.py run --ipv6range 2a03:b0c0:2:d0::1432:4000/124 --external-ipv6-ranges 2606:4700:/64
[INFO] You have IPv6 connectivity. Continuing.
[INFO] No cleanup of previous setup needed.
[DEBUG] Debug info:
ranges --> ['2800:3f0:4000::/36', '2404:6800:4000::/36', '2c0f:fb50:4000::/36', '2a00:1450:4000::/36', '2606:4700:/64', '2001:4860:4000::/36', '2607:f8b0:4000::/36']
random_ipv6_address --> 2a03:b0c0:2:d0::1432:400e
gateway --> 2a03:b0c0:2:d0::1
interface_index --> 2
interface_name --> eth0
ipv6_subnet --> 2a03:b0c0:2:d0::1432:4000/124
random_ipv6_address_mask --> 124
[INFO] Correctly using the new random IPv6 address, continuing.
[Error] Failed to remove the test IPv6 subnet.
            May be expected if the route were not yet configured and that was a cleanup due to an error.

[Error]  Failed to remove the configured IPv6 subnets 2800:3f0:4000::/36,2404:6800:4000::/36,2c0f:fb50:4000::/36,2a00:1450:4000::/36,2606:4700:/64,2001:4860:4000::/36,2607:f8b0:4000::/36
            May be expected if the route were not yet configured and that was a cleanup due to an error

[INFO] Finished cleaning up previous setup.
[INFO] Waiting for the propagation in the Linux kernel.
Traceback (most recent call last):
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 189, in run
    IPROUTE.route(
  File "/usr/lib/python3/dist-packages/pyroute2/iproute/linux.py", line 2172, in route
    .apply_filter(RouteFieldFilter())
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/requests/main.py", line 48, in apply_filter
    self.update(self)
  File "/usr/lib/python3/dist-packages/pyroute2/requests/main.py", line 40, in update
    self[key] = value
    ~~~~^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/requests/main.py", line 23, in __setitem__
    for nkey, nvalue in self.filter(key, value).items():
                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/requests/main.py", line 34, in filter
    return getattr(
           ^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/requests/common.py", line 115, in set_dst
    return self.parse_target('dst', context, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/requests/common.py", line 100, in parse_target
    ret[key] = ipaddress.ip_address(value).compressed
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ipaddress.py", line 54, in ip_address
    raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
ValueError: '2606:4700:' does not appear to be an IPv4 or IPv6 address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/smart-ipv6-rotator/smart-ipv6-rotator.py", line 4, in <module>
    main()
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 268, in main
    args.func(args)
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 63, in _parse_args
    return func(**params)
           ^^^^^^^^^^^^^^
  File "/root/smart-ipv6-rotator/smart_ipv6_rotator/__init__.py", line 199, in run
    raise Exception(
Exception: [Error] Failed to configure the test IPv6 route. The setup did not work!
        Exception:
'2606:4700:' does not appear to be an IPv4 or IPv6 address

Sounds good 👍 Yea I wasn't in a rush with this PR, so was happy to not give you more stuff to do.

[WardPearce]

@unixfox have added what was requested.

[unixfox]

Thank you for the continuous efforts. I have thoroughly tested the latest changes and it's great!

I'll announce the change on the invidious matrix room.

[WardPearce]

Thank you for the continuous efforts. I have thoroughly tested the latest changes and it's great!

I'll announce the change on the invidious matrix room.

Big thanks to the whole Invidious team for working with me and making great suggestions!

[WardPearce]

@unixfox one small change needed, is the github description needs to be updated. Should probably just be "IPv6 rotator for specific subnets - unblock restrictions on IPv6 enabled websites"