Linode: Request failed: 408

frost19k commented 2 years ago

This issue has been discussed previously in #411

Following the advice in that issue, I have confirmed with Linode that my account limit is 20 instances & I still get the error when trying to launch 10 instances - which is less than the max batch limit of 15.

Generally, 2 or 3 will fail to launch but in the worst case all but one fail to launch.

┌──(root💀b04ff43991ee)-[~/Tools/reconftw]
└─# axiom-fleet 'reconFTW' -i 10
Initializing new fleet 'reconFTW' with 10 instances...
INITIALIZING IN 5 SECONDS, CTRL+C to quit... 
Instances: [ reconFTW01 reconFTW02 reconFTW03 reconFTW04 reconFTW05 reconFTW06 reconFTW07 reconFTW08 reconFTW09 reconFTW10  ]
Request failed: 408leet reconFTW initialization...
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Request failed: 408
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Request failed: 408leet reconFTW initialization...
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Request failed: 408
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Request failed: 408leet reconFTW initialization...
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Request failed: 408
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Request failed: 408leet reconFTW initialization...
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Request failed: 408leet reconFTW initialization...
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Request failed: 408leet reconFTW initialization...
┌errors─┬──────────────────┐
│ field │ reason           │
├───────┼──────────────────┤
│       │ Please try again │
└───────┴──────────────────┘
Initialized instance 'reconFTW03' at ''!zation...
Initialized instance 'reconFTW02' at ''!
Initialized instance 'reconFTW04' at ''!zation...
Initialized instance 'reconFTW05' at ''!
Initialized instance 'reconFTW06' at ''!
Initialized instance 'reconFTW07' at ''!zation...
Initialized instance 'reconFTW08' at ''!zation...
Initialized instance 'reconFTW01' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW09' at ''!zation...
Initialized instance 'reconFTW10' at ''!zation...
Selected: [ reconFTW01 ]conFTW initialization...
Fleet started succesfully!
To delete your fleet, just run 'axiom-rm "reconFTW*" -f'

six2dez commented 2 years ago

Same here

frost19k commented 2 years ago

Same here

Damn, if that's where the contributors are at then I'm screwed! :-P

I've noticed I can reliably launch 5 at a time. Failures are rare in batches of 5 (maybe one in fifty) - but not absent.

six2dez commented 2 years ago

Seems that Linode limit has been decreased to 5, anyway I have opened a ticket asking for this. I will update when I have any answer.

0xtavian commented 2 years ago

@six2dez @frost19k This is a temp workaround. I’ll be merging something like this to master soonish.

#!/usr/bin/env bash

makefleet(){
AXIOM_PATH="$HOME/.axiom"
name=$1
total=$2
provider="$(cat "$AXIOM_PATH/axiom.json" | jq -r '.provider')"

if [[ "$provider" == "ibm" ]]; then
 limit=45

elif [[ "$provider" == "azure" ]]; then
 limit=10

elif [[ "$provider" == "do" ]]; then
 limit=10

elif [[ "$provider" == "linode" ]]; then
 limit=5
fi

while  [ $total != 0 ]; do
 if [ $limit -lt $total ]; then
  axiom-fleet "$name" -i $limit
  total=$((total-$limit))
 else
  axiom-fleet "$name" -i $total
  total=$((total-$total))
fi

done

}
makefleet $1 $2

frost19k commented 2 years ago

@0xtavian

So, I'd had a similar idea & I decided to run axiom-fleet in a for loop. The first two loops run properly. The third loop throws an error.

This only happens with the -r flag. Without it, the loop runs successfully. So this probably won't show in your script.

Note: Sometimes the error token is "09" - but I can't reproduce that.

┌──(root💀07f52806f9b8)-[~/Tools/reconftw]
└─# for i in $(seq 3); do axiom-fleet 'reconFTW' -i 4 -r ap-south,eu-central,us-east; done
Initializing new fleet 'reconFTW' with 4 instances...
Cycling through following regions:ap-south,eu-central,us-east...
INITIALIZING IN 5 SECONDS, CTRL+C to quit... 
Instances: [ reconFTW01 reconFTW02 reconFTW03 reconFTW04  ]
Initialized instance 'reconFTW03' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW02' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW01' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW04' at '<Redacted IP Addr>'!
Selected: [ reconFTW01 reconFTW02 reconFTW03 reconFTW04 ]
Fleet started succesfully!
To delete your fleet, just run 'axiom-rm "reconFTW*" -f'
Initializing new fleet 'reconFTW' with 4 instances...
Cycling through following regions:ap-south,eu-central,us-east...
INITIALIZING IN 5 SECONDS, CTRL+C to quit... 
Instances: [ reconFTW05 reconFTW06 reconFTW07 reconFTW08  ]
Initialized instance 'reconFTW05' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW06' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW08' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW07' at '<Redacted IP Addr>'!
Selected: [ reconFTW01 reconFTW02 reconFTW03 reconFTW04 reconFTW05 reconFTW06 reconFTW07 reconFTW08 ]
Fleet started succesfully!
To delete your fleet, just run 'axiom-rm "reconFTW*" -f'
Initializing new fleet 'reconFTW' with 4 instances...
Cycling through following regions:ap-south,eu-central,us-east...
INITIALIZING IN 5 SECONDS, CTRL+C to quit... 
/root/.axiom/interact/axiom-fleet: line 209: 08: value too great for base (error token is "08")
/root/.axiom/interact/axiom-fleet: line 211: [: -gt: unary operator expected
Selected: [ reconFTW01 reconFTW02 reconFTW03 reconFTW04 reconFTW05 reconFTW06 reconFTW07 reconFTW08 ]
Fleet started succesfully!
To delete your fleet, just run 'axiom-rm "reconFTW*" -f'

┌──(root💀07f52806f9b8)-[~/Tools/reconftw]
└─# axiom-fleet 'reconFTW' -i 4 -r ap-south,eu-central,us-east 
Initializing new fleet 'reconFTW' with 4 instances...
Cycling through following regions:ap-south,eu-central,us-east...
INITIALIZING IN 5 SECONDS, CTRL+C to quit... 
/root/.axiom/interact/axiom-fleet: line 209: 08: value too great for base (error token is "08")
/root/.axiom/interact/axiom-fleet: line 211: [: -gt: unary operator expected
Selected: [ reconFTW01 reconFTW02 reconFTW03 reconFTW04 reconFTW05 reconFTW06 reconFTW07 reconFTW08 ]
Fleet started succesfully!
To delete your fleet, just run 'axiom-rm "reconFTW*" -f'

0xtavian commented 2 years ago

@frost19k this is perfect. thanks you. I’m pretty sure I know what the issue is. I’ll investigate. Thanks again.

0xtavian commented 2 years ago

@frost19k do you mind running axiom-update ( e.g. cd ~/.axiom/ && git pull) and trying again? thanks again for bringing this up

frost19k commented 2 years ago

@0xtavian Ayyy!!! It does seem to be fixed :-D

This is great! Thank you so much.

┌──(root💀980d2e534a16)-[~/Tools/reconftw]
└─# date
Wed Jan  5 07:45:47 AM IST 2022

┌──(root💀980d2e534a16)-[~/Tools/reconftw]
└─# axiom-update
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 14 (delta 5), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (14/14), 20.32 KiB | 24.00 KiB/s, done.
From https://github.com/pry0cc/axiom
   64a9028..26ab8cb  master     -> origin/master
 + 2235bf2...54ab27e dev        -> origin/dev  (forced update)
Updating 64a9028..26ab8cb
Fast-forward
 interact/account-helpers/do.sh |   2 +-
 interact/axiom-account         |   2 +-
 interact/axiom-configure       | 118 ++++++++++++++++++++++-----------------------------
 interact/axiom-fleet           |  39 ++++++++++-------
 interact/axiom-scan            | 152 ++++++++++++++++++++++++++++++++++++++++++++++++++----------------
 5 files changed, 191 insertions(+), 122 deletions(-)

┌──(root💀980d2e534a16)-[~/Tools/reconftw]
└─# axiom-fleet 'reconFTW' -i 12 -r ap-south,us-east
Initializing new fleet 'reconFTW' with 12 instances...
Cycling through following regions:ap-south,us-east...
INITIALIZING IN 5 SECONDS, CTRL+C to quit... 
Instances: [ reconFTW01 reconFTW02 reconFTW03 reconFTW04 reconFTW05 reconFTW06 reconFTW07 reconFTW08 reconFTW09 reconFTW10 reconFTW11 reconFTW12  ]
Initialized instance 'reconFTW01' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW05' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW07' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW02' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW04' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW11' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW10' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW12' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW09' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW03' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW06' at '<Redacted IP Addr>'!
Initialized instance 'reconFTW08' at '<Redacted IP Addr>'!
Selected: [ reconFTW01 reconFTW02 reconFTW03 reconFTW04 reconFTW05 reconFTW06 reconFTW07 reconFTW08 reconFTW09 reconFTW10 reconFTW11 reconFTW12 ]
Fleet started succesfully!
To delete your fleet, just run 'axiom-rm "reconFTW*" -f'

pry0cc / axiom

Linode: Request failed: 408 #500