search

mdesilva / AmazonFlexUnlimited

Automate the process of grabbing job blocks from Amazon Flex
173 stars 101 forks source link

Hosting on AWS for minimum latency #28

Closed figamore closed 2 years ago

figamore commented 2 years ago

Hello,

I know the readme says that the bot did not work on AWS, but that is not the case, as many other bots use AWS. I just tested the latency to flex-capacity-na from my AWS server and it is only 0.9ms, compared to an average of 33ms in my 1Gbps/1Gbps residential connection. I'm wondering if this will create an issue long term? Has anybody tried running the script on AWS for an extended time?

Xande1 commented 2 years ago

I've tried on a AWS server and a SUSE also owned by AWS. It's unable to pass authentication. That would certainty make a huge difference between actually having a chance of grabbing a block or just receive 410.

ghost commented 2 years ago

I tried it on aws a month ago. It worked fine, but i havent tried it now. As for the latency on my aws server i was getting around 0.2-0.3ms but still it was missing the blocks, thats why i dont use it anymore. Also did you use the ping command to the flex? Because ping only sends 64 bytes to the flex server, as for the post request its size is usually around 30 to 50 kilobytes. So, still you will not be able to achieve that latency.

ususus1987 commented 2 years ago

I just tried it on AWS EC2, running Ubuntu Server and I ran it and it was working great on California server. Then, I found that their server is located in Virginia, So I made another server in Virginia and I was able to 500 request in 62 sec. The latency was 0.5ms which is ideal, but guess what, I missed another offer and the error shown was 410. Can you believe that?

ghost commented 2 years ago

Use the request.elapsed on the request in the accept offer function. It will show you how long the post request took from Amazon server to your server. Then let me know, I used to get around 0.8 to 0.20s. I might help you with that

figamore commented 2 years ago

I just tried it on AWS EC2, running Ubuntu Server and I ran it and it was working great on California server.

Then, I found that their server is located in Virginia, So I made another server in Virginia and I was able to 500 request in 62 sec.

The latency was 0.5ms which is ideal, but guess what, I missed another offer and the error shown was 410.

Can you believe that?

You're going WAYY too fast and may get banned. Try putting a 3-5 second delay between each request.

ususus1987 commented 2 years ago

I just tried it on AWS EC2, running Ubuntu Server and I ran it and it was working great on California server. Then, I found that their server is located in Virginia, So I made another server in Virginia and I was able to 500 request in 62 sec. The latency was 0.5ms which is ideal, but guess what, I missed another offer and the error shown was 410. Can you believe that?

You're going WAYY too fast and may get banned. Try putting a 3-5 second delay between each request.

I was thing thinking about that, Thanks. Would you please tell me how to do it please?

ususus1987 commented 2 years ago

Use the request.elapsed on the request in the accept offer function. It will show you how long the post request took from Amazon server to your server. Then let me know, I used to get around 0.8 to 0.20s. I might help you with that

Thanks, Im not a Python Expert, but I understand the language. if you can just let me know how to do it so, I can get back to you with results

ghost commented 2 years ago

Replace this line return requests.post(FlexUnlimited.routes.get("GetOffers"), headers=self.__requestHeaders, json=requestBody)

with this line request = requests.post(FlexUnlimited.routes.get("GetOffers"), headers=self.__requestHeaders, json=requestBody) print(request.elapsed.total_seconds()) return request

And for the delay use the time.sleep(), right after self.__retryCount += 1 If you want 1 sec delay then time.sleep(1), and if 2 sec then time.sleep(2) and so on

ususus1987 commented 2 years ago

The second worked The first one did not work!!! in your first comment, you mentioned "Use the request.elapsed on the request in the accept offer function" the you sent me is in the GetOffer function.

ghost commented 2 years ago

my bad It is getOffer function.

Ahdrumond commented 1 year ago

I just tried it on AWS EC2, running Ubuntu Server and I ran it and it was working great on California server. Then, I found that their server is located in Virginia, So I made another server in Virginia and I was able to 500 request in 62 sec. The latency was 0.5ms which is ideal, but guess what, I missed another offer and the error shown was 410. Can you believe that?

Hi! Is there any chance you could tell me where to find a toturial on how to do this. I have the code running on my machine perfectly but I would like to see how it behaves on a virtual server.

thanks