search

wallix / awless

A Mighty CLI for AWS
http://awless.io/
Apache License 2.0
4.97k stars 263 forks source link

ec2-price.com down breaks pricer #216

Open boldandbusted opened 6 years ago

boldandbusted commented 6 years ago 
 awless inspect -i pricer
[info]    Running full sync before inspection (disable it with --local flag)

Fetching prices at http://ec2-price.com for region global

fetching price for 'm1.xlarge': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>5
02 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 't2.micro': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>50
2 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 'm4.large': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>50
2 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 't2.medium': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>5
02 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 't2.small': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>50
2 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 't2.large': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>50
2 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 'm4.2xlarge': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 'm4.xlarge': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>5
02 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
Instance        Count   Estimated total/day (no EBS)
--------        -----   ----------------------------
m4.2xlarge      2
m4.large        17
t2.micro        15
t2.medium       45
m4.xlarge       4
t2.small        8
t2.large        2
m1.xlarge       1
                        $0.00

Dunno if this is maintained by the project - apologies if it is not and there's nothing that can be done. Thanks for awless! :)

simcap commented 6 years ago

Hi @boldandbusted , thanks for reporting.

Unfortunately the API we use in the inspector (http://ec2-price.com/) is out of service now. All the code for the pricer is basically located here

Hopefully we could switch to a better service, but we have not find one yet. Other solutions are feasible (ex: caching prices locally and using AWS API to get prices, etc.) also but we are not focusing on that at the moment or in the near future.

Although this could be a good contribution for someone who has the time since inspectors are quite independent.

Thanks for your support!

adzuci commented 6 years ago

I would also love to have this feature back, but am doubtful that I could replace the existing dependency on ec2-price.

I use https://ec2instances.info for quick lookups, but it looks like all the pricing info that is needed should be in https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/index.json, however that's a 50mb file and somewhat awkward to parse through.

Theres a few places that talk about parsing that file in python though, not sure if it would be much harder in go: https://forrestbrazeal.com/2016/09/25/adventures-in-aws-understanding-the-price-list-api

fxaguessy commented 6 years ago

Hi, We knew that pricing information are available in JSON on AWS servers, but the problem is that we don't want to download and parse 50Mb JSON each time we launch this inspector. That's why we rely(ed) on ec2-price to get the data. The solution would have been to expose a service fetching these data from AWS and exposing them with a more friendly API. But we don't have the time/will to do that. If someone expose such an API, we would be happy to migrate from ec2-price to such an API. Another (less satisfactory) solution would be to download in the cache folder of awless ~/.awless/cache these big JSON files and update them periodically and use them when launching the inspector.

ps: Parsing JSON files in go would not be a problem as it is one basis of the standard library.

Niemi commented 5 years ago

Oh, that is newer. Can we user some filters in prices query?

aleksei@myhost$ `awless -p PROD inspect -i pricer --local Fetching prices at http://ec2-price.com for region global

fetching price for 'c4.large': strconv.ParseFloat: parsing "\r\n502 Bad Gateway\r\n<body bgcolor=\"white\">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n\r\n": invalid syntax fetching price for 't2.small': strconv.ParseFloat: parsing "\r\n502 Bad Gateway\r\n<body bgcolor=\"white\">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n\r\n": invalid syntax fetching price for 'c4.xlarge': strconv.ParseFloat: parsing "\r\n502 Bad Gateway\r\n<body bgcolor=\"white\">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n\r\n": invalid syntax fetching price for 't2.micro': strconv.ParseFloat: parsing "\r\n502 Bad Gateway\r\n<body bgcolor=\"white\">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n\r\n": invalid syntax fetching price for 'c5.large': strconv.ParseFloat: parsing "\r\n502 Bad Gateway\r\n<body bgcolor=\"white\">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n\r\n": invalid syntax fetching price for 'm3.medium': strconv.ParseFloat: parsing "\r\n502 Bad Gateway\r\n<body bgcolor=\"white\">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n\r\n": invalid syntax fetching price for 'c5.xlarge': strconv.ParseFloat: parsing "\r\n502 Bad Gateway\r\n<body bgcolor=\"white\">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n\r\n": invalid syntax fetching price for 't2.medium': strconv.ParseFloat: parsing "\r\n502 Bad Gateway\r\n<body bgcolor=\"white\">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n\r\n": invalid syntax Instance Count Estimated total/day (no EBS)

t2.micro 4
t2.small 5
c4.large 1
c5.xlarge 2
c4.xlarge 1
m3.medium 1
t2.medium 24
c5.large 6
$0.00
aleksei@myhost:~$ awless version version=v0.1.11, commit=d6094ca2f507ce9ecd8c8f8de030fbea60923ebf, build-date=2018-06-21T12:04:05+02:00, build-arch=amd64, build-os=linux, build-for=targz `

simcap commented 5 years ago

The pricer is not to be used anymore since the service http://ec2-price.com/ that the pricer used is no longer viable.

dserodio commented 5 years ago

What would be a nice pricing API for awless to consume? Implementing it might be a cool weekend project

danielhoherd commented 4 years ago

As a suggestion to fix this feature, we could use the ec2-instances.info scraper. https://github.com/powdahound/ec2instances.info/blob/master/scrape.py