Closed Azure365Addict closed 8 months ago
On the subscriptions tab did you select the costs check box for the subscription?
On Tue, 20 Feb 2024, 13:21 TrapMusicJunkie, @.***> wrote:
Costs are not showing up. All other columns are fine.
Is Billing Administrator role is enough to see the costs? image.png (view on web) https://github.com/badlydressedboy/azure-data-costs-ui/assets/78150989/5b5da6e3-7fa7-4748-8db3-20a8df91b9f2
— Reply to this email directly, view it on GitHub https://github.com/badlydressedboy/azure-data-costs-ui/issues/3, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ47QSUSETLLV33HEFO2VTYUSBJZAVCNFSM6AAAAABDQ6ZQBSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE2DIMJUGY3DKNI . You are receiving this because you are subscribed to this thread.Message ID: @.***>
On the subscriptions tab did you select the costs check box for the subscription?
Yes, Sir. Got everything checked:
Did you download the exe or did you clone the code? If you have the code you can put a breakpoint in the http call for the costs to see what is wrong. If you got the exe then you will have to wait for me to add code that displays cost query errors. This might be end of day today or tomorrow.
As long as you can see costs in the portal you should be able to see them in the tool. Presume you can see them in portal?
On Tue, 20 Feb 2024, 13:30 TrapMusicJunkie, @.***> wrote:
On the subscriptions tab did you select the costs check box for the subscription?
Yes, Sir. Got everything checked:
image.png (view on web) https://github.com/badlydressedboy/azure-data-costs-ui/assets/78150989/1dd18a18-56b2-48b8-b2b9-b0f7cbc3e022
— Reply to this email directly, view it on GitHub https://github.com/badlydressedboy/azure-data-costs-ui/issues/3#issuecomment-1954018973, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ47QXP7W3J5SKJ4KJFELDYUSCMXAVCNFSM6AAAAABDQ6ZQBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGAYTQOJXGM . You are receiving this because you commented.Message ID: @.***>
I have downloaded the exe. I can see the costs in portal.
Ok, I've got you covered, will add logging around costs queries, will let you know when ready to download.
On Tue, 20 Feb 2024, 14:24 TrapMusicJunkie, @.***> wrote:
I have downloaded the exe. I can see the costs in portal.
image.png (view on web) https://github.com/badlydressedboy/azure-data-costs-ui/assets/78150989/5d95dcad-9303-4bb8-891e-4206179de606
image.png (view on web) https://github.com/badlydressedboy/azure-data-costs-ui/assets/78150989/b7f0bf7f-4850-494c-92fc-8fbbfd3b64fb
— Reply to this email directly, view it on GitHub https://github.com/badlydressedboy/azure-data-costs-ui/issues/3#issuecomment-1954104761, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ47QVU2KVYCOG6D4GVX5LYUSIWPAVCNFSM6AAAAABDQ6ZQBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGEYDINZWGE . You are receiving this because you commented.Message ID: @.***>
After costs dont load, do you get any messages on the subscriptions errors column?
[image: image.png]
On Tue, Feb 20, 2024 at 1:30 PM TrapMusicJunkie @.***> wrote:
On the subscriptions tab did you select the costs check box for the subscription?
Yes, Sir. Got everything checked:
image.png (view on web) https://github.com/badlydressedboy/azure-data-costs-ui/assets/78150989/1dd18a18-56b2-48b8-b2b9-b0f7cbc3e022
— Reply to this email directly, view it on GitHub https://github.com/badlydressedboy/azure-data-costs-ui/issues/3#issuecomment-1954018973, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ47QXP7W3J5SKJ4KJFELDYUSCMXAVCNFSM6AAAAABDQ6ZQBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJUGAYTQOJXGM . You are receiving this because you commented.Message ID: @.***>
Ahh sorry just noticed that I actually get some errors. I get "No expected DB costs found" error. Once I hit refresh it changes to "Subscription 'Enterprise' costs query Too Many Requests".
Tried with Billing Administrator role, Contributor role to subscription and Cost Management Reader - no luck.
I am logged into Visual Studio if it is relevant.
The "Too Many Requests" message is an indicator that requests are being throttled. This is why the subscriptions tab lets you select only a subset of your subscriptions. Try only selecting 1 subscription to begin with and see if costs get picked up. If they do then experiment with how many subscriptions you can select before you get that error. It is all dependent on how many things you have in each subscription so there is no rule for everybody, just a little bit of experimentation.
We have about 20 subscriptions. Some of them have like 1-2 resources. Costs are not visible even if I choose only 1 subscription. I always get "No expected DB costs found" and than "Too Many Requests". I will try to log in via Azure CLI and see if I encounter the same issue.
Tested while logged in via Azure CLI but have the same issue. Not sure if it is relevant but when I hit "Analyse spend" button "Over Provision%" column gets refreshed.
New version with logging/tracing which will help make the issue visible will be available next week. Will let you know when.
On Thu, 22 Feb 2024, 14:14 TrapMusicJunkie, @.***> wrote:
Tested while logged in via Azure CLI but have the same issue. Not sure if it is relevant but when I hit "Analyse spend" button "Over Provision%" column gets refreshed.
— Reply to this email directly, view it on GitHub https://github.com/badlydressedboy/azure-data-costs-ui/issues/3#issuecomment-1959332142, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ47QUA3IAGXHHDWKJJQATYU4ZAXAVCNFSM6AAAAABDQ6ZQBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJZGMZTEMJUGI . You are receiving this because you commented.Message ID: @.***>
Grab the latest release and look in the info log which will be created on the logs folder, wherever the exe runs. Look for:
Thank you for adding the logs.
Looking at the info log I get the following errors:
_2024-02-28 10:28:43.1772|INFO|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|HTTP Header x-ms-ratelimit-remaining-subscription-reads: 2024-02-28 10:28:43.1772|ERROR|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|Couldnt get costs for subscription: {my_subscription_name}; Bad Request 2024-02-28 10:28:43.1772|INFO|Azure.Costs.Ui.Wpf.DataContextVM.RefreshDatabases(DataContextVM.cs:497)|No expected DB costs found for sub: {my_subscription_name} 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB dbname? Costs.count: 0
(replaced original db names with "db_name").
Tried via Azure CLI with Billing Administrator role + Contributor role.
I will look into this and get back to you. The bad request is the point of failure, something about what is being sent in the rest call is funky, I will add logging to display the call content...
On Wed, 28 Feb 2024, 09:48 TrapMusicJunkie, @.***> wrote:
Thank you for adding the logs.
Looking at the info log I get the following errors:
2024-02-28 10:28:43.1772|INFO|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|HTTP Header x-ms-ratelimit-remaining-subscription-reads: 2024-02-28 10:28:43.1772|ERROR|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|Couldnt get costs for subscription: {my_subscription_name}; Bad Request 2024-02-28 10:28:43.1772|INFO|Azure.Costs.Ui.Wpf.DataContextVM.RefreshDatabases(DataContextVM.cs:497)|No expected DB costs found for sub: {my_subscription_name} 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0 2024-02-28 10:40:21.4903|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db! 2024-02-28 10:40:21.4903|INFO|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:823)|why no cost for DB db_name? Costs.count: 0
(replaced original db names with "db_name").
Tried via Azure CLI with Billing Administrator role + Contributor role.
— Reply to this email directly, view it on GitHub https://github.com/badlydressedboy/azure-data-costs-ui/issues/3#issuecomment-1968607053, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ47QTNC2EOJQSM4KE4GEDYV34PFAVCNFSM6AAAAABDQ6ZQBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRYGYYDOMBVGM . You are receiving this because you commented.Message ID: @.***>
OK grab latest - it now logs 2 extra things on cost query failure:
Let me know what you find. The problem and fix should be in plain sight now.
OK. Checked the latest version and I get the following error:
2024-02-29 11:44:40.3402|ERROR|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|Couldnt get costs for subscription: {my_subscription_name}; Bad Request
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|Request url: https://management.azure.com/subscriptions/{my_subscription_id}/providers/Microsoft.CostManagement/query?api-version=2021-10-01
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|Request content: {"type":"ActualCost"
,"timeframe": "Custom"
, "timePeriod":{"from":"2024-01-30T10:44:38Z","to":"2024-02-29T10:44:38Z"}
,"dataSet":{
"granularity":"None"
,"filter": {
"dimensions": {
"name": "serviceName"
,"operator": "In"
,"values": [
"Azure Data Factory v2"
,"SQL Database"
,"SQL Server"
,"Storage"
,"Virtual machines"
,"Bandwidth"
,"Virtual Network"
,"Advanced Threat Protection"
,"Purview"
,"Azure Purview"
,"Power BI Embedded"
,"Azure Synapse Analytics"
,"Synapse SQL Pool"
]
}
}
,"aggregation":{
"totalCost":{
"name":"Cost"
,"function":"Sum"
},
"totalCostUSD":{
"name":"CostUSD"
,"function":"Sum"
}
}
,"sorting":[
{"direction":"descending"
,"name":"Cost"}
]
,"grouping":[
{"type":"Dimension"
,"name":"ResourceId"
}
,{"type":"Dimension"
,"name":"ServiceName"
}
,{"type":"Dimension"
,"name":"MeterSubCategory"
}
,{"type":"Dimension"
,"name":"Product"
}
,{"type":"Dimension"
,"name":"Meter"
}
,{"type":"Dimension"
,"name":"ChargeType"
}
,{"type":"Dimension"
,"name":"PublisherType"
}
,{"type":"Dimension"
,"name":"Provider"
}
,{"type":"Dimension"
,"name":"MeterCategory"
}
,{"type":"Dimension"
,"name":"ResourceType"
}
]
}
}
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db!
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db!
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db!
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db!
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db!
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db!
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db!
2024-02-29 11:44:40.3440|ERROR|Azure.Costs.Ui.Wpf.DataContextVM.MapCostToDb(DataContextVM.cs:792)|elastic db!
I used your url and content in postman and it worked fine so it is likely something environmental. I have added outputting the error detail text of the bad request to the log, please get latest release again and retry. Should be interesting. Please send the error message and also try to call the endpoint in postman - have you used postman? It's free/easy to use.
Here is the error detail:
2024-03-01 11:09:15.6833|ERROR|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|Response text: {"error":{"code":"BadRequest","message":"Invalid query definition, Dataset is invalid or not supplied. (Request ID: ID_HERE)"}} 2024-03-01 11:09:15.6833|ERROR|Azure.Costs.Common.APIAccess.GetSubscriptionCosts|Use the above token, url and content to test in postman (set authorization: bearer token & paste token in, url is a POST and paste content into body as raw)
So I also tried to use postman and I got the same error
"error": {
"code": "BadRequest",
"message": "Invalid query definition, Dataset is invalid or not supplied. (Request ID: d13cf81e-6dff-4a84-b046-01ee60c6f14f)"
}
}
Did you have to modify the default postman headers to get it to that point? It should have errored out with a content type error unless you changed the headers.
Can you run your json request content through this free tool: https://www.atatus.com/tools/json-lint
If it is valid json take the formatted version on the right and try it in postman again.
I did not modify postman headers. Just copied and pasted url, token and body.
Can you run your json request content through this free tool: https://www.atatus.com/tools/json-lint
Basically, I have pasted the request content from error log and got some weird output. Not sure if I am doing it correctly?
If you get rid of the first 2 {{ does it work?
On Tue, 5 Mar 2024, 10:08 TrapMusicJunkie, @.***> wrote:
I did not modify postman headers. Just copied and pasted url, token and body.
Can you run your json request content through this free tool: https://www.atatus.com/tools/json-lint
Basically, I have pasted the request content from error log and got some weird output. Not sure if I am doing it correctly?
image.png (view on web) https://github.com/badlydressedboy/azure-data-costs-ui/assets/78150989/ae3cebb9-32a5-4662-85d0-b76e8c52cc4f
— Reply to this email directly, view it on GitHub https://github.com/badlydressedboy/azure-data-costs-ui/issues/3#issuecomment-1978397870, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ47QRRBE26FNJPSKDO7U3YWWKRPAVCNFSM6AAAAABDQ6ZQBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGM4TOOBXGA . You are receiving this because you were assigned.Message ID: @.***>
No, I get the following output:
Parse error on line 2:
...: "ActualCost", , , "timefra
----------------------^
Expecting 'STRING', got ',',
How did you get json that starts with 3 { ? Your json you posted in the comment 7 above this one, start with a single {. If you use the json you posted above, does it work?
You are right. The fresh one generated today starts with 3 {
Once I entered the old one (with a single { )posted in my earlier post it seems like it got validated:
Please download latest release - it gets rids of the 3 { bug, which should give you valid json and potentially a successful result in postman.
json is valid and postman results in a different error this time (same as the one in the AzureDataCosts-Error.log):
I have removed product and provider fields from the query so the error above should be fixed. Grab latest and let me know.
I would say it is even worse but actually at some point it says that it got costs for subscription OK:
From the other hand when I check postman I can see that it tries to fetch the costs:
What country are you in? Do your regional settings use . or , as a decimal point in numbers?
I am located in Poland. We use , as a decimal:
OK makes sense now. I have added code to accept , as well as .. Pls grab latest...
Seems like we have a success here 🥇
Thank you very much for your patience and time on this! Really appreciated 👍
Will have few more questions but I will post them in the discussions section.
AWESOME! That was painful but really happy it paid off, thanks also for sticking with it!
Learnings:
In this case I will close this issue. Thanks!
Costs are not showing up. All other columns are fine.
Is Billing Administrator role enough to see the costs?