awslabs / cid-framework

Apache License 2.0
30 stars 28 forks source link

HIVE_INVALID_METADATA glue table compute_optimizer_ebs_volume #160

Closed djcors closed 4 months ago

djcors commented 4 months ago

hi team.

im deployed compute optimizer dashboard, but i have this issue when i run a query in athena

HIVE_INVALID_METADATA: Glue table 'co_delfos_tecnologia_raw_dev_rl.compute_optimizer_ebs_volume' column 'tags' has invalid data type: struct<owner:string,country:string,confidentiality:string,dynatrace:string,availability:string,functionality:string,env:string,information-domain:string,Name:string,capacity:string,map-migrated:string,tfmodule:string,integrity:string,Schedule:string,pci:string,bia:string,sox:string,date:string,eks\:cluster-name:string,karpenter.sh/discovery:string,eks\:nodegroup-name:string,deployment:string,karpenter.sh/provisioner-name:string,kubernetes.io/cluster/devops-agents-operations-qa:string,kubernetes.io/cluster/devops-agents-devsecops-et-qa:string,kubernetes.io/cluster/devops-agents-release-qa:string,NEQUI\:cumplimiento:string,NEQUI\:dominio-informacion:string,NEQUI\:clasificacion-confidencialidad:string,NEQUI\:clasificacion-disponibilidad:string,NEQUI\:datos-personales:string,NEQUI\:clasificacion-integridad:string,Proyecto:string,nequi\:bkdiario:string,Ambiente:string,nequi\:environment:string,nequi\:country:string,migrated:string,impact:string,Owner:string,PrincipalId:string,ProjectName:string,Project:string,Department:string,Environment:string,Stage:string,Security:string,Time-stop:string,Schedulable:string,projectname:string,ebs:string,Scope:string> This query ran against the "co_delfos_tecnologia_raw_dev_rl" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 29f03462-f560-408b-8bbc-8d9737377b24

iakov-aws commented 4 months ago

Do you use crawler to create this table? if so please note that CO data are not crawler friendly.

please proceed with compute optimizer dashboard installation https://catalog.workshops.aws/awscid/en-US/dashboards/advanced/compute-optimizer

this contains static view that can deal with this kind of data (converting tags to string)