search

dwarszawski / amundsen-atlas-types

Atlas custom type definitions
Apache License 2.0
16 stars 11 forks source link

"Given typename StatisticsEntry was invalid" when creating types #17

Open hashhar opened 4 years ago

hashhar commented 4 years ago

Steps:

  1. Install Apache Atlas release-2.1.0-rc1 by building from source (not sure if it matters).
  2. Run the Atlas quick start to create some sample data and types as bin/quick_start.py

  3. Install amundsen-atlas-types from source (python3 setup.py install) and run the following from a Python shell:

    from amundsenatlastypes import Initializer

init = Initializer()
init.create_required_entities(fix_existing_data=True)

  4. This logs the following at the start but then is able to create other entities without any error.

    Trying to create Column Entity                                            
Already Exists, updating Column Entity
Something wrong happened: HTTP request failed for PUT http://localhost:21000/api/atlas/v2/types/typedefs: Not found 404: {"errorCode":"ATLAS-404-
00-001","errorMessage":"Given typename StatisticsEntry was invalid"}       
Applied Column Entity Definition

Now trying to get the type shows:

$ curl -X GET http://localhost:21000/api/atlas/v2/types/structdef/name/StatisticsEntry -u admin:admin
{"errorCode":"ATLAS-404-00-001","errorMessage":"Given typename StatisticsEntry was invalid"}

Is there something I'm doing wrong or can someone else reproduce this?

mgorsk1 commented 4 years ago

@hashhar thanks for filing this one. I am quite certain that 1.1.0 version of amundsenatlastypes doesn't have StatisticsEntry entity - are you sure you are using latest code ?

second reason why it might be following is incosistencies between Atlas sample data and our entities. can you confirm it doesn't work even if you omit step 2 (loading sample data) ?

hashhar commented 4 years ago

On master I can indeed see StatisticsEntry in structDefs at https://github.com/dwarszawski/amundsen-atlas-types/blob/master/amundsenatlastypes/schema/01_column_schema.json.

It works without error if I skip loading sample data. :+1:

pPanda-beta commented 4 years ago

I can confirm even without loading the sample data this problem exists.

'{"errorCode":"ATLAS-404-00-001","errorMessage":"Given typename StatisticsEntry was invalid"}'

And the logs from atlas 

2020-07-08 08:47:04,778 WARN  - [pool-2-thread-2 - 070b577b-3d87-4020-863a-75d0ff57f8a5:] ~ No RelationshipDef defined between Table and Reader on attribute: Table.readers (AtlasTypeRegistry:258)
2020-07-08 08:47:04,789 ERROR - [pool-2-thread-2 - 070b577b-3d87-4020-863a-75d0ff57f8a5:] ~ graph rollback due to exception  (GraphTransactionInterceptor:167)
org.apache.atlas.exception.AtlasBaseException: Table.owner : attribute delete not supported
    at org.apache.atlas.repository.store.graph.v2.AtlasStructDefStoreV2.updateVertexPreUpdate(AtlasStructDefStoreV2.java:406)
    at org.apache.atlas.repository.store.graph.v2.AtlasEntityDefStoreV2.updateVertexPreUpdate(AtlasEntityDefStoreV2.java:336)
    at org.apache.atlas.repository.store.graph.v2.AtlasEntityDefStoreV2.updateByName(AtlasEntityDefStoreV2.java:212)
    at org.apache.atlas.repository.store.graph.v2.AtlasEntityDefStoreV2.update(AtlasEntityDefStoreV2.java:179)
    at org.apache.atlas.repository.store.graph.v2.AtlasEntityDefStoreV2.update(AtlasEntityDefStoreV2.java:44)
    at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.updateGraphStore(AtlasTypeDefGraphStore.java:1035)
    at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.updateTypesDef(AtlasTypeDefGraphStore.java:481)
    at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
    at org.apache.atlas.repository.store.graph.v2.AtlasTypeDefGraphStoreV2$$EnhancerBySpringCGLIB$$f4fc3ad8.updateTypesDef(<generated>)
    at org.apache.atlas.web.rest.TypesREST.updateAtlasTypeDefs(TypesREST.java:407)

Versions :

pPanda-beta commented 4 years ago

Same happened while updating Table schema as well,

HTTP request failed for PUT http://172.21.255.199:21000/api/atlas/v2/types/typedefs: Bad request 400: {"errorCode":"ATLAS-400-00-00D","errorMessage":"Table.owner : attribute delete not supported"}
dwarszawski commented 4 years ago

@pPanda-beta where that Table.owner attribute comes from? Have you tried to run on the fresh instance of Atlas?

pPanda-beta commented 4 years ago

@dwarszawski - I use docker, every time I'm cleaning up the volumes its as fresh as a daisy.

May be this version is not supported by amundsen types : Atlas : release-2.1.0-rc2

dwarszawski commented 4 years ago

@pPanda-beta amundsen-atlas-types in version 1.1.0 was released before atlas-2.1.0-rc was created. Could you run it from master branch? Let me try to switch our instance to rc2 and try to reproduce this problem.