search

datahub-project / datahub

The Metadata Platform for your Data Stack
https://datahubproject.io
Apache License 2.0
9.7k stars 2.87k forks source link

python3 or python2? kafka_etl #1688

Closed fageiguanbing closed 4 years ago

fageiguanbing commented 4 years ago

use python3 error: run python3 kafka_etl.py

  File "kafka_etl.py", line 61
    print topic
              ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(topic)?

use python2 error: run pip install --user -r requirements.txt

Collecting confluent-kafka[avro]==1.1.0
  Using cached https://files.pythonhosted.org/packages/bc/52/f7464e17cfb2c4b083cff5f3ac39522ad3cbb851ed8c79810884fbe0902d/confluent_kafka-1.1.0-cp27-cp27m-macosx_10_6_intel.whl
Collecting python-schema-registry-client==1.2.1
  Using cached https://files.pythonhosted.org/packages/ac/56/c08ba77164fb26539746c3c81e3ad0940af581d75a36e44a4541938f5253/python-schema-registry-client-1.2.1.tar.gz
Collecting kazoo==2.5.0
  Using cached https://files.pythonhosted.org/packages/76/af/942ec55feb58f8774e88fa6fd63433b53f4ab1b0b1397ff9355280162ec0/kazoo-2.5.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4"
  Using cached https://files.pythonhosted.org/packages/6f/2c/a9386903ece2ea85e9807e0e062174dc26fdce8b05f216d00491be29fad5/enum34-1.1.10-py2-none-any.whl
Collecting requests; python_version < "3.2"
  Using cached https://files.pythonhosted.org/packages/1a/70/1935c770cb3be6e3a8b78ced23d7e0f3b187f5cbfab4749523ed65d7c9b1/requests-2.23.0-py2.py3-none-any.whl
Collecting futures; python_version < "3.2"
  Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Collecting avro; python_version < "3.0" and extra == "avro"
  Using cached https://files.pythonhosted.org/packages/d7/0b/592692ed26de33f35bf596780e6adb85c47e3e58061369bbc99125b902ec/avro-1.9.2.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/4d/3zsvmn6j4019gfhmfphgfxsh0000gn/T/pip-install-M6XXRJ/avro/setup.py'"'"'; __file__='"'"'/private/var/folders/4d/3zsvmn6j4019gfhmfphgfxsh0000gn/T/pip-install-M6XXRJ/avro/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/4d/3zsvmn6j4019gfhmfphgfxsh0000gn/T/pip-install-M6XXRJ/avro/pip-egg-info
         cwd: /private/var/folders/4d/3zsvmn6j4019gfhmfphgfxsh0000gn/T/pip-install-M6XXRJ/avro/
    Complete output (47 lines):
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
      warnings.warn(msg)
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'testsuite'
    warning: no previously-included files matching '*.pyo' found under directory 'testsuite'
    no previously-included directories found matching 'docs/_build'

    Installed /private/var/folders/4d/3zsvmn6j4019gfhmfphgfxsh0000gn/T/pip-install-M6XXRJ/avro/.eggs/pycodestyle-2.6.0-py2.7.egg
    Searching for isort
    Reading https://pypi.python.org/simple/isort/
    Best match: isort 4.3.21
    Downloading https://files.pythonhosted.org/packages/43/00/8705e8d0c05ba22f042634f791a61f4c678c32175763dcf2ca2a133f4739/isort-4.3.21.tar.gz#sha256=54da7e92468955c4fceacd0c86bd0ec997b0e1ee80d97f67c35a78b719dccab1
    Processing isort-4.3.21.tar.gz
    Writing /var/folders/4d/3zsvmn6j4019gfhmfphgfxsh0000gn/T/easy_install-wqDmjg/isort-4.3.21/setup.cfg
    Running isort-4.3.21/setup.py -q bdist_egg --dist-dir /var/folders/4d/3zsvmn6j4019gfhmfphgfxsh0000gn/T/easy_install-wqDmjg/isort-4.3.21/egg-dist-tmp-uhQWE1
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
      warnings.warn(msg)
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/4d/3zsvmn6j4019gfhmfphgfxsh0000gn/T/pip-install-M6XXRJ/avro/setup.py", line 84, in <module>
        'zstandard': ['zstandard'],
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/dist.py", line 268, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/dist.py", line 313, in fetch_build_eggs
        replace_conflicting=True,
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 843, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 1088, in best_match
        return self.obtain(req, installer)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 1100, in obtain
        return installer(requirement)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/dist.py", line 380, in fetch_build_egg
        return cmd.easy_install(req)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/command/easy_install.py", line 638, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/command/easy_install.py", line 668, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/command/easy_install.py", line 851, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/command/easy_install.py", line 1079, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/setuptools/command/easy_install.py", line 1067, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error in isort setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Expected version spec in futures; python_version < "3.2" at ; python_version < "3.2"
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
casperfrx commented 4 years ago

Yeah it was written for python 2.

You can solve it of course by changing it to print(topic).

You should however run into more issues, I had them as well.

I'd suggest you use python3.7 with python3-pip and change the requirements.txt to:

confluent-kafka
avro-python3==1.8.2
kazoo==2.5.0
requests

And let pip figure it out itself. Note that I specifically required avro-python3 1.8.2 because there's a dependency issue with the newer version which breaks atm.

fageiguanbing commented 4 years ago

Yeah it was written for python 2.

You can solve it of course by changing it to print(topic).

You should however run into more issues, I had them as well.

I'd suggest you use python3.7 with python3-pip and change the requirements.txt to:

confluent-kafka
avro-python3==1.8.2
kazoo==2.5.0
requests

And let pip figure it out itself. Note that I specifically required avro-python3 1.8.2 because there's a dependency issue with the newer version which breaks atm.

I use python3.7 to run kafka_etl.py, and I have changed it to print(topic), new issues. T.T

Traceback (most recent call last):
  File "kafka_etl.py", line 5, in <module>
    from confluent.schemaregistry.client import CachedSchemaRegistryClient
ModuleNotFoundError: No module named 'confluent'

Does it seem that some dependencies not import?

casperfrx commented 4 years ago

Change the import statement from: from confluent to: from confluent_kafka.avro

On Wed, Jun 3, 2020, 07:52 发给官兵 notifications@github.com wrote:

Yeah it was written for python 2.

You can solve it of course by changing it to print(topic).

You should however run into more issues, I had them as well.

I'd suggest you use python3.7 with python3-pip and change the requirements.txt to:

confluent-kafka avro-python3==1.8.2 kazoo==2.5.0 requests

And let pip figure it out itself. Note that I specifically required avro-python3 1.8.2 because there's a dependency issue with the newer version which breaks atm.

I use python3.7 to run kafka_etl.py, and I have changed it to print(topic), new issues. T.T

Traceback (most recent call last): File "kafka_etl.py", line 5, in from confluent.schemaregistry.client import CachedSchemaRegistryClient ModuleNotFoundError: No module named 'confluent'

Does it seem that some dependencies not import?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/linkedin/datahub/issues/1688#issuecomment-637970553, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQMENQN3GT2OVQEF2O3QGLRUXQKDANCNFSM4NRJJ5BA .

fageiguanbing commented 4 years ago

Change the import statement from: from confluent to: from confluent_kafka.avro … On Wed, Jun 3, 2020, 07:52 发给官兵 @.***> wrote: Yeah it was written for python 2. You can solve it of course by changing it to print(topic). You should however run into more issues, I had them as well. I'd suggest you use python3.7 with python3-pip and change the requirements.txt to: confluent-kafka avro-python3==1.8.2 kazoo==2.5.0 requests And let pip figure it out itself. Note that I specifically required avro-python3 1.8.2 because there's a dependency issue with the newer version which breaks atm. I use python3.7 to run kafka_etl.py, and I have changed it to print(topic), new issues. T.T Traceback (most recent call last): File "kafka_etl.py", line 5, in from confluent.schemaregistry.client import CachedSchemaRegistryClient ModuleNotFoundError: No module named 'confluent' Does it seem that some dependencies not import? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1688 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQMENQN3GT2OVQEF2O3QGLRUXQKDANCNFSM4NRJJ5BA .

kafka_etl.py:54: DeprecationWarning: CachedSchemaRegistry constructor is being deprecated. Use CachedSchemaRegistryClient(dict: config) instead. Existing params ca_location, cert_location and key_location will be replaced with their librdkafka equivalents as keys in the conf dict: `ssl.ca.location`, `ssl.certificate.location` and `ssl.key.location` respectively
  client = CachedSchemaRegistryClient(SCHEMAREGISTRY)
Schema not found:404
topic-trans-value
Traceback (most recent call last):
  File "kafka_etl.py", line 64, in <module>
    build_kafka_dataset_mce(dataset_name, str(schema), int(schema_version))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

and long(time.time() long seems undefined or not import. I'm not a python developer, I feel there will be more and more problems. Do you have scripts?

casperfrx commented 4 years ago

In transit atm so not readily available.

But change int(version) simply to 1

and remove long from time so long(time.time()) to time.time()

Those should be all.

On Wed, Jun 3, 2020, 09:16 发给官兵 notifications@github.com wrote:

Change the import statement from: from confluent to: from confluent_kafka.avro … <#m436325067260601451> On Wed, Jun 3, 2020, 07:52 发给官兵 @.***> wrote: Yeah it was written for python 2. You can solve it of course by changing it to print(topic). You should however run into more issues, I had them as well. I'd suggest you use python3.7 with python3-pip and change the requirements.txt to: confluent-kafka avro-python3==1.8.2 kazoo==2.5.0 requests And let pip figure it out itself. Note that I specifically required avro-python3 1.8.2 because there's a dependency issue with the newer version which breaks atm. I use python3.7 to run kafka_etl.py, and I have changed it to print(topic), new issues. T.T Traceback (most recent call last): File "kafka_etl.py", line 5, in from confluent.schemaregistry.client import CachedSchemaRegistryClient ModuleNotFoundError: No module named 'confluent' Does it seem that some dependencies not import? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1688 (comment) https://github.com/linkedin/datahub/issues/1688#issuecomment-637970553>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQMENQN3GT2OVQEF2O3QGLRUXQKDANCNFSM4NRJJ5BA .

kafka_etl.py:54: DeprecationWarning: CachedSchemaRegistry constructor is being deprecated. Use CachedSchemaRegistryClient(dict: config) instead. Existing params ca_location, cert_location and key_location will be replaced with their librdkafka equivalents as keys in the conf dict: ssl.ca.location, ssl.certificate.location and ssl.key.location respectively

client = CachedSchemaRegistryClient(SCHEMAREGISTRY)

Schema not found:404

topic-trans-value

Traceback (most recent call last):

File "kafka_etl.py", line 64, in 

build_kafka_dataset_mce(dataset_name, str(schema), int(schema_version))

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

and long(time.time() long seems undefined or not import. I'm not a python developer, I feel there will be more and more problems. Do you have scripts?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/linkedin/datahub/issues/1688#issuecomment-638006350, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQMENUFFI7PLE7YY6H7CULRUX2DJANCNFSM4NRJJ5BA .

fageiguanbing commented 4 years ago

In transit atm so not readily available. But change int(version) simply to 1 and remove long from time so long(time.time()) to time.time() Those should be all. … On Wed, Jun 3, 2020, 09:16 发给官兵 @.> wrote: Change the import statement from: from confluent to: from confluent_kafka.avro … <#m436325067260601451> On Wed, Jun 3, 2020, 07:52 发给官兵 @.> wrote: Yeah it was written for python 2. You can solve it of course by changing it to print(topic). You should however run into more issues, I had them as well. I'd suggest you use python3.7 with python3-pip and change the requirements.txt to: confluent-kafka avro-python3==1.8.2 kazoo==2.5.0 requests And let pip figure it out itself. Note that I specifically required avro-python3 1.8.2 because there's a dependency issue with the newer version which breaks atm. I use python3.7 to run kafka_etl.py, and I have changed it to print(topic), new issues. T.T Traceback (most recent call last): File "kafka_etl.py", line 5, in from confluent.schemaregistry.client import CachedSchemaRegistryClient ModuleNotFoundError: No module named 'confluent' Does it seem that some dependencies not import? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1688 (comment) <#1688 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQMENQN3GT2OVQEF2O3QGLRUXQKDANCNFSM4NRJJ5BA . kafka_etl.py:54: DeprecationWarning: CachedSchemaRegistry constructor is being deprecated. Use CachedSchemaRegistryClient(dict: config) instead. Existing params ca_location, cert_location and key_location will be replaced with their librdkafka equivalents as keys in the conf dict: ssl.ca.location, ssl.certificate.location and ssl.key.location respectively client = CachedSchemaRegistryClient(SCHEMAREGISTRY) Schema not found:404 topic-trans-value Traceback (most recent call last): File "kafka_etl.py", line 64, in build_kafka_dataset_mce(dataset_name, str(schema), int(schema_version)) TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType' and long(time.time() long seems undefined or not import. I'm not a python developer, I feel there will be more and more problems. Do you have scripts? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1688 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQMENUFFI7PLE7YY6H7CULRUX2DJANCNFSM4NRJJ5BA .

It works.

kafka_etl.py:54: DeprecationWarning: CachedSchemaRegistry constructor is being deprecated. Use CachedSchemaRegistryClient(dict: config) instead. Existing params ca_location, cert_location and key_location will be replaced with their librdkafka equivalents as keys in the conf dict: `ssl.ca.location`, `ssl.certificate.location` and `ssl.key.location` respectively
  client = CachedSchemaRegistryClient(SCHEMAREGISTRY)
MetadataChangeEvent-value

{'auditHeader': None, 'proposedSnapshot': ('com.linkedin.pegasus2avro.metadata.snapshot.DatasetSnapshot', {'urn': 'urn:li:dataset:(urn:li:dataPlatform:kafka,MetadataChangeEvent,PROD)', 'aspects': [{'schemaName': 'MetadataChangeEvent', 'platform': 'urn:li:dataPlatform:kafka', 'version': 1, 'created': {'time': 1591171510.2122471, 'actor': 'urn:li:corpuser:'}, 'lastModified': {'time': 1591171510.2122471, 'actor': 'urn:li:corpuser:'}, 'hash': '', 'platformSchema': {'documentSchema': '{"type": "record", "name": "MetadataChangeEvent", "namespace": "com.linkedin.pegasus2avro.mxe", "fields": [{"type": ["null", {"type": "record", "name": "KafkaAuditHeader", "namespace": "com.linkedin.events", "fields": [{"compliance": [{"policy": "EVENT_TIME"}], "type": "long", "name": "time", "doc": "The time at which the event was emitted into kafka."}, {"compliance": "NONE", "type": "string", "name": "server", "doc": "The fully qualified name of the host from which the event is being emitted."}, {"compliance": "NONE", "type": ["null", "string"], "name": "instance", "default": null, "doc": "The instance on the server from which the event is being emitted. e.g. i001"}, {"compliance": "NONE", "type": "string", "name": "appName", "doc": "The name of the application from which the event is being emitted. see go/appname"}, {"compliance": "NONE", "type": {"type": "fixed", "name": "UUID", "namespace": "com.linkedin.events", "size": 16}, "name": "messageId", "doc": "A unique identifier for the message"}, {"compliance": "NONE", "type": ["null", "int"], "name": "auditVersion", "default": null, "doc": "The version that is being used for auditing. In version 0, the audit trail buckets events into 10 minute audit windows based on the EventHeader timestamp. In version 1, the audit trail buckets events as follows: if the schema has an outer KafkaAuditHeader, use the outer audit header timestamp for bucketing; else if the EventHeader has an inner KafkaAuditHeader use that inner audit header\'s timestamp for bucketing"}, {"compliance": "NONE", "type": ["null", "string"], "name": "fabricUrn", "default": null, "doc": "The fabricUrn of the host from which the event is being emitted. Fabric Urn in the format of urn:li:fabric:{fabric_name}. See go/fabric."}, {"compliance": "NONE", "type": ["null", "string"], "name": "clusterConnectionString", "default": null, "doc": "This is a String that the client uses to establish some kind of connection with the Kafka cluster. The exact format of it depends on specific versions of clients and brokers. This information could potentially identify the fabric and cluster with which the client is producing to or consuming from."}], "doc": "This header records information about the context of an event as it is emitted into kafka and is intended to be used by the kafka audit application.  For more information see go/kafkaauditheader"}], "name": "auditHeader", "default": null, "doc": "Kafka audit header. See go/kafkaauditheader for more info."}, {"type": [{"type": "record", "name": "CorpUserSnapshot", "namespace": "com.linkedin.pegasus2avro.metadata.snapshot", "fields": [{"java": {"class": "com.linkedin.pegasus2avro.common.urn.CorpuserUrn"}, "type": "string", "name": "urn", "doc": "URN for the entity the metadata snapshot is associated with."}, {"type": {"type": "array", "items": [{"type": "record", "name": "CorpUserInfo", "namespace": "com.linkedin.pegasus2avro.identity", "fields": [{"type": "boolean", "name": "active", "doc": "Whether the corpUser is active, ref: https://iwww.corp.linkedin.com/wiki/cf/display/GTSD/Accessing+Active+Directory+via+LDAP+tools"}, {"type": ["null", "string"], "name": "displayName", "default": null, "doc": "displayName of this user ,  e.g.  Hang Zhang(DataHQ)"}, {"type": "string", "name": "email", "doc": "email address of this user"}, {"type": ["null", "string"], "name": "title", "default": null, "doc": "title of this user"}, {"java": {"class": "com.linkedin.pegasus2avro.common.urn.CorpuserUrn"}, "type": ["null", "string"], "name": "managerUrn", "default": null, "doc": "direct manager of this user"}, {"type": ["null", "long"], "name": "departmentId", "default": null, "doc": "department id this user belong to"}, {"type": ["null", "string"], "name": "departmentName", "default": null, "doc": "department name this user belong to"}, {"type": ["null", "string"], "name": "firstName", "default": null, "doc": "first name of this user"}, {"type": ["null", "string"], "name": "lastName", "default": null, "doc": "last name of this user"}, {"type": ["null", "string"], "name": "fullName", "default": null, "doc": "Common name of this user, format is firstName + lastName (split by a whitespace)"}, {"type": ["null", "string"], "name": "countryCode", "default": null, "doc": "two uppercase letters country code. e.g.  US"}], "doc": "Linkedin corp user information"}, {"type": "record", "name": "CorpUserEditableInfo", "namespace": "com.linkedin.pegasus2avro.identity", "fields": [{"type": ["null", "string"], "name": "aboutMe", "default": null, "doc": "About me section of the user"}, {"type": {"type": "array", "items": "string"}, "name": "teams", "default": [], "doc": "Teams that the user belongs to e.g. Metadata"}, {"type": {"type": "array", "items": "string"}, "name": "skills", "default": [], "doc": "Skills that the user possesses e.g. Machine Learning"}, {"java": {"class": "com.linkedin.pegasus2avro.common.url.Url", "coercerClass": "com.linkedin.pegasus2avro.common.url.UrlCoercer"}, "type": "string", "name": "pictureLink", "default": "https://raw.githubusercontent.com/linkedin/datahub/master/datahub-web/packages/data-portal/public/assets/images/default_avatar.png", "doc": "A URL which points to a picture which user wants to set as a profile photo"}], "doc": "Linkedin corp user information that can be edited from UI"}]}, "name": "aspects", "doc": "The list of metadata aspects associated with the CorpUser. Depending on the use case, this can either be all, or a selection, of supported aspects."}], "doc": "A metadata snapshot for a specific CorpUser entity."}, {"type": "record", "name": "DatasetSnapshot", "namespace": "com.linkedin.pegasus2avro.metadata.snapshot", "fields": [{"java": {"class": "com.linkedin.pegasus2avro.common.urn.DatasetUrn"}, "type": "string", "name": "urn", "doc": "URN for the entity the metadata snapshot is associated with."}, {"type": {"type": "array", "items": [{"type": "record", "name": "DatasetProperties", "namespace": "com.linkedin.pegasus2avro.dataset", "fields": [{"type": ["null", "string"], "name": "description", "default": null, "doc": "Documentation of the dataset"}, {"java": {"class": "java.net.URI"}, "type": ["null", "string"], "name": "uri", "default": null, "doc": "The abstracted URI such as hdfs:///data/tracking/PageViewEvent, file:///dir/file_name. Uri should not include any environment specific properties. Some datasets might not have a standardized uri, which makes this field optional (i.e. kafka topic)."}, {"type": {"type": "array", "items": "string"}, "name": "tags", "default": [], "doc": "tags for the dataset"}, {"type": {"type": "map", "values": "string"}, "name": "customProperties", "default": {}, "doc": "A key-value map to capture any other non-standardized properties for the dataset"}], "doc": "Properties associated with a Dataset"}, {"type": "record", "name": "DatasetDeprecation", "namespace": "com.linkedin.pegasus2avro.dataset", "fields": [{"type": "boolean", "name": "deprecated", "doc": "Whether the dataset is deprecated by owner."}, {"type": ["null", "long"], "name": "decommissionTime", "default": null, "doc": "The time user plan to decommission this dataset."}, {"type": "string", "name": "note", "doc": "Additional information about the dataset deprecation plan, such as the wiki, doc, RB."}, {"java": {"class": "com.linkedin.pegasus2avro.common.urn.Urn"}, "type": ["null", "string"], "name": "actor", "default": null, "doc": "The corpuser URN which will be credited for modifying this deprecation content."}], "doc": "Dataset deprecation status"}, {"type": "record", "name": "UpstreamLineage", "namespace": "com.linkedin.pegasus2avro.dataset", "fields": [{"type": {"type": "array", "items": {"type": "record", "name": "Upstream", "namespace": "com.linkedin.pegasus2avro.dataset", "fields": [{"type": {"type": "record", "name": "AuditStamp", "namespace": "com.linkedin.pegasus2avro.common", "fields": [{"type": "long", "name": "time", "doc": "When did the resource/association/sub-resource move into the specific lifecycle stage represented by this AuditEvent."}, {"java": {"class": "com.linkedin.pegasus2avro.common.urn.Urn"}, "type": "string", "name": "actor", "doc": "The entity (e.g. a member URN) which will be credited for moving the resource/association/sub-resource into the specific lifecycle stage. It is also the one used to authorize the change."}, {"java": {"class": "com.linkedin.pegasus2avro.common.urn.Urn"}, "type": ["null", "string"], "name": "impersonator", "default": null, "doc": "The entity (e.g. a service URN) which performs the change on behalf of the Actor and must be authorized to act as the Actor."}], "doc": "Data captured on a resource/association/sub-resource level giving insight into when that resource/association/sub-resource moved into a particular lifecycle stage, and who acted to move it into that specific lifecycle stage."}, "name": "auditStamp", "doc": "Audit stamp containing who reported the lineage and when"}, {"java": {"class": "com.linkedin.pegasus2avro.common.urn.DatasetUrn"}, "type": "string", "name": "dataset", "doc": "The upstream dataset the lineage points to"}, {"type": {"type": "enum", "symbolDocs": {"COPY": "Direct copy without modification", "TRANSFORMED": "Transformed data with modification (format or content change)", "VIEW": "Represents a view defined on the sources e.g. Hive view defined on underlying hive tables or a Hive table pointing to a HDFS dataset or DALI view defined on multiple sources"}, "name": "DatasetLineageType", "namespace": "com.linkedin.pegasus2avro.dataset", "symbols": ["COPY", "TRANSFORMED", "VIEW"], "doc": "The various types of supported dataset lineage"}, "name": "type", "doc": "The type of the lineage"}], "doc": "Upstream lineage information about a dataset including the source reporting the lineage"}}, "name": "upstreams", "doc": "List of upstream dataset lineage information"}], "doc": "Upstream lineage of a dataset"}, {"type": "record", "name": "InstitutionalMemory", "namespace": "com.linkedin.pegasus2avro.common", "fields": [{"type": {"type": "array", "items": {"type": "record", "name": "InstitutionalMemoryMetadata", "namespace": "com.linkedin.pegasus2avro.common", "fields": [{"java": {"class": "com.linkedin.pegasus2avro.common.url.Url", "coercerClass": "com.linkedin.pegasus2avro.common.url.UrlCoercer"}, "type": "string", "name": "url", "doc": "Link to an engineering design document or a wiki page."}, {"type": "string", "name": "description", "doc": "Description of the link."}, {"type": "com.linkedin.pegasus2avro.common.AuditStamp", "name": "createStamp", "doc": "Audit stamp associated with creation of this record"}], "doc": "Metadata corresponding to a record of institutional memory."}}, "name": "elements", "doc": "List of records that represent institutional memory of an entity. Each record consists of a link, description, creator and timestamps associated with that record."}], "doc": "Institutional memory of an entity. This is a way to link to relevant documentation and provide description of the documentation. Institutional or tribal knowledge is very important for users to leverage the entity."}, {"type": "record", "name": "Ownership", "namespace": "com.linkedin.pegasus2avro.common", "fields": [{"type": {"type": "array", "items": {"type": "record", "name": "Owner", "namespace": "com.linkedin.pegasus2avro.common", "fields": [{"java": {"class": "com.linkedin.pegasus2avro.common.urn.Urn"}, "type": "string", "name": "owner", "doc": "Owner URN, e.g. urn:li:corpuser:ldap, urn:li:corpGroup:group_name, and urn:li:multiProduct:mp_name"}, {"type": {"type": "enum", "symbolDocs": {"CONSUMER": "A person, group, or service that consumes the data", "DATAOWNER": "A person or group that is owning the data", "DELEGATE": "A person or a group that overseas the operation, e.g. a DBA or SRE.", "DEVELOPER": "A person or group that is in charge of developing the code", "PRODUCER": "A person, group, or service that produces/generates the data", "STAKEHOLDER": "A person or a group that has direct business interest"}, "name": "OwnershipType", "namespace": "com.linkedin.pegasus2avro.common", "symbols": ["DEVELOPER", "DATAOWNER", "DELEGATE", "PRODUCER", "CONSUMER", "STAKEHOLDER"], "doc": "Owner category or owner role"}, "name": "type", "doc": "The type of the ownership"}, {"type": ["null", {"type": "record", "name": "OwnershipSource", "namespace": "com.linkedin.pegasus2avro.common", "fields": [{"type": {"type": "enum", "symbolDocs": {"AUDIT": "Auditing system or audit logs", "DATABASE": "Database, e.g. GRANTS table", "FILE_SYSTEM": "File system, e.g. file/directory owner", "ISSUE_TRACKING_SYSTEM": "Issue tracking system, e.g. Jira", "MANUAL": "Manually provided by a user", "OTHER": "Other sources", "SERVICE": "Other ownership-like service, e.g. Nuage, ACL service etc", "SOURCE_CONTROL": "SCM system, e.g. GIT, SVN"}, "name": "OwnershipSourceType", "namespace": "com.linkedin.pegasus2avro.common", "symbols": ["AUDIT", "DATABASE", "FILE_SYSTEM", "ISSUE_TRACKING_SYSTEM", "MANUAL", "SERVICE", "SOURCE_CONTROL", "OTHER"]}, "name": "type", "doc": "The type of the source"}, {"type": ["null", "string"], "name": "url", "default": null, "doc": "A reference URL for the source"}], "doc": "Source/provider of the ownership information"}], "name": "source", "default": null, "doc": "Source information for the ownership"}], "doc": "Ownership information"}}, "name": "owners", "doc": "List of owners of the entity."}, {"type": "com.linkedin.pegasus2avro.common.AuditStamp", "name": "lastModified", "doc": "Audit stamp containing who last modified the record and when."}], "doc": "Ownership information of an entity."}, {"type": "record", "name": "Status", "namespace": "com.linkedin.pegasus2avro.common", "fields": [{"type": "boolean", "name": "removed", "default": false, "doc": "whether the entity is removed or not"}], "doc": "The status metadata of an entity, e.g. dataset, metric, feature, etc."}, {"type": "record", "name": "SchemaMetadata", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"validate": {"strlen": {"max": 500, "min": 1}}, "type": "string", "name": "schemaName", "doc": "Schema name e.g. PageViewEvent, identity.Profile, ams.account_management_tracking"}, {"java": {"class": "com.linkedin.pegasus2avro.common.urn.DataPlatformUrn"}, "type": "string", "name": "platform", "doc": "Standardized platform urn where schema is defined. The data platform Urn (urn:li:platform:{platform_name})"}, {"type": "long", "name": "version", "doc": "Every change to SchemaMetadata in the resource results in a new version. Version is server assigned. This version is differ from platform native schema version."}, {"type": "com.linkedin.pegasus2avro.common.AuditStamp", "name": "created", "doc": "An AuditStamp corresponding to the creation of this resource/association/sub-resource"}, {"type": "com.linkedin.pegasus2avro.common.AuditStamp", "name": "lastModified", "doc": "An AuditStamp corresponding to the last modification of this resource/association/sub-resource. If no modification has happened since creation, lastModified should be the same as created"}, {"type": ["null", "com.linkedin.pegasus2avro.common.AuditStamp"], "name": "deleted", "default": null, "doc": "An AuditStamp corresponding to the deletion of this resource/association/sub-resource. Logically, deleted MUST have a later timestamp than creation. It may or may not have the same time as lastModified depending upon the resource/association/sub-resource semantics."}, {"java": {"class": "com.linkedin.pegasus2avro.common.urn.DatasetUrn"}, "type": ["null", "string"], "name": "dataset", "default": null, "doc": "Dataset this schema metadata is associated with."}, {"type": ["null", "string"], "name": "cluster", "default": null, "doc": "The cluster this schema metadata resides from"}, {"type": "string", "name": "hash", "doc": "the SHA1 hash of the schema content"}, {"type": [{"type": "record", "name": "EspressoSchema", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "documentSchema", "doc": "The native espresso document schema."}, {"type": "string", "name": "tableSchema", "doc": "The espresso table schema definition."}], "doc": "Schema text of an espresso table schema."}, {"type": "record", "name": "OracleDDL", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "tableSchema", "doc": "The native schema in the dataset\'s platform. This is a human readable (json blob) table schema."}], "doc": "Schema holder for oracle data definition language that describes an oracle table."}, {"type": "record", "name": "MySqlDDL", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "tableSchema", "doc": "The native schema in the dataset\'s platform. This is a human readable (json blob) table schema."}], "doc": "Schema holder for MySql data definition language that describes an MySql table."}, {"type": "record", "name": "PrestoDDL", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "rawSchema", "doc": "The raw schema in the dataset\'s platform. This includes the DDL and the columns extracted from DDL."}], "doc": "Schema holder for presto data definition language that describes a presto view."}, {"type": "record", "name": "KafkaSchema", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "documentSchema", "doc": "The native kafka document schema. This is a human readable avro document schema."}], "doc": "Schema holder for kafka schema."}, {"type": "record", "name": "BinaryJsonSchema", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "schema", "doc": "The native schema text for binary JSON file format."}], "doc": "Schema text of binary JSON schema."}, {"type": "record", "name": "OrcSchema", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "schema", "doc": "The native schema for ORC file format."}], "doc": "Schema text of an ORC schema."}, {"type": "record", "name": "Schemaless", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "The dataset has no specific schema associated with it"}, {"type": "record", "name": "KeyValueSchema", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "keySchema", "doc": "The raw schema for the key in the key-value store."}, {"type": "string", "name": "valueSchema", "doc": "The raw schema for the value in the key-value store."}], "doc": "Schema text of a key-value store schema."}, {"type": "record", "name": "OtherSchema", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "rawSchema", "doc": "The native schema in the dataset\'s platform."}], "doc": "Schema holder for undefined schema types."}], "name": "platformSchema", "doc": "The native schema in the dataset\'s platform."}, {"type": {"type": "array", "items": {"type": "record", "name": "SchemaField", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "fieldPath", "doc": "Flattened name of the field. Field is computed from jsonPath field. For data translation rules refer to wiki page above."}, {"type": ["null", "string"], "name": "jsonPath", "default": null, "doc": "Flattened name of a field in JSON Path notation."}, {"type": "boolean", "name": "nullable", "default": false, "doc": "Indicates if this field is optional or nullable"}, {"type": ["null", "string"], "name": "description", "default": null, "doc": "Description"}, {"type": {"type": "record", "name": "SchemaFieldDataType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": [{"type": "record", "name": "BooleanType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "Boolean field type."}, {"type": "record", "name": "FixedType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "Fixed field type."}, {"type": "record", "name": "StringType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "String field type."}, {"type": "record", "name": "BytesType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "Bytes field type."}, {"type": "record", "name": "NumberType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "Number data type: long, integer, short, etc.."}, {"type": "record", "name": "EnumType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "Enum field type."}, {"type": "record", "name": "NullType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "Null field type."}, {"type": "record", "name": "MapType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": ["null", "string"], "name": "keyType", "default": null, "doc": "Key type in a map"}, {"type": ["null", "string"], "name": "valueType", "default": null, "doc": "Type of the value in a map"}], "doc": "Map field type."}, {"type": "record", "name": "ArrayType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": ["null", {"type": "array", "items": "string"}], "name": "nestedType", "default": null, "doc": "List of types this array holds."}], "doc": "Array field type."}, {"type": "record", "name": "UnionType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": ["null", {"type": "array", "items": "string"}], "name": "nestedTypes", "default": null, "doc": "List of types in union type."}], "doc": "Union field type."}, {"type": "record", "name": "RecordType", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [], "doc": "Record field type."}], "name": "type", "doc": "Data platform specific types"}], "doc": "Schema field data types"}, "name": "type", "doc": "Platform independent field type of the field."}, {"type": "string", "name": "nativeDataType", "doc": "The native type of the field in the dataset\'s platform as declared by platform schema."}, {"type": "boolean", "name": "recursive", "default": false, "doc": "There are use cases when a field in type B references type A. A field in A references field of type B. In such cases, we will mark the first field as recursive."}], "doc": "SchemaField to describe metadata related to dataset schema. Schema normalization rules: http://go/tms-schema"}}, "name": "fields", "doc": "Client provided a list of fields from document schema."}, {"type": ["null", {"type": "array", "items": "string"}], "name": "primaryKeys", "default": null, "doc": "Client provided list of fields that define primary keys to access record. Field order defines hierarchical espresso keys. Empty lists indicates absence of primary key access patter. Value is a SchemaField@fieldPath."}, {"type": ["null", {"type": "map", "values": {"type": "record", "name": "ForeignKeySpec", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": [{"type": "record", "name": "DatasetFieldForeignKey", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"java": {"class": "com.linkedin.pegasus2avro.common.urn.DatasetUrn"}, "type": "string", "name": "parentDataset", "doc": "dataset that stores the resource."}, {"type": {"type": "array", "items": "string"}, "name": "currentFieldPaths", "doc": "List of fields in hosting(current) SchemaMetadata that conform a foreign key. List can contain a single entry or multiple entries if several entries in hosting schema conform a foreign key in a single parent dataset."}, {"type": "string", "name": "parentField", "doc": "SchemaField@fieldPath that uniquely identify field in parent dataset that this field references."}], "doc": "For non-urn based foregin keys."}, {"type": "record", "name": "UrnForeignKey", "namespace": "com.linkedin.pegasus2avro.schema", "fields": [{"type": "string", "name": "currentFieldPath", "doc": "Field in hosting(current) SchemaMetadata."}], "doc": "If SchemaMetadata fields make any external references and references are of type com.linkeidn.common.Urn or any children, this models can be used to mark it."}], "name": "foreignKey", "doc": "Foreign key definition in metadata schema."}], "doc": "Description of a foreign key in a schema."}}], "name": "foreignKeysSpecs", "default": null, "doc": "Map captures all the references schema makes to external datasets. Map key is ForeignKeySpecName typeref."}], "doc": "SchemaMetadata to describe metadata related to store schema"}]}, "name": "aspects", "doc": "The list of metadata aspects associated with the dataset. Depending on the use case, this can either be all, or a selection, of supported aspects."}], "doc": "A metadata snapshot for a specific dataset entity."}], "name": "proposedSnapshot", "doc": "Snapshot of the proposed metadata change. Include only the aspects affected by the change in the snapshot."}, {"type": ["null"], "name": "proposedDelta", "default": null, "doc": "Delta of the proposed metadata partial update."}], "doc": "Kafka event for proposing a metadata change for an entity. A corresponding MetadataAuditEvent is emitted when the change is accepted and committed, otherwise a FailedMetadataChangeEvent will be emitted instead."}'}, 'fields': [{'fieldPath': '', 'description': '', 'nativeDataType': 'string', 'type': {'type': {'com.linkedin.pegasus2avro.schema.StringType': {}}}}]}]}), 'proposedDelta': None} has been successfully produced!
Schema not found:404
topic-trans-value

{'auditHeader': None, 'proposedSnapshot': ('com.linkedin.pegasus2avro.metadata.snapshot.DatasetSnapshot', {'urn': 'urn:li:dataset:(urn:li:dataPlatform:kafka,topic-trans,PROD)', 'aspects': [{'schemaName': 'topic-trans', 'platform': 'urn:li:dataPlatform:kafka', 'version': 1, 'created': {'time': 1591171511.279111, 'actor': 'urn:li:corpuser:'}, 'lastModified': {'time': 1591171511.279111, 'actor': 'urn:li:corpuser:'}, 'hash': '', 'platformSchema': {'documentSchema': 'None'}, 'fields': [{'fieldPath': '', 'description': '', 'nativeDataType': 'string', 'type': {'type': {'com.linkedin.pegasus2avro.schema.StringType': {}}}}]}]}), 'proposedDelta': None} has been successfully produced!
Schema not found:404
flink-test-value

{'auditHeader': None, 'proposedSnapshot': ('com.linkedin.pegasus2avro.metadata.snapshot.DatasetSnapshot', {'urn': 'urn:li:dataset:(urn:li:dataPlatform:kafka,flink-test,PROD)', 'aspects': [{'schemaName': 'flink-test', 'platform': 'urn:li:dataPlatform:kafka', 'version': 1, 'created': {'time': 1591171512.313022, 'actor': 'urn:li:corpuser:'}, 'lastModified': {'time': 1591171512.313022, 'actor': 'urn:li:corpuser:'}, 'hash': '', 'platformSchema': {'documentSchema': 'None'}, 'fields': [{'fieldPath': '', 'description': '', 'nativeDataType': 'string', 'type': {'type': {'com.linkedin.pegasus2avro.schema.StringType': {}}}}]}]}), 'proposedDelta': None} has been successfully produced!

But in docker log throws Exception. I was imported the sample data yesterday. It looks normal on the page. My topic: flink-test-value and topic-trans-value The topic of sample data is normal, but my topic seems to import failure.

schema-registry         | [2020-06-03 08:05:10,200] INFO 172.18.0.1 - - [03/Jun/2020:08:05:10 +0000] "GET /subjects/MetadataChangeEvent-value/versions/latest HTTP/1.1" 200 5823  12 (io.confluent.rest-utils.requests)
schema-registry         | [2020-06-03 08:05:10,271] INFO 172.18.0.1 - - [03/Jun/2020:08:05:10 +0000] "POST /subjects/MetadataChangeEvent-value/versions HTTP/1.1" 200 10  10 (io.confluent.rest-utils.requests)
schema-registry         | [2020-06-03 08:05:11,277] ERROR Request Failed with exception  (io.confluent.rest.exceptions.DebuggableExceptionMapper)
schema-registry         | io.confluent.rest.exceptions.RestNotFoundException: Subject not found.
schema-registry         |   at io.confluent.kafka.schemaregistry.rest.exceptions.Errors.subjectNotFoundException(Errors.java:51)
schema-registry         |   at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.validateAndGetSchema(KafkaSchemaRegistry.java:748)
schema-registry         |   at io.confluent.kafka.schemaregistry.rest.resources.SubjectVersionsResource.getSchemaByVersion(SubjectVersionsResource.java:106)
schema-registry         |   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
schema-registry         |   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
schema-registry         |   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
schema-registry         |   at java.lang.reflect.Method.invoke(Method.java:498)
schema-registry         |   at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
schema-registry         |   at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
schema-registry         |   at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
schema-registry         |   at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
schema-registry         |   at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
schema-registry         |   at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)
schema-registry         |   at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)
schema-registry         |   at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)
schema-registry         |   at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)
schema-registry         |   at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
schema-registry         |   at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
schema-registry         |   at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
schema-registry         |   at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
schema-registry         |   at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
schema-registry         |   at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
schema-registry         |   at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)
schema-registry         |   at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:679)
schema-registry         |   at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:392)
schema-registry         |   at org.glassfish.jersey.servlet.ServletContainer.serviceImpl(ServletContainer.java:385)
schema-registry         |   at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:560)
schema-registry         |   at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:501)
schema-registry         |   at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:438)
schema-registry         |   at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1591)
schema-registry         |   at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
schema-registry         |   at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
schema-registry         |   at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
schema-registry         |   at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482)
schema-registry         |   at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
schema-registry         |   at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
schema-registry         |   at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
schema-registry         |   at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
schema-registry         |   at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:173)
schema-registry         |   at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
schema-registry         |   at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
schema-registry         |   at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
schema-registry         |   at org.eclipse.jetty.server.Server.handle(Server.java:494)
schema-registry         |   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374)
schema-registry         |   at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268)
schema-registry         |   at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
schema-registry         |   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
schema-registry         |   at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
schema-registry         |   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
schema-registry         |   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
schema-registry         |   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
schema-registry         |   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
schema-registry         |   at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:367)
schema-registry         |   at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
schema-registry         |   at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)
schema-registry         |   at java.lang.Thread.run(Thread.java:748)
schema-registry         | [2020-06-03 08:05:11,281] INFO 172.18.0.1 - - [03/Jun/2020:08:05:11 +0000] "GET /subjects/topic-trans-value/versions/latest HTTP/1.1" 404 51  6 (io.confluent.rest-utils.requests)
schema-registry         | [2020-06-03 08:05:11,316] INFO 172.18.0.1 - - [03/Jun/2020:08:05:11 +0000] "POST /subjects/MetadataChangeEvent-value/versions HTTP/1.1" 200 10  19 (io.confluent.rest-utils.requests)
schema-registry         | [2020-06-03 08:05:12,303] ERROR Request Failed with exception  (io.confluent.rest.exceptions.DebuggableExceptionMapper)
schema-registry         | io.confluent.rest.exceptions.RestNotFoundException: Subject not found.
schema-registry         |   at io.confluent.kafka.schemaregistry.rest.exceptions.Errors.subjectNotFoundException(Errors.java:51)
schema-registry         |   at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.validateAndGetSchema(KafkaSchemaRegistry.java:748)
schema-registry         |   at io.confluent.kafka.schemaregistry.rest.resources.SubjectVersionsResource.getSchemaByVersion(SubjectVersionsResource.java:106)
schema-registry         |   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
schema-registry         |   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
schema-registry         |   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
schema-registry         |   at java.lang.reflect.Method.invoke(Method.java:498)
schema-registry         |   at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
schema-registry         |   at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
schema-registry         |   at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
schema-registry         |   at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
schema-registry         |   at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
schema-registry         |   at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)
schema-registry         |   at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)
schema-registry         |   at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)
schema-registry         |   at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)
schema-registry         |   at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
schema-registry         |   at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
schema-registry         |   at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
schema-registry         |   at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
schema-registry         |   at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
schema-registry         |   at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
schema-registry         |   at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)
schema-registry         |   at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:679)
schema-registry         |   at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:392)
schema-registry         |   at org.glassfish.jersey.servlet.ServletContainer.serviceImpl(ServletContainer.java:385)
schema-registry         |   at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:560)
schema-registry         |   at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:501)
schema-registry         |   at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:438)
schema-registry         |   at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1591)
schema-registry         |   at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
schema-registry         |   at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
schema-registry         |   at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
schema-registry         |   at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482)
schema-registry         |   at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
schema-registry         |   at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204)
schema-registry         |   at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
schema-registry         |   at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
schema-registry         |   at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
schema-registry         |   at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:173)
schema-registry         |   at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
schema-registry         |   at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
schema-registry         |   at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
schema-registry         |   at org.eclipse.jetty.server.Server.handle(Server.java:494)
schema-registry         |   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374)
schema-registry         |   at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268)
schema-registry         |   at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
schema-registry         |   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
schema-registry         |   at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
schema-registry         |   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
schema-registry         |   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
schema-registry         |   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
schema-registry         |   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
schema-registry         |   at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:367)
schema-registry         |   at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
schema-registry         |   at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)
schema-registry         |   at java.lang.Thread.run(Thread.java:748)
schema-registry         | [2020-06-03 08:05:12,317] INFO 172.18.0.1 - - [03/Jun/2020:08:05:12 +0000] "GET /subjects/flink-test-value/versions/latest HTTP/1.1" 404 51  16 (io.confluent.rest-utils.requests)
schema-registry         | [2020-06-03 08:05:12,497] INFO 172.18.0.1 - - [03/Jun/2020:08:05:12 +0000] "POST /subjects/MetadataChangeEvent-value/versions HTTP/1.1" 200 10  159 (io.confluent.rest-utils.requests)
casperfrx commented 4 years ago 
You also need to build the mxe-schemas module as below.
./gradlew :metadata-events:mxe-schemas:build

Build the schema project as well. It contains the missing schema. See the README of it.