search

bitsondatadev / trino-getting-started

Apache License 2.0
234 stars 97 forks source link

Test Hive Metastore 4.x and see if we can archive my HMS image #43

Open bitsondatadev opened 3 months ago

bitsondatadev commented 3 months ago 
          Looks like this fixed the issue.

https://github.com/bitsondatadev/trino-getting-started/issues/38#issuecomment-1925347891

Any reason why this wasn't used instead of custom HMS + psql?

There were some oddities about earlier HMS images in 3.x. If the latest 4.x metastore has those worked out then I would love to get rid of my HMS Docker image.

I'll open a new Issue to remind myself to look at that later.

Originally posted by @bitsondatadev in https://github.com/bitsondatadev/trino-getting-started/issues/38#issuecomment-2025768063

bitsondatadev commented 3 months ago

The only recent thing I remember we added quickly to this was the ability to run on an M1 processor, but I'm hopeful this is resolved in the latest.

metadaddy commented 3 weeks ago

Hi @bitsondatadev - this works for me in the trino-b2 tutorial:

  hive-metastore:
    image: 'apache/hive:4.0.0'
    hostname: hive-metastore
    ports:
      - '9083:9083' # Metastore Thrift
    volumes:
      - ./conf/core-site.xml:/opt/hadoop/etc/hadoop/core-site.xml:ro
      - ./conf/metastore-site.xml:/opt/apache-hive-metastore-4.0.0-bin/conf/metastore-site.xml:ro
    environment:
      SERVICE_NAME: metastore
      METASTORE_DB_HOSTNAME: mariadb
      HIVE_AUX_JARS_PATH: /opt/hadoop/share/hadoop/tools/lib/hadoop-aws-3.3.6.jar:/opt/hadoop/share/hadoop/tools/lib/aws-java-sdk-bundle-1.12.367.jar
    depends_on:
      - mariadb
    networks:
      - trino-network

There may be more optimal ways to do this, but a few hours of trial and error led me here.

metadaddy commented 3 weeks ago

See https://github.com/bitsondatadev/trino-getting-started/pull/50