search

dremio / iceland

Apache License 2.0
1 stars 4 forks source link

Add `injection` iceberg usecase #2

Open jbonofre opened 9 months ago

jbonofre commented 9 months ago

This first PR create the skeleton and add a first injection use case.

This injection use case:

  1. provides a script to fetch GDELT data as CSV file, ready to be parsed and inserted in the Iceberg table
  2. CreateTable creates a Iceberg table to store GDELT events
  3. DataInjection parses the GDELT CSV file and insert data in the Iceberg table
  4. Analytics provides the first simple queries

NB: when merged, I will create a blog post based on this PR.

jbonofre commented 9 months ago

@ajantha-bhat can you please take a look ?

ajantha-bhat commented 9 months ago

I just reviewed high level. I will install and try it out later on.