search

delta-io / kafka-delta-ingest

A highly efficient daemon for streaming data from Kafka into Delta Lake
Apache License 2.0
337 stars 71 forks source link

Docker image build can't find `deltalake-core` dependency #170

Closed beanscouts closed 1 month ago

beanscouts commented 3 months ago

Following the set up instructions from main and docker build -t kdi:0.1 . -f Dockerfile.Debian fails to resolve deltalake-core dependency:

> [builder 4/4] RUN cargo install --path .:
0.200   Installing kafka-delta-ingest v0.2.0 (/usr/src/kafka-delta-ingest)
0.328     Updating crates.io index
20.97     Updating git repository `https://github.com/delta-io/delta-rs`
22.61 error: failed to compile `kafka-delta-ingest v0.2.0 (/usr/src/kafka-delta-ingest)`, intermediate artifacts can be found at `/usr/src/kafka-delta-ingest/target`
22.61
22.61 Caused by:
22.61   no matching package named `deltalake-core` found
22.61   location searched: https://github.com/delta-io/delta-rs?branch=main
22.61   required by package `kafka-delta-ingest v0.2.0 (/usr/src/kafka-delta-ingest)`
------
Dockerfile.Debian:4
--------------------
   2 |     WORKDIR /usr/src/kafka-delta-ingest
   3 |     COPY . .
   4 | >>> RUN cargo install --path .
   5 |
   6 |     FROM debian:bullseye
--------------------
ERROR: failed to solve: process "/bin/sh -c cargo install --path ." did not complete successfully: exit code: 101
rtyler commented 3 months ago

@beanscouts I have some local changes I had been meaning to push which update the main Dockerfile. I was intending to just delete Dockerfile.debian Do you have a strong reason for that, or would me cleaning up the main Dockerfile be sufficient

beanscouts commented 3 months ago

@rtyler No strong reason, no. Just following the README.md instructions to get running with this software. I'll hold out for the stashed changes - thanks!