hackingthemarkets / tradekit

a collection of open source server components and Python libraries for financial data projects and automated trading
Apache License 2.0
637 stars 250 forks source link

Docker-compose up failed on mac m1 BigSur #13

Open arenouf14 opened 2 years ago

arenouf14 commented 2 years ago

Run into a problem on TA-lib and h5py install while running docker-compose up on a mac M1. Error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz && tar -xvzf ta-lib-0.4.0-src.tar.gz && cd ta-lib/ && ./configure --prefix=/usr && make && make install]: exit code: 1

Seems the problem comes from the architecture that is used by default when running from mac m1. Solved by adding explicitly the platform in the Dockerfile (first line): FROM --platform=linux/amd64 python:3.8 But have a warning explaining I ll have poor performances.

I tried FROM --platform=linux/arm64 python:3.8 and it works with good performances but I removed h5py because of a similar error appearing (I think I need to install from source for this library probably but will do it if I need).

---- GIT DIFF

diff --git a/Dockerfile b/Dockerfile
index 7c75123..5c6d041 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,12 +1,12 @@
-FROM python:3.8
+FROM --platform=linux/arm64 python:3.8

-MAINTAINER Part Time Larry "parttimelarry@gmail.com"
+MAINTAINER cryptonouf "cryptonouf@protonmail.com"

 ADD . /app

 WORKDIR /app

-RUN apt-get update && apt-get install -y build-essential redis-server sqlite3 \
+RUN apt-get update && apt-get install -y build-essential automake redis-server sqlite3 \
     curl software-properties-common

 # install nodejs and wscat websocket client
@@ -18,6 +18,7 @@ RUN apt-get install -y nodejs && npm install wscat
 RUN wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz && \
   tar -xvzf ta-lib-0.4.0-src.tar.gz && \
   cd ta-lib/ && \
+  cp /usr/share/automake-1.16/config.guess . && \
   ./configure --prefix=/usr && \
   make && \
   make install
diff --git a/requirements.txt b/requirements.txt
index 08ae490..e02df4f 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -72,7 +72,7 @@ websockets

 # other data stores, formats
 redis
-h5py
+#h5py

 # utilities
 beautifulsoup4
arenouf14 commented 2 years ago

I added compilation from source for HDF5 in order to put back h5py in the requirements.txt file. Run in the same problems... I tested first to remove it and run in the container the pip install command after installing hdf5 from source and it worked. But when I put the changes in the Dockerfile as hereafter it breaks the same as with TA-lib before.

Content of my patch file:

diff --git a/Dockerfile b/Dockerfile
index 7c75123..7e5166d 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,12 +1,12 @@
-FROM python:3.8
+FROM --platform=linux/arm64 python:3.8

-MAINTAINER Part Time Larry "parttimelarry@gmail.com"
+MAINTAINER cryptonouf "cryptonouf@protonmail.com"

 ADD . /app

 WORKDIR /app

-RUN apt-get update && apt-get install -y build-essential redis-server sqlite3 \
+RUN apt-get update && apt-get install -y build-essential automake redis-server sqlite3 \
     curl software-properties-common

 # install nodejs and wscat websocket client
@@ -18,10 +18,19 @@ RUN apt-get install -y nodejs && npm install wscat
 RUN wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz && \
   tar -xvzf ta-lib-0.4.0-src.tar.gz && \
   cd ta-lib/ && \
+  cp /usr/share/automake-1.16/config.guess . && \
   ./configure --prefix=/usr && \
   make && \
   make install

+# download and build hdf5
+RUN wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.1/src/hdf5-1.12.1.tar.gz && \
+  tar xvfz hdf5-1.12.1.tar.gz && \
+  cd hdf5-1.12.1 && \
+  ./configure --prefix=/usr/local/ && \
+  make && \
+  make install
+
 RUN rm -R ta-lib ta-lib-0.4.0-src.tar.gz

 # install popular Python packages