chdb-io / chdb

chDB is an in-process OLAP SQL Engine 🚀 powered by ClickHouse
https://clickhouse.com/docs/en/chdb
Apache License 2.0
2.03k stars 72 forks source link

cannot import in amazon2 linux #90

Closed blackrez closed 1 year ago

blackrez commented 1 year ago

Describe what's wrong

I'm trying to use chdb on AWS Lambda, I use Docker for testing and building the application. And when I import chdb, it crashed due to clickhouse can't access to /etc/localtime. The file exists, I tried to change the timezone, it failed too.

How to reproduce

(base) ➜  ~ docker run -d --name chdb --rm public.ecr.aws/lambda/python:3.11
f500dc48b67d268020b2034d92b84f3af7505edcfbce2dc84cb955b296efc13c
docker exec -it chdb /bin/bash
bash-4.2# pip install chdb
Collecting chdb
  Downloading chdb-0.11.5-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (90.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.0/90.0 MB 3.5 MB/s eta 0:00:00
Installing collected packages: chdb
Successfully installed chdb-0.11.5
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.1.2 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
Python 3.11.4 (main, Jul 31 2023, 08:41:59) [GCC 7.3.1 20180712 (Red Hat 7.3.1-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import chdb
Poco::Exception. Code: 1000, e.code() = 0, Exception: Cannot load time zone /etc/localtime (version 23.6.1.1)
lmangani commented 1 year ago

Hello @blackrez interesting. We have a working lambda container running successfully. Could you check out the following links and see if you perhaps can spot a difference between your method and the working one?

@alijared might have some suggestions 👁️

auxten commented 1 year ago

Can you help cat /etc/localtime. It seems the timezone in this file is not a valid one. You can also try mv /etc/localtime /etc/localtime.bak to see if it works.

blackrez commented 1 year ago

For more information about this https://man7.org/linux/man-pages/man5/localtime.5.html. This is a binary file. I just tested with clickhouse local and it didn't work too. I will open a bug in clickhouse.

auxten commented 1 year ago

For more information about this https://man7.org/linux/man-pages/man5/localtime.5.html. This is a binary file. I just tested with clickhouse local and it didn't work too. I will open a bug in clickhouse.

Yes, but cat /etc/localtime will also show many useful info, as the the content on my host shows like this:

(chdb311) auxten/chdb [main●] » cat /etc/localtime
TZif�����y��Y^��    �p�ӽ����|@�;>�Ӌ{��B���E"�L���<��fp���A|��R i�� ~��!I}�"g� #)_�$G� %|&'e &�^(G (�@~�p�CDTCSTTZif2
        ����~6C)�������������y������Y^������    �p�����ӽ������������|@�����;>�����Ӌ{������B�������E"�����L�������<������fp�����������A|��R i�� ~��!I}�"g� #)_�$G� %|&'e &�^(G (�@q�~�pLMTCDTCST
CST-8
blackrez commented 1 year ago

Ok, by default I have this

bash-4.2# ls -alh /etc/localtime
lrwxrwxrwx 1 root root 23 Jul  3 14:06 /etc/localtime -> /usr/share/zoneinfo/UTC
bash-4.2# cat /etc/localtime
TZif2UTCTZif2�UTC
UTC0
nmreadelf commented 1 year ago

hello, you can set TZ environment, on my case:

bash-4.2# TZ=US/Central python -m chdb 'select version()'
"23.6.1.1"
nmreadelf commented 1 year ago

i checked TZ environment in public.ecr.aws/lambda/python:3.11.

bash-4.2# echo $TZ
:/etc/localtime

the normal TZ environment value format should be {country}/{city}. e.g: TZ=US/Central

it seens TZ environment value problem, another solution is unset TZ environment. my case :

bash-4.2# unset TZ
bash-4.2# python -m chdb 'select version()'
"23.6.1.1"
lmangani commented 1 year ago

@blackrez can we mark this as resolved with the suggestions, or are you still experiencing issues?

blackrez commented 1 year ago

Hello, it is fixed.