botpress / v12

Botpress OSS ā€“ v12
https://v12.botpress.com
GNU Affero General Public License v3.0
66 stars 81 forks source link

[BUG]: AKS kubernetes deployment Linux distribution ubuntu is not supported by node-fasttext botpress 12.25.0 #1547

Open rakeshsekar1987 opened 2 years ago

rakeshsekar1987 commented 2 years ago

Describe the bug When starting the botpress language server in offline mode on AKS kubernetes pod I get the following error

11/08/2021 13:32:50.750 [NLU] Launcher ======================================== -- Mon, Nov 8 2021 7:02:50 pm | Botpress Language Server Mon, Nov 8 2021 7:02:50 pm | Version 1.1.0 Mon, Nov 8 2021 7:02:50 pm | ======================================== Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.751 [NLU] Launcher authToken: disabled (anyone can query your language server) Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.751 [NLU] Launcher adminToken: disabled (anyone can add, remove or change languages) Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.752 [NLU] Launcher limit: disabled (no protection - anyone can query without limitation) Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.752 [NLU] Launcher mode: offline (languages need to be downloaded manually from a machine with Internet access) Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.752 [NLU] Launcher Serving 100 language dimensions from /botpress/data/embeddings Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.756 [NLU] lang:service The language server may silently crash due to a lack of free memory space. Mon, Nov 8 2021 7:02:50 pm | Current usage : (31.11/31.36)Gb, Mon, Nov 8 2021 7:02:50 pm | Predicted usage : 1.09Gb. Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.756 [NLU] lang:service Found Languages: en Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.757 [NLU] lang:service Loading Embeddings for EN Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.758 [NLU] lang:api Language Server is ready at http://localhost:3100/ Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.769 [NLU] lang:service Model = "/botpress/data/embeddings/bp.en.100.bin": Linux distribution ubuntu is not supported by node-fasttext. Mon, Nov 8 2021 7:02:50 pm | STACK TRACE Mon, Nov 8 2021 7:02:50 pm | VError: Model = "/botpress/data/embeddings/bp.en.100.bin": Linux distribution ubuntu is not supported by node-fasttext. Mon, Nov 8 2021 7:02:50 pm | at /snapshot/nlu/node_modules/@botpress/nlu-engine/dist/ml/fasttext/index.js Mon, Nov 8 2021 7:02:50 pm | 11/08/2021 13:32:50.769 [NLU] lang:service [EN] Error loading language. It will be unavailable.

To Reproduce Steps to reproduce the behavior:

  1. My docker file , im using only particular embeddings image

FROM ubuntu:20.04 ADD . /botpress WORKDIR /botpress RUN apt update && apt-get install -y wget unzip
ARG VERSION RUN wget -O botpress.zip https://s3.amazonaws.com/botpress-binaries/botpress-${VERSION}-linux-x64.zip && \ unzip botpress.zip RUN apt update && \ apt install -y ca-certificates && \ update-ca-certificates && \ wget -O duckling https://s3.amazonaws.com/botpress-binaries/duckling-example-exe && \ chmod +x duckling && \ chmod +x bp && \ chgrp -R 0 /botpress && \ chmod -R g=u /botpress && \ apt install -y tzdata && \ ln -fs /usr/share/zoneinfo/America/New_York /etc/localtime && \ dpkg-reconfigure --frontend noninteractive tzdata && \ ./bp extract ENV BP_MODULE_NLU_DUCKLINGURL=http://localhost:8000 ENV BP_IS_DOCKER=false ENV LANG=C.UTF-8 ARG RUNUSER RUN chown -R ${RUNUSER}:${RUNUSER} /botpress EXPOSE 3100 CMD ./bp lang --offline --dim 100 --langDir /botpress/data/embeddings Expected behavior But when I run the same docker image in local it works perfectly fine image

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

Additional context Add any other context about the problem here.

franklevasseur commented 2 years ago

Hi @rakeshsekar1987

Can you please do the following steps:

  1. fetch https://github.com/botpress/nlu/releases/download/v1.0.0-rc.4/nlu-v1_0_0_rc_4-linux-x64
  2. run NODE_FASTTEXT_VERBOSE=1 ./nlu-v1_0_0_rc_4-linux-x64 lang --dim 100
  3. take a screenshot or copy the logs and paste them in the current issue thread so I can see what's going on
franklevasseur commented 2 years ago

Hi @kamranahmedse

any news on this? please keep me in the loop

kamranahmedse commented 2 years ago
franklevasseur commented 2 years ago

OMG, I'm so sorry @kamranahmedse !! I meant to tag @rakeshsekar1987 ...

You must excuse my poor dyslexic brain šŸ˜