Python (loadPythonContext) is not supported because you are using the fallback build of native extension

[MuravevAV]

I built a docker image using dev.Dockerfile. I'm successfully running a docker container and I want to use global.py file to run a dynamic model. But when I start *global.py** I get an error like this. Please tell me how to create a docker image to avoid this error?:

_Python (loadPythonContext) is not supported because you are using the fallback build of native extension. Read more: https://github.com/cube-js/cube/blob/master/packages/cubejs-backend-native/README.md#supported-architectures-and-platforms

at NativeInstance.loadPythonContext (/cubejs/packages/cubejs-backend-native/js/index.ts:403:15)
at DataSchemaCompiler.loadPythonContext (/cubejs/packages/cubejs-schema-compiler/src/compiler/DataSchemaCompiler.js:64:34)
at DataSchemaCompiler.doCompile (/cubejs/packages/cubejs-schema-compiler/src/compiler/DataSchemaCompiler.js:83:37)
at CompilerApi.getCompilers (/cubejs/packages/cubejs-server-core/src/core/CompilerApi.js:66:26)
at CompilerApi.metaConfig (/cubejs/packages/cubejs-server-core/src/core/CompilerApi.js:217:13)
at ApiGateway.meta (/cubejs/packages/cubejs-api-gateway/src/gateway.ts:516:26)
at /cubejs/packages/cubejs-api-gateway/src/gateway.ts:321:11_

[igorlukanin]

Hi @MuravevAV 👋

Could you please provide more info on your setup (platform?) and how you build the custom image?

[MuravevAV]

Hi @igorlukanin! Thank you for your help! My setup platform is Oracle Linux 7.9 , Cent OS 7 and MAC OS. I tested all the steps (below) on these three platforms and got the same error. I need Vertica_driver and I did the following steps:

1. I installed Docker desktop on my Mac
2. Then I downloaded this Code in ZIP format to my MAC from the link https://github.com/KnowItAllAus/cube.js-korowa/tree/vertica_driver
3. I unpacked this ZIP project
4. Then I launched Visual Studio Code and opened this project in Visual Studio Code
5. I opened terminal in Visual Studio Code and ran the Docker image build command: docker build -t cubejs/cube:vertica -f dev.Dockerfile ../../
6. Docker image built successfully
7. Then I checked that the image was created, the command: docker images
8. Then I started the Docker image:

   docker run -p 4000:4000\
   -v ${PWD}:/cube/conf\
   -e CUBEJS_DEV_MODE=true \
   -e CUBEJS_DB_TYPE=vertica \
   -e CUBEJS_DB_HOST=127.*****.0 \
   -e CUBEJS_DB_NAME=devdb \
   -e CUBEJS_DB_PORT=5433 \
   -e CUBEJS_DB_USER=******** \
   -e CUBEJS_DB_PASS=****** \
   cubejs/cube:vertica

9. Docker container started successfully
10. I launched internet-browser and entered the container address 127.*****.0:4000
11. The CUBE interface opened in the browser
12. In the "Data Models" tab in the "Tables" window, I saw a successful connection to my Vertiсa database - I saw vertica tables and objects
13. I selected the necessary Vertica tables and generated a data model in the "Playground" window
14. The model was successfully calculated and I received the calculation result
15. Then I wanted to use a Python script to run the dynamic model
16. I created a global.py file using TemplateContext class https://cube.dev/docs/reference/python/cube Here's my code:

from cube import TemplateContext
template = TemplateContext()

@template.function('load_data')
def load_data():
    client = MyApiClient("example.com")
    return client.load_data()

class MyApiClient:
   def __init__(self, api_url):
     self.api_url = api_url

   # mock API call
   def load_data(self):
     api_response = {
       "cubes": [
         {
           "name": "cube_from_api_with_dimensions",
           "measures": [
             { "name": "quote_pricemidpoint", "type": "sum", "sql": "quote_price_midpoint" },
           ],
           "dimensions": [
             { "name": "quotecode", "sql": "quote__quoterole__quote_code", "type": "string" },
             { "name": "calendardate", "sql": "calendar_reportdate_calendar_date_full", "type": "time" }
           ]
         }
       ]
     }
     return api_response

17. I put the global.py file in the "models" folder and the "inja_cube.yml" file in the "cubes" folder
18. Then I restarted the docker container, opened 127.*****.0:4000 and saw this error in the CUBE interface on the "Playground" tab: Python (loadPythonContext) is not supported because you are using the fallback build of native extension.
19. I thought that the (loadPythonContext) method does not see the Python libraries, but I checked the build of Docker image and there is Python3.9 and libpython
20. I decided to check the original CUBE image and ran: docker pull cubejs/cube:latest

    docker run -p 4000:4000\
    -v ${PWD}:/cube/conf\
    -e CUBEJS_DEV_MODE=true \
    cubejs/cube

21. And I saw that in the cubejs/cube:latest container (when running global.py) there is no error - loadPythonContext

I would like to get a docker image as cubejs/cube:latest (because there is no (loadPythonContext) error there) with a built-in Vertica_driver.

[igorlukanin]

@MuravevAV Thanks for such a complete list of reproduction steps, really appreciate that.

These steps strike my attention:

I need Vertica_driver I downloaded this Code in ZIP format to my MAC from the link https://github.com/KnowItAllAus/cube.js-korowa/tree/vertica_driver

I don't think that building a custom image of Cube as a whole is really necessary in this case.

Please take a look in the docs: https://cube.dev/docs/product/configuration/data-sources#third-party-drivers

Vertica driver is published as an npm package, so you can install it inside the container and use with driver_factory: https://cube.dev/docs/reference/configuration/config#driver_factory

Alternatively, you can extend the Docker image without rebuilding it from scratch. This would be better because I can see that the fork is many releases behind the current version of Cube: https://cube.dev/docs/product/deployment/core#extend-the-docker-image

I hope this helps.

[MuravevAV]

Thank you @igorlukanin! Based on your recommendation, I did this:

1. I built a new Docker image using this Dockerfile and dockerignor:

Dockerfile:

FROM cubejs/cube:latest

COPY . .
RUN npm i @knowitall/vertica-driver

.dockerignore

node_modules
schema
cube.py
cube.js
.env
node_modules
npm-debug.log

2. Docker Image was successfully built
3. I started a docker container:

   docker run -p 4000:4000\
   -v ${PWD}:/cube/conf\
   -e CUBEJS_DEV_MODE=true \
   -e CUBEJS_DB_TYPE=vertica \
   -e CUBEJS_DB_HOST=127.*****.0 \
   -e CUBEJS_DB_NAME=devdb \
   -e CUBEJS_DB_PORT=5433 \
   -e CUBEJS_DB_USER=******** \
   -e CUBEJS_DB_PASS=****** \
   vertica:v1

4. This container started successfully
5. I opened the CUBE interface and there was an error in the "Data Model" -> "Tables" window (the error indicates that the vertical driver is not supported):

   Error: Unsupported db type: vertica
   at driverDependencies (/cube/node_modules/@cubejs-backend/server-core/src/core/DriverResolvers.ts:27:9)
   at lookupDriverClass (/cube/node_modules/@cubejs-backend/server-core/src/core/DriverResolvers.ts:39:23)
   at Function.createDriver (/cube/node_modules/@cubejs-backend/server-core/src/core/DriverResolvers.ts:70:40)
   at CubejsServerCore.resolveDriver (/cube/node_modules/@cubejs-backend/server-core/src/core/server.ts:826:31)
   at processTicksAndRejections (node:internal/process/task_queues:96:5)
   at CubejsServerCore.getDriver (/cube/node_modules/@cubejs-backend/server-core/src/core/server.ts:799:22)
   at /cube/node_modules/@cubejs-backend/server-core/src/core/DevServer.ts:108:22
   at /cube/node_modules/@cubejs-backend/server-core/src/core/DevServer.ts:78:9

   Maybe I incorrectly wrote the commands in the Dockerfile? Or I started the docker container incorrectly?

[MuravevAV]

I tried different options for building Docker images. When I build the image using the dev.Docker file, then Vertica driver works, but Python does not work (Error: Python (loadPythonContext) is not supported because you are using the fallback build of native extension.). If I build Docker image using a latest.Dockerfile, then Python works, but the Vertica driver does not work (Error: Unsupported db type: vertica). Please, help create Docker image in which both Python and the vertica driver will work. We want to make cube cluster.