Rename default value of HAYHOOKS_ADDITIONAL_PYTHONPATH

[alex-stoica]

Currently, it is opt/custom-components. I recommend renaming it to opt/custom_components for improved consistency with Python naming conventions. This change will facilitate more intuitive imports, such as: from custom_components.some_custom_component import MyCustomComponent

[masci]

Hi @alex-stoica do you mind opening a quick PR? I'll review and merge!

[karbasia]

@masci @alex-stoica is there anything special needed to import custom components?

My HAYHOOKS_ADDITIONAL_PYTHONPATH is set to /opt/custom_components and I see the files on my container. When attempting to import the pipeline, I am greeted with this error message:

Error deploying pipeline: Unable to parse Haystack Pipeline ingest_pipeline: Component 'custom_components.ocr_pdf.OCRPDFToDocument' not imported.

My pipelines run fine locally (using the same custom_components folder in my local environment) and pipelines without custom components are also correctly imported.

I have a simple docker compose too:

services:
  hayhooks:
    container_name: hayhooks
    image: hayhooks:local
    ports:
      - 1416:1416
    volumes:
      - ./pipelines:/opt/pipelines
      - ./custom_components:/opt/custom_components
      - ./data:/data
    healthcheck:
      test: ["CMD-SHELL", "curl --silent --fail localhost:1416/status || exit 1"]
      interval: 120s
      timeout: 30s
      retries: 3
    env_file:
      - .env
networks:
  default:
    name: local

[karbasia]

Ok, got it. I had to remove "custom_components" from my yaml. The type simply became type: ocr_pdf.OCRPDFToDocument

[alex-stoica]

@karbasia what I did to work is

    environment:
      - PYTHONPATH=/opt

In my docker file + maintained custom_components in my pipeline yaml A working example I used is in https://github.com/deepset-ai/hayhooks/pull/27 (with the only difference that - HAYHOOKS_ADDITIONAL_PYTHONPATH=/opt/custom_components is no longer necessary as this PR already was merged to the main branch

Anyway, I guess that removing custom_components works too

[karbasia]

Thanks for the info! I ended up revisiting my configuration and updating my custom_components mounts so that I can keep the custom_components in my imports and yaml.

Your solution is much cleaner though, so I'll make the necessary adjustments.