I'm having an issue regarding the building of python-based components in containers.
First of all let me say that Im trying to pack a set of components in the same image, which I think is not a expected behaviour (correct me if im wrong). The idea is to have 1 docker image with a set of components shared between different pipelines. I dont want to maintain 8/9 images, so the idea is that this generic_components are not going to be actively changing all the time so we can have only 1 image with all of these components
So we have this src/generic_components folder with this kind of structure:
src/generic_components/
|-- component_metadata/
|--- download_from_azure.yaml
|--- download_from_ftp.yaml
|--- visualize_dataset.yaml
|-- download/
|-- download_from_azure/
|--- download_from_azure_comp.py
|--- test/
|--- download_from_ftp/
|--- download_from_ftp_comp.py
|--- test/
|-- visualization/
|--- visualize_dataset/
|--- visualize_dataset_comp.py
|--- test/
|-- utils/
|--- modules that are used in the _comp files
All of them use the same target_image and the same base_image.
So I'm using:
And everything runs fine. Files are copied, yamls are generated and the image is created and pushed but, when use in a pipeline, it fails in the execution because in the generated kfp_config.ini I have this:
Hello there!
I'm having an issue regarding the building of python-based components in containers. First of all let me say that Im trying to pack a set of components in the same image, which I think is not a expected behaviour (correct me if im wrong). The idea is to have 1 docker image with a set of components shared between different pipelines. I dont want to maintain 8/9 images, so the idea is that this generic_components are not going to be actively changing all the time so we can have only 1 image with all of these components
So we have this src/generic_components folder with this kind of structure:
All of them use the same target_image and the same base_image. So I'm using:
kfp components build src/generic_components/ --component-filepattern */*/*_comp.py --push-image
And everything runs fine. Files are copied, yamls are generated and the image is created and pushed but, when use in a pipeline, it fails in the execution because in the generated kfp_config.ini I have this:
but the real path (in the docker image) is:
This is probably because Im building the image in a windows 10 enviroment but my docker-image is based on linux so:
Thank you so much!