Improve start_ckan_development to run docker-entrypoint.d also from extensions

ccancellieri commented 3 years ago

Hi, I'm using this project mainly to debug through docker my extensions. One of my extensions (terriajs) requires to run some paster scripts (I'm on 2.8) to configure the ini file adding plugin specific plugin configurations.

I've seen the patch approach and the current way you manage plugins and I would prefer to transparently add scripts to the setup of my plugin in the same clean way you are doing with src_extensions plugin folder: if it's present then configure the plugin.

This will help to modularize not having to set specific variables to env and then pass them to the scripts etc.

So I produced this small patch which will use the same approach you are using for the main configuration where you run all the python and scripts file found into /docker-entrypoint.d

Recursively this can be applied to any plugin found into the folder src_extensions.

The result is that if you add that folder (or maybe something with different name not docker related?) the start_ckan_development script will load also them as it is doing for the root only.

In this way each plugin is also able to auto configure it'self not having to specify special variables or changing this project scripts anymore.

Note I'm over 2.8 so I'm providing a first draft of this fix only for that but I can port to other versions as well.

ccancellieri commented 3 years ago

Bad idea, I'll use a script in addition to my python.py leveraging on what already exist sorry for noise

ccancellieri commented 3 years ago

Actually I feel a bit confused, I see a good mechanism (/src/ mounted volume), easy to use for dev but the I see: patch/ docker-entrypoint.d/

Which are kind of overlaps since in a sh script (into docker-entrypoint.d) you can use patch by yourself....

and no way to run scripts over plugin provided into /src

maybe it's not a bad Idea to discuss if you are open to change/optimize above mentioned mechanisms

amercader commented 3 years ago

@ccancellieri I'm confused about your use case. The patch and docker-entrypoint.d folders have two different goals:

patch is meant to include .diff files that will be applied to the sources of ckan or extensions in the container
docker-entrypoint.d is meant to include scripts that will be run when the container starts

Of course docker-entrypoint scripts can call patch directly (or any other command really) but patch is added as a convenience mechanism.

So for your original need ("run some paster scripts (I'm on 2.8) to configure the ini file adding plugin specific plugin configurations.") wouldn't docker-entrypoint cover your needs?

ccancellieri commented 3 years ago

Thank you Adrià for your reply, in fact, patch is convenient! I'm including it in my scripts as well, ok good to know

My goal was: being able to have the plugin folder I'm patching or running my script on (absolute path) available into my scripts under docker-entrypoint.d. probably at build time I've to save it into the script. My question is: Is there a variable into the bash environment available which can help to bind a script to the matching (by name?) the corresponding plugin? This is why I was adding the extension, you only run: python setup.py develop... Actually before of that you also 'cd' to that path, which would be great to have (at start-time also).

One of my goal is also avoid any additional dockerization, leaving to the my (already existing) clean, setup and startup scripts to do the job live at service-boot (mainly for test/dev purpose) I'm trying to setup a comfortable and portable development environment to better support our multiple plugins development for the migration (from py2.7/ckan2.8 to py3/ckan2,9).

I see advantages but then I was thinking also that it has not so much value if compared to the additional given cost (maintenance), so I'm currently setting up the plugin using python and relative paths (it's in fact aware of the context where the specific plugin is running)

thanks for your time

Il giorno lun 21 giu 2021 alle ore 12:54 Adrià Mercader < @.***> ha scritto:

@ccancellieri https://github.com/ccancellieri I'm confused about your use case. The patch and docker-entrypoint.d folders have two different goals:

patch is meant to include .diff files that will be applied to the sources of ckan or extensions in the container

docker-entrypoint.d is meant to include scripts that will be run when the container starts

Of course docker-entrypoint scripts can call patch directly (or any other command really) but patch is added as a convenience mechanism.

So for your original need ("run some paster scripts (I'm on 2.8) to configure the ini file adding plugin specific plugin configurations.") wouldn't docker-entrypoint cover your needs?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/okfn/docker-ckan/issues/74#issuecomment-864936896, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG5WW34IWLWYL3FDDASNG3TT4K53ANCNFSM4676RJJQ .

-- Mr. Carlo Cancellieri skype: ccancellieri Twitter: @cancellieric LinkedIn: http://it.linkedin.com/in/ccancellieri/

ccancellieri commented 3 years ago

in short this is what I've added: ----------------- ckan-dev/2.8/setup/start_ckan_development.sh

index ad2f4b8..8679903 100755 @@ -38,6 +38,14 @@ do echo "Updating `test.ini` reference to `test-core.ini` for plugin $i" paster --plugin=ckan config-tool $i/test.ini "use = config:../../src/ckan/test-core.ini" fi +

if [ -f $i/setup.sh ];
then
cd $i
echo "Found setup.sh file in $i"
. ./setup.sh
cd $APP_DIR
fi fi done

Which will run my setup.sh script located into the plugin (in addition to setup.py) FROM the plugin directory (so I'm able to easily locate accessory files to use in my setup.sh script).

This is a push approach: the plugin once it's in the src/ folder will be able to self install

Instead of adding a plugin into docker.entrypoint.d which will run out of the context of the plugin and is more useful for general purpose customizations.

Thank you. C.

Il giorno lun 21 giu 2021 alle ore 22:07 carlo cancellieri < @.***> ha scritto:

Thank you Adrià for your reply, in fact, patch is convenient! I'm including it in my scripts as well, ok good to know

My goal was: being able to have the plugin folder I'm patching or running my script on (absolute path) available into my scripts under docker-entrypoint.d. probably at build time I've to save it into the script. My question is: Is there a variable into the bash environment available which can help to bind a script to the matching (by name?) the corresponding plugin? This is why I was adding the extension, you only run: python setup.py develop... Actually before of that you also 'cd' to that path, which would be great to have (at start-time also).

One of my goal is also avoid any additional dockerization, leaving to the my (already existing) clean, setup and startup scripts to do the job live at service-boot (mainly for test/dev purpose) I'm trying to setup a comfortable and portable development environment to better support our multiple plugins development for the migration (from py2.7/ckan2.8 to py3/ckan2,9).

I see advantages but then I was thinking also that it has not so much value if compared to the additional given cost (maintenance), so I'm currently setting up the plugin using python and relative paths (it's in fact aware of the context where the specific plugin is running)

thanks for your time

Il giorno lun 21 giu 2021 alle ore 12:54 Adrià Mercader < @.***> ha scritto:

@ccancellieri https://github.com/ccancellieri I'm confused about your use case. The patch and docker-entrypoint.d folders have two different goals:

patch is meant to include .diff files that will be applied to the sources of ckan or extensions in the container

docker-entrypoint.d is meant to include scripts that will be run when the container starts

Of course docker-entrypoint scripts can call patch directly (or any other command really) but patch is added as a convenience mechanism.

So for your original need ("run some paster scripts (I'm on 2.8) to configure the ini file adding plugin specific plugin configurations.") wouldn't docker-entrypoint cover your needs?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/okfn/docker-ckan/issues/74#issuecomment-864936896, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG5WW34IWLWYL3FDDASNG3TT4K53ANCNFSM4676RJJQ .

-- Mr. Carlo Cancellieri skype: ccancellieri Twitter: @cancellieric LinkedIn: http://it.linkedin.com/in/ccancellieri/

-- Mr. Carlo Cancellieri skype: ccancellieri Twitter: @cancellieric LinkedIn: http://it.linkedin.com/in/ccancellieri/

ccancellieri commented 2 years ago

FYI: I've reverted my suggestion switching using your approach. I'm using ckan/docker.entrypoint.d/spatial.sh

cd ${APP_DIR}/src_extensions/ckanext-spatial/
...

okfn / docker-ckan

Improve start_ckan_development to run docker-entrypoint.d also from extensions #74

in short this is what I've added: ----------------- ckan-dev/2.8/setup/start_ckan_development.sh

fi fi done