Closed Sanjanare closed 5 years ago
Hi @Sanjanare, I was trying this against my own private gitlab instance and I was able to commit & push changes.
I guess, maybe you left out the branch of remote git repository?
ENV FLOW_PROVIDER=git \
GIT_REMOTE_URL=https://*************/nifi-registry-git.git \
`GIT_CHECKOUT_BRANCH= \
FLOW_PROVIDER_GIT_REMOTE_ACCESS_USER=amx-bot \
FLOW_PROVIDER_GIT_REMOTE_ACCESS_PASSWORD=******** \
GIT_CONFIG_USER_NAME=amx-bot \
GIT_CONFIG_USER_EMAIL=amx-bot@*******
This value:
`GIT_CHECKOUT_BRANCH= \
Should not be empty. Try master
branch.
Full example:
FROM michalklempa/nifi-registry
ENV FLOW_PROVIDER=git \
GIT_REMOTE_URL=https://*************/nifi-registry-git.git \
GIT_CHECKOUT_BRANCH=master \
FLOW_PROVIDER_GIT_REMOTE_ACCESS_USER=amx-bot \
FLOW_PROVIDER_GIT_REMOTE_ACCESS_PASSWORD=******** \
GIT_CONFIG_USER_NAME=amx-bot \
GIT_CONFIG_USER_EMAIL=amx-bot@*******
And you do not have to create child Dockerfile for this to try.
Just use docker run
command:
docker run --name nifi-registry \
-p 18080:18080 \
-e 'FLOW_PROVIDER=git' \
-e 'GIT_REMOTE_URL=https://*************/nifi-registry-git.git' \
-e 'GIT_CHECKOUT_BRANCH=master' \
-e 'FLOW_PROVIDER_GIT_FLOW_STORAGE_DIRECTORY=/opt/nifi-registry/flow-storage-git' \
-e 'FLOW_PROVIDER_GIT_REMOTE_TO_PUSH=origin' \
-e 'FLOW_PROVIDER_GIT_REMOTE_ACCESS_USER=amx-bot ' \
-e 'FLOW_PROVIDER_GIT_REMOTE_ACCESS_PASSWORD=********' \
-e 'GIT_CONFIG_USER_NAME=amx-bot' \
-e 'GIT_CONFIG_USER_EMAIL=amx-bot@********'' \
-d \
michalklempa/nifi-registry:latest
Give it a try and let me know.
@michalklempa i will try and let you know thank you :) . I am creating a child Dockerfile since it is in ci/cd process and we are using helm files to deploy
@michalklempa when i add GIT_CHECKOUT_BRANCH=master it gives out below error
docker run -p 18080:18080 nifi-registry:latest
FLOW_PROVIDER_GIT_REMOTE_ACCESS_PASSWORD is set, trying to set git credential helper for HTTPS password
git config --global credential.${GIT_REMOTE_URL}.helper '!f() { sleep 1; echo -e "username=${FLOW_PROVIDER_GIT_REMOTE_ACCESS_USER}\npassword=*****"; }; f'
Found git remote: https://**********/nifi-registry-git.git, cloning into: /opt/nifi-registry/flow-storage, with remote: master and branch:
git clone -o $FLOW_PROVIDER_GIT_REMOTE_TO_PUSH -b $GIT_CHECKOUT_BRANCH $GIT_REMOTE_URL $FLOW_PROVIDER_GIT_FLOW_STORAGE_DIRECTORY
fatal: Too many arguments.
usage: git clone [<options>] [--] <repo> [<dir>]
if i remove master and try it works fine and initializes nifi-registry
@michalklempa nvm it worked after adding all the env variables like below:
FROM michalklempa/nifi-registry
ENV FLOW_PROVIDER=git \
GIT_REMOTE_URL=https://*********/nifi-registry-git.git \
GIT_CHECKOUT_BRANCH=master \
FLOW_PROVIDER_GIT_FLOW_STORAGE_DIRECTORY=/opt/nifi-registry/flow-storage-git \
FLOW_PROVIDER_GIT_REMOTE_TO_PUSH=origin \
FLOW_PROVIDER_GIT_REMOTE_ACCESS_USER=amx-bot \
FLOW_PROVIDER_GIT_REMOTE_ACCESS_PASSWORD=****** \
GIT_CONFIG_USER_NAME=amx-bot \
GIT_CONFIG_USER_EMAIL=amx-bot@*******
@michalklempa i am trying to configure ldap properties in nifi registry and i see authorizations.xml is by default configured to use file-user-group-provider but i want to configure LdapUserGroupProvider , i could not find the variables for LdapUserGroupProvider in the documentation https://github.com/michalklempa/docker-nifi-registry/#standalone-instance-ldap can you tell me how to configure those ?
Thanks you so much for you help in before task
@Sanjanare LdapUserGroupProvider - you point to changing conf/authorizers.xml
?
If so, I am sorry to admit, that these configuration options are not inserted from environmental variables. There are too many options in my opinion, to handle them using env vars.
If you can provide me an example of kind-of working authorizers.xml
file, I can maybe think of variable names and incorporate the changes needed. You can reach me out using email michal.klempa@gmail.com. Also, if you are willing to contribute such a change, lets go with pull request.
In the meantime, in k8s, I think there is an option to provide mounted volume and/or ConfigMap (https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) - so you could be able to provide the authorizers.xml
as a whole file and omit any changes to this file by Docker image.
To do so, mount your authorizers.xml
file into k8s pod on location:
/opt/nifi-registry/nifi-registry-0.4.0/conf/authorizers.xml
And avoid setting INITIAL_ADMIN_IDENTITY
environment variable (as this decides, whether start.sh
tries to render template into the file):
Snippet from start.sh
:
if [[ -n "${INITIAL_ADMIN_IDENTITY}" ]]; then
/usr/local/bin/dockerize -template ${PROJECT_TEMPLATE_DIR}/authorizers.xml.gotemplate:${PROJECT_CONF_DIR}/authorizers.xml
fi
Virtually any configuration file can be provided as ConfigMap and/or mounted file/volume. See https://github.com/michalklempa/docker-nifi-registry/blob/develop/sh/start.sh#L74 There you find consecutive if-statements - each for one configuration file. Just avoid setting the env vars which trigger templating and you can provide own configuration files.
See this example for mounting: https://github.com/michalklempa/docker-nifi-registry#providing-configuration-by-mounting-files Same can be achieved in k8s.
Hope this helps.
@michalklempa we didnot make changes to the authorizers.xml file , we dropped from this plan , but will send u PR if we add anything . ,thank you so much :)
@michalklempa since registry is working now , i will get authorizers.xml set of Users List, i will reach out to you in email after i setup . Thanks for ur help
@Sanjanare may we close this issue?
@michalklempa Hi , i have tried the dockerfile for gitlab persistance provider , i am running it on kubernetes
Here is the docker image i am using
i get below logs after running the dockerfile
And the nifi-registry starts initializing.
When i give the http://localhost:18080 in nifi to connect to registry, it is able to see the buckets in nifi-registry , after adding the version control to the nifi i can see the commit changes in the nifi-registry but i am unable to see this changes in the gitlab repo.
it is am empty repository and even after commiting changes to nifi-regsitry it is still empty as i understand i should be able to see the commits in gitlab repo ? please let me know if i am missing anything . Thanks