search

fititnt / uwazi-docker

Dockerized version of Uwazi (“openness" in Swahili). HURIDOCS designed Uwazi to make human rights information more open and accessible to the defenders who need it.
The Unlicense
11 stars 4 forks source link

Uwazi 1.4 is out #36

Closed vasyugan closed 5 years ago

vasyugan commented 5 years ago

Huridocs just released Uwazi 1.4, with support for managing translations, a feature I was really looking forward to.

fititnt commented 5 years ago

Did it worked for you (with at least less bugs than before) on dockerized uwazi? Just confirm this (I will also act here as reviewer, so we can have at least two humans double checking

vasyugan commented 5 years ago

Am Donnerstag, den 29.11.2018, 19:19 -0800 schrieb Emerson Rocha Luiz:

Did it worked for you (with at least less bugs than before) on dockerized uwazi? Just confirm this (I will also act here as reviewer, so we can have at least two humans double checking

As far as I am concerned, there was little difference, just that it finally has the possibility to manage languages that I desperately needed. There was an issue with credentials of one account suddenly not working anymore, but I am not clear where that came from and whether this was related to the upgrade. What this make me thinking about is what mechanism we could have to upgrade a dockerized Uwazi, because what I did was probably not very elegant, yet I did it to make sure existing data is preserved.

  1. Ran docker-compose down
  2. docker image rm (name of the uwazi-docker image)
  3. Move current uwazi-docker dir (which also contains all user data in subdirs) to uwazi-docker.old
  4. Checked out the uwazi-docker code from github
  5. Changed the release number from 1.3. to 1.4 in the Dockerfile
  6. ran docker image build .
  7. moved uwazi-docker out of the way
  8. moved uwazi-docker.old to uwazi-docker
  9. docker-compose up -d
    1. docker exec -it [name of uwazi docker container] bash
    2. yarn migrate
    3. yarn reindex
    4. docker-compose down
    5. docker-compose up -d I guess some of the steps are redundant but I wanted to make absolutely sure that I have a clean upgrade and that none of the existing user data is lostAlso I am conscious of the fact that I ran yarn migrate and yarn reindex at a stage where uwazi is already running, and I am less than sure that is is meant to be so and I am not clear about what possible side effects are

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c5 5493e4bb","name":"GitHub"},"entity":{"external_key":"github/fititnt/u wazi-docker","title":"fititnt/uwazi-docker","subtitle":"GitHub repository","main_image_url":" https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":" https://github.com/fititnt/uwazi-docker"}},"updates":{"snippets":[{"icon":"PERSON","message":"@fititnt in #36: Did it worked for you (with at least less bugs than before) on dockerized uwazi? Just confirm this (I will also act here as reviewer, so we can have at least two humans double checking"}],"action":{"name":"View Pull Request","url":" https://github.com/fititnt/uwazi-docker/pull/36#issuecomment-443077371 "}}} [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": " https://github.com/fititnt/uwazi-docker/pull/36#issuecomment-443077371 ", "url": " https://github.com/fititnt/uwazi-docker/pull/36#issuecomment-443077371 ", "name": "View Pull Request" }, "description": "View this Pull Request on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } }, { "@type": "MessageCard", "@context": "http://schema.org/extensions", "hideOriginalBody": "false", "originator": "AF6C5A86-E920-430C-9C59-A73278B5EFEB", "title": "Re: [fititnt/uwazi-docker] Uwazi 1.4 is out (#36)", "sections": [ { "text": "", "activityTitle": "Emerson Rocha Luiz", "activityImage": " https://assets-cdn.github.com/images/email/message_cards/avatar.png", "activitySubtitle": "@fititnt", "facts": [

] } ], "potentialAction": [ { "name": "Add a comment", "@type": "ActionCard", "inputs": [ { "isMultiLine": true, "@type": "TextInput", "id": "IssueComment", "isRequired": false } ], "actions": [ { "name": "Comment", "@type": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"IssueComment\",\n\"repositoryFullName\": \"fititnt/uwazi- docker\",\n\"issueId\": 36,\n\"IssueComment\": \"{{IssueComment.value}}\"\n}" } ] }, { "name": "Close pull request", "@type": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"PullRequestClose\",\n\"repositoryFullName\": \"fititnt/uwazi- docker\",\n\"pullRequestId\": 36\n}" }, { "targets": [ { "os": "default", "uri": " https://github.com/fititnt/uwazi-docker/pull/36#issuecomment-443077371 " } ], "@type": "OpenUri", "name": "View on GitHub" }, { "name": "Unsubscribe", "@type": "HttpPOST", "target": "https://api.github.com", "body": "{\n\"commandName\": \"MuteNotification\",\n\"threadId\": 417996347\n}" } ], "themeColor": "26292E" } ]

fititnt commented 5 years ago

Yes, I fully agree that the update process, for now, is not automated with uwazi docker, so the person have to login on the new uwazi and run the commands that are need to update the uwazi without docker.

The issue that mentions part of this is #28, so if someone else goes there, also will see that you here documented at least one process of how to update the databases too

fititnt commented 5 years ago

@vasyugan I'm doing a clean install on your actual pull request. Give me some time to check if no hard errors show UP. 

# fititnt at bravo in /alligo/code/fititnt/uwazi-docker on git:master x [3:46:44]
$ git checkout -b vasyugan-master master
M   docker-compose.yml
Switched to a new branch 'vasyugan-master'

# fititnt at bravo in /alligo/code/fititnt/uwazi-docker on git:vasyugan-master x [3:47:10]
$ git pull https://github.com/vasyugan/uwazi-docker.git master
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 7 (delta 2), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), done.
From https://github.com/vasyugan/uwazi-docker
 * branch            master     -> FETCH_HEAD
Merge made by the 'recursive' strategy.
 Dockerfile | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

# fititnt at bravo in /alligo/code/fititnt/uwazi-docker on git:vasyugan-master x [3:47:55]
$ docker-compose run -e IS_FIRST_RUN=true --rm uwazi # Install/Re-install from empty data

(...)
fititnt commented 5 years ago

Strange. This is one unrelated error building the docker image.

Also this remember me that (maybe in another commit or pull request, could be a good ideal force usage of mongo 3.4 also as the shell (that is only used if I remember to initiate the database the first time, but for now maybe we just accept this pull request.

(...)
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Processing triggers for sgml-base (1.29) ...
Removing intermediate container fac4b374067f
 ---> bfca57aaa1d0
Step 4/8 : RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5   && echo "deb http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.6 main" | tee /etc/apt/sources.list.d/mongodb-org-3.6.list   && apt-get update   && apt-get install -y mongodb-org-shell mongodb-org-tools   && apt-get clean && rm -rf /var/lib/apt/lists/*
 ---> Running in 5980d71ffce7
Warning: apt-key output should not be parsed (stdout is not a terminal)
Executing: /tmp/apt-key-gpghome.l1Y4lb8lwg/gpg.1.sh --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
gpg: key 58712A2291FA4AD5: public key "MongoDB 3.6 Release Signing Key <packaging@mongodb.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1
deb http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.6 main
Ign:1 http://deb.debian.org/debian stretch InRelease
Hit:2 http://security.debian.org/debian-security stretch/updates InRelease
Ign:3 http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.6 InRelease
Hit:4 http://deb.debian.org/debian stretch-updates InRelease
Get:5 http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.6 Release [2393 B]
Hit:6 http://deb.debian.org/debian stretch Release
Get:7 http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.6 Release.gpg [801 B]
Get:9 http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.6/main amd64 Packages [7721 B]
Fetched 10.9 kB in 0s (19.8 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 mongodb-org-shell : Depends: libssl1.0.0 (>= 1.0.1) but it is not installable
 mongodb-org-tools : Depends: libssl1.0.0 (>= 1.0.1) but it is not installable
E: Unable to correct problems, you have held broken packages.
ERROR: Service 'uwazi' failed to build: The command '/bin/sh -c apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5   && echo "deb http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.6 main" | tee /etc/apt/sources.list.d/mongodb-org-3.6.list   && apt-get update   && apt-get install -y mongodb-org-shell mongodb-org-tools   && apt-get clean && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
fititnt commented 5 years ago

Related https://github.com/nodejs/docker-node/issues/936#issuecomment-442371387

fititnt commented 5 years ago

For now using FROM node:8-jessie instead of FROM node:8-slim imediatly allows the build as before, but as this comment from https://github.com/nodejs/docker-node/issues/936#issuecomment-442904306 from 14 hours ago the FROM node:8-jessie-slim could work, but for now it does not build.

@vasyugan I will accept this PR, but I think we could wait a bit for if FROM node:8-jessie-slim will work or just stick with FROM node:8-jessie. But since these two are unrelated with this pull request (they break the actual version on the master too), we could just accept this PR and made these chances later.

For now you do not need to make one extra commit to change the first line with the new FROM, but if you soon make a next PR just with one of these changes, I will accept because will work.

I'm documenting this becase if someone more would like to test/use the uwazi-docker, will find these same issues (and I'm not talking just about uwazi, but pretty everyone that was using FROM node:8-slim and could have some compatibility issues.

Ah, and @vasyugan , if you ask me that is not a good idea that these things could eventally break, think that for larger docker projects, we normaly would cache the full image on docker-hub (not build every time), but for now we do not have more people here to do the full thing.

vasyugan commented 5 years ago

Strange. This is one unrelated error building the docker image.

Also this remember me that (maybe in another commit or pull request, could be a good ideal force usage of mongo 3.4 also as the shell (that is only used if I remember to initiate the database the first time,

Since the uwazi installation instructions specify that you need 3.4, why are you using 3.6 then?

fititnt commented 5 years ago

Thats another thing I forgot to test long time ago. I thought they would soon upgrade to Mongo's new version, so I also did not bother downgrade the mongo version used only on the import.

The specific version of mongo used on the database was exactly the version on the documentation.

But yest, if you also know the commands that change that part of the dockerfile to use the 3.4, you can optimize this too.

fititnt commented 5 years ago

(Also, few more minutes to test this pull request, first I was testing on the master with the FROM node:8-jessie and worked. Will do it too on this PR just to be sure)

vasyugan commented 5 years ago

(Also, few more minutes to test this pull request, first I was testing on the master with the FROM node:8-jessie and worked. Will do it too on this PR just to be sure)

Why wouldn't you want to move to stretch, as jessie by this time is really old and unsupported upstream?

fititnt commented 5 years ago

Why wouldn't you want to move to stretch, as jessie by this time is really old and unsupported upstream?

It's good for me to move and I totally would accept PRs with that change. on the #37 I just hot pached to allow it still work for some extra time and the master here still working, but there is no extra reason to stay with jessie beyond just do the full testing.

Maybe the only change need would be the reference to the mongo-db repository. But I'm a bit busy with other projects to do the full testing alone on this one, and I do not even have, at this time, one uwazi running in production to check the full changes, so this explain why I'm not also upgrading everything.

txau commented 5 years ago

@fititnt @vasyugan just to give you a heads up there is a breaking change with the elasticsearch version, now 5.6 is required (previously 5.5).

Also, if you are using the blank state you should be ok but for existing dbs running "yarn migrate" is necessary along with the other steps explained in https://github.com/huridocs/uwazi#upgrading-uwazi-and-data-migrations.

fititnt commented 5 years ago

@txau I just created one issue on the main repository at https://github.com/huridocs/uwazi/issues/2101.

See https://github.com/fititnt/uwazi-docker/blob/master/docker-entrypoint.sh.

Have some comands that could be easily parsed by bash (the ones who returno -1,0, +1, or some constants) this could make more easy to do one of these things:

  1. warn the user, but let the person run (for example, if uwazi-docker does not have some automation to update)
  2. block the uwazi-docker from running, so the user will have to debug and will see the messages and take immediate action (I know this maybe would sound extreme, but in more automated enviroment, could be better do it than users complaining later that things are strange)
  3. uwazi-docker doing the migration on the fly.

I'm aware that this logic could take a few more released versions of uwazi to be fully tested (for example, even if the 1.5 have some draft of this, to make full testing we would need the 1.6 to test again).

Even if the uwazi software can handle somewhat nice different versions of the database without stopping, I think the dockerized version could enforce by default be more strict on this by default.

fititnt commented 5 years ago

(Also, few more minutes to test this pull request, first I was testing on the master with the FROM node:8-jessie and worked. Will do it too on this PR just to be sure)

Why wouldn't you want to move to stretch, as jessie by this time is really old and unsupported upstream?

@vasyugan another thing that maybe I will do is re-release older versions of uwazi-docker (see https://github.com/fititnt/uwazi-docker/releases) with this change to stick with the jessie.

In theory, this change on the default OS on the FROM node:8-slim also made older versions of the uwazi-docker do not work on the first try. And in some cases, some way to be able to run the full environment of older versions of a software could be useful for compare (for example, discover when something stoped worked, see performance downgrades, etc) or just test how to upgrade versions.

This is one additional reason of why the next versions of uwazi-docker, even if use the stretch, I think could be better to enforce the stretch on the Dockerfile instead of just rely on the version decided by the FROM node:8-slim.