search

apache / couchdb

Seamless multi-master syncing database with an intuitive HTTP/JSON API, designed for reliability
https://couchdb.apache.org/
Apache License 2.0
6.26k stars 1.03k forks source link

Apache CouchDB build issues on PowerPC64LE #2044

Closed sarveshtamba closed 5 years ago

sarveshtamba commented 5 years ago

Im trying to build Apache CouchDB on PowerPC64LE.

I have installed all dependencies and following all the steps mentioned at https://github.com/apache/couchdb/blob/master/INSTALL.Unix.md

Ran into the following issues:- Issue 1:- ========== root@70a8290c0358:~/couchdb/couchdb# make release ==> config (compile) ERROR: OTP release 18 does not match required regex 19|20|21 ERROR: compile failed while processing /root/couchdb/couchdb/src/config: rebar_abort Makefile:123: recipe for target 'couch' failed make: *** [couch] Error 1 root@70a8290c0358:~/couchdb/couchdb#

Solution/Workaround:- ====================== Latest version of erlang on PowerPC64LE is Erlang/OTP 18, could not find OTP 19 or greater for PowerPC64LE architecture. sar@70a8290c0358:~/couchdb/couchdb$ erl Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false]

Hence put in the following change in the file "rebar.config.script":-
-    {require_otp_vsn, "19|20|21"},
+    {require_otp_vsn, "18|19|20|21"},

Issue 2:- ========== root@70a8290c0358:~/couchdb/couchdb# make release . . . make[1]: Entering directory '/home/sar/couchdb/couchdb/src/docs' sphinx-build -b html -a -W -n -A local=1 -D latex_elements.papersize=a4 -d build/doctree src/ build/html Traceback (most recent call last): File "/usr/local/bin/sphinx-build", line 7, in from sphinx.cmd.build import main File "/home/sar/.local/lib/python2.7/site-packages/sphinx/cmd/build.py", line 39 file=stderr) ^ SyntaxError: invalid syntax Makefile:32: recipe for target 'html' failed make[1]: [html] Error 1 make[1]: Leaving directory '/home/sar/couchdb/couchdb/src/docs' Makefile:520: recipe for target 'src/docs/build' failed make: [src/docs/build] Error 2

Solution/Workaround:- ====================== Used commands "./configure --disable-docs" and "make release" so as to not build any documentation or manpages using Sphinx.

With this, I am able to build Apache CouchDB on PowerPC64LE, testing is pending though.

Any thoughts/help on the above two issues would be highly appreciated.

sarveshtamba commented 5 years ago

Please note, I was able to configure and build couchdb using non-root user though with disable-docs option.

wohali commented 5 years ago

Thanks @sarveshtamba .

We don't support Erlang 18 anymore. Explicitly, newer versions of CouchDB than 2.3.1 will require Erlang 19+ (or maybe newer). We will not go back to supporting lower versions of Erlang, so you'll need to compile and install Erlang from scratch.

You probably need Python 3 at this point for the documentation build; our build machines no longer run Sphinx with Python 2. I'd give that a try.

sarveshtamba commented 5 years ago

Hi,

Thanks for the reply.

I installed the following on Ubuntu 18.04 for PowerPC64LE inside a clean container:-

I tried building latest CouchDB from the master branch - https://github.com/apache/couchdb.git However now I am facing the following error:- "CouchDB does not support this version of Erlang ("20.2.2")."

Does CouchDB depend on specific versions of Erlang? If so, are the exact version requirements documented anywhere?

Below are the version/configure details:-

sar@ca6f5b10ec5b:~/couchdb/couchdb$ py
py3clean           pybabel            pydoc3             pygettext3.6       python3-config     python3.6m         python3m-config
py3compile         pybabel-python3    pydoc3.6           pygmentize         python3.6          python3.6m-config
py3versions        pybuild            pygettext3         python3            python3.6-config   python3m

sar@ca6f5b10ec5b:~/couchdb/couchdb$ python3 -V
Python 3.6.7

sar@ca6f5b10ec5b:~/couchdb/couchdb$ pip3 -V
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

sar@ca6f5b10ec5b:~/couchdb/couchdb$ erl -V
Erlang/OTP 20 [erts-9.2] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:10] [kernel-poll:false]

Eshell V9.2  (abort with ^G)
1>
[2]+  Stopped                 erl -V

sar@ca6f5b10ec5b:~/couchdb/couchdb$ ./configure
==> configuring couchdb in rel/couchdb.config
==> updating dependencies
CouchDB does not support this version of Erlang ("20.2.2").
Check https://docs.couchdb.org/en/latest/whatsnew/index.html for the
latest information on supported releases.
sar@ca6f5b10ec5b:~/couchdb/couchdb$
wohali commented 5 years ago

Hi there,

Unfortunately, we have had to blacklist a number of Erlang releases for bugs that can lead to scheduler collapse based on reports from IBM staff and confirmation from the Erlang developers.

See #1857 and #2047 .

Sorry about that.

sarveshtamba commented 5 years ago

Hi @wohali ,

I have managed to build Erlang/OTP 21.3.8.3 on Ubunutu 16.04 after resolving all dependencies for Erlang:-

    sar@50c649c461fc:~/otp$ erl -eval '{ok, Version} = file:read_file(filename:join([code:root_dir(), "releases", erlang:system_info(otp_release), "OTP_VERSION"])), io:fwrite(Version), halt().' -noshell
    21.3.8.3
    sar@50c649c461fc:/usr/local/lib/erlang$ cat releases/RELEASES
    %% coding: utf-8
    [{release,"Erlang/OTP","21","10.3.5.2",
              [{kernel,"6.3.1.1","/usr/local/lib/erlang/lib/kernel-6.3.1.1"},
               {stdlib,"3.8.2.1","/usr/local/lib/erlang/lib/stdlib-3.8.2.1"},
               {sasl,"3.3","/usr/local/lib/erlang/lib/sasl-3.3"}],
              permanent}].
    sar@50c649c461fc:~/couchdb/couchdb$ erl
    Erlang/OTP 21 [erts-10.3.5.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

    Eshell V10.3.5.2  (abort with ^G)
    1>

I tried building latest CouchDB from the master branch - https://github.com/apache/couchdb.git However now I am facing the following error:- 

    Running "copy:dist" (copy) task

    Running "copy:couchdb" (copy) task
    Created 4 directories, copied 22 files

    Done.
    make[1]: Entering directory '/home/sar/couchdb/couchdb/src/docs'
    sphinx-build -b html -a -W -n -A local=1 -D latex_elements.papersize=a4 -d build/doctree src/ build/html
    Running Sphinx v1.3.6
    making output directory...
    loading pickled environment... not yet created
    building [mo]: all of 0 po files
    building [html]: all source files
    updating environment: 124 added, 0 changed, 0 removed
    reading sources... [100%] whatsnew/index
    looking for now-outdated files... none found
    pickling environment... done
    checking consistency... done
    preparing documents...
    Theme error:
    unsupported theme option 'canonical_url' given
    Makefile:32: recipe for target 'html' failed
    make[1]: *** [html] Error 1
    make[1]: Leaving directory '/home/sar/couchdb/couchdb/src/docs'
    Makefile:520: recipe for target 'src/docs/build' failed
    make: *** [src/docs/build] Error 2

Am I missing anything?

wohali commented 5 years ago

Your Sphinx is outdated:

Running Sphinx v1.3.6

You need v1.7 at least, I believe.

I recommend you install python3-pip and do # pip3 install sphinx instead of relying on the very old Ubuntu 16.04 package.

sarveshtamba commented 5 years ago

Your Sphinx is outdated:

Running Sphinx v1.3.6

You need v1.7 at least, I believe.

I recommend you install python3-pip and do # pip3 install sphinx instead of relying on the very old Ubuntu 16.04 package.

Tried that, getting the below error:-

==> setup (compile)
Compiled src/setup_httpd_handlers.erl
Compiled src/setup_epi.erl
Compiled src/setup_sup.erl
Compiled src/setup_app.erl
Compiled src/setup_httpd.erl
Compiled src/setup.erl
==> rel (compile)
==> couchdb (compile)
Installing CouchDB into rel/couchdb/ ...
==> rel (generate)
WARN:  'generate' command does not apply to directory /home/sar/couchdb/couchdb
cp: cannot stat 'src/docs/build/man/apachecouchdb.1': No such file or directory
Makefile:407: recipe for target 'release' failed
make: *** [release] Error 1
sar@50c649c461fc:~/couchdb/couchdb$
wohali commented 5 years ago

Try with a brand new git clone, there's some bugs in the Makefile if the docs only partially fail to build that no one's ever cleaned up.

Sphinx makes the manpage in a second step, see https://github.com/apache/couchdb-documentation/blob/master/Makefile#L44-L45 . If that failed but the first step succeeded, it won't try again to build it today. You could also cd src/docs && make man to diagnose if there is a problem.

If that all still fails, let me know and I'll look at it next week.

sarveshtamba commented 5 years ago

Hi @wohali ,

I tried with a clean git clone of couchdb with all the pre-requisites installed.

I ran into the following issue:-

    make[1]: Entering directory '/home/sar/couchdb/couchdb/src/docs'
    sphinx-build -b html -a -W -n -A local=1 -D latex_elements.papersize=a4 -d build/doctree src/ build/html
    Running Sphinx v2.1.0
    making output directory... done
    building [mo]: all of 0 po files
    building [html]: all source files
    updating environment: 124 added, 0 changed, 0 removed
    reading sources... [100%] whatsnew/index
    looking for now-outdated files... none found
    pickling environment... done
    checking consistency... done
    preparing documents... done
    writing output... [100%] whatsnew/index

    Warning, treated as error:
    /home/sar/couchdb/couchdb/src/docs/src/api/server/common.rst:1489:Could not lex literal_block as "http". Highlighting skipped.
    Makefile:32: recipe for target 'html' failed
    make[1]: *** [html] Error 2
    make[1]: Leaving directory '/home/sar/couchdb/couchdb/src/docs'
    Makefile:520: recipe for target 'src/docs/build' failed
    make: *** [src/docs/build] Error 2

However I executed the workaround cd src/docs && make man as suggested and then re-built couchdb, the build completed successfully.

    Installing CouchDB into rel/couchdb/ ...
    ==> rel (generate)
    WARN:  'generate' command does not apply to directory /home/sar/couchdb/couchdb
    ... done

        You can now copy the rel/couchdb directory anywhere on your system.
        Start CouchDB with ./bin/couchdb from within that directory.

Question 1:- Is this a known/documented issue and any resolution for this workaround?

Next, I started the couchdb instance as per steps in the INSTALL.Unix.md readme, and ran into following issues at startup:-

[error] 2019-06-10T08:25:53.103978Z couchdb@localhost emulator -------- Error in process <0.326.0> on node couchdb@localhost with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,378}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,407}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,91}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,134}]}]}

I was able to resolve this after creating the three system databases manually on startup in another shell, while the couchdb instance was running in the first shell as per http://docs.couchdb.org/en/latest/setup/single-node.html:

curl -X PUT http://127.0.0.1:5984/_users
curl -X PUT http://127.0.0.1:5984/_replicator
curl -X PUT http://127.0.0.1:5984/_global_changes

Question 2:- Initially I tried inside a docker container, however opening another shell in a docker container is not possible. How do we proceed in this case?

Post this I was able to perform some operations as per https://guide.couchdb.org/draft/tour.html:-

    root@pts00452-vm13:~# curl http://127.0.0.1:5984/
    {"couchdb":"Welcome","version":"2.1.0-cf00dc2","features":["scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
    root@pts00452-vm13:~# curl -X GET http://127.0.0.1:5984/_all_dbs
    ["_global_changes","_replicator","_users"]
    root@pts00452-vm13:~# curl -X PUT http://127.0.0.1:5984/baseball
    {"ok":true}
    root@pts00452-vm13:~# curl -X GET http://127.0.0.1:5984/_all_dbs
    ["_global_changes","_replicator","_users","baseball"]
    root@pts00452-vm13:~# curl -X PUT http://127.0.0.1:5984/baseball
    {"error":"file_exists","reason":"The database could not be created, the file already exists."}
    root@pts00452-vm13:~# curl -X PUT http://127.0.0.1:5984/plankton
    {"ok":true}
    root@pts00452-vm13:~# curl -X GET http://127.0.0.1:5984/_all_dbs
    ["_global_changes","_replicator","_users","baseball","plankton"]
    root@pts00452-vm13:~# curl -X DELETE http://127.0.0.1:5984/plankton
    {"ok":true}
    root@pts00452-vm13:~# curl -X GET http://127.0.0.1:5984/_all_dbs
    ["_global_changes","_replicator","_users","baseball"]

Let me know your thoughts on this.

sarveshtamba commented 5 years ago

@wohali any inputs?

wohali commented 5 years ago

I'm not sure about your problem with the build, but this shouldn't be happening. Will leave this open until someone can look at it. I'm out of time for this week, sorry.

You can open another shell in a Docker container: https://stackoverflow.com/questions/17903705/is-it-possible-to-start-a-shell-session-in-a-running-container-without-ssh

For any other user-level questions these are not CouchDB bugs and you should seek assistance on our user@couchdb.apache.org mailing list or Slack, whichever you prefer.

sarveshtamba commented 5 years ago

Entire test suite for CouchDB v2.3(current master) executes successfully/passes with Erlang v21 and Elixir v1.8 on PowerPC64LE. Closing this issue. Thanks for all your help and support in getting this through.