search

reverbrain / eblob

Eblob is an append-only low-level IO library, which saves data in blob files. Created as low-level backend for elliptics
GNU Lesser General Public License v3.0
104 stars 29 forks source link

Build RPM on Centos #178

Open odiszapc opened 7 years ago

odiszapc commented 7 years ago

Excuse me, how to build eblob RPM on Centos 7?

Here are Debian instructions: http://doc.reverbrain.com/elliptics:server_eblob_tutorial

I trie to build:

mkdir build
cd build
cmake ..
make

What's next? Many thanks!

odiszapc commented 7 years ago

What I have tried:

[root@dev eblob]# rpmbuild -bb eblob-bf.spec
error: File /root/rpmbuild/SOURCES/eblob-0.23.14.tar.bz2: No such file or directory
bioothod commented 7 years ago

To build a package you have to create a tarball, it is usually something like this:

$ cd /root/rpmbuild/SOURCES
$ git clone http://github.com/reverbrain/eblob eblob-0.23.14
$ tar -ca --exclude-vcs -f eblob-0.23.14.tar.bz2 eblob-0.23.14
$ rpmbuild -bb eblob-0.23.14/eblob-bf.spec
bioothod commented 7 years ago

Or you can download tarball from github: https://github.com/reverbrain/eblob/archive/v0.23.14.tar.gz You will need to change spec file to point to github repository instead of local archive

odiszapc commented 7 years ago

Thanks. For now I have built all the RPMs on latest Centos 7. However I faced up with a conflict situation related to msgpack-devel dependency. During building with rpmbuild some ellitpics packets depends on msgpack-devel packet but during installation of target RPM it depends on compat-msgpack-devel. These two packets can not be installed simultaneously because they conflict with each other. It was resolved by replacing all msgpack-devel occurences with compat-msgpack-devel. It looks like a bug, but I need you confirmation.

For those who interested, here are steps to build and install Elliptics RPMs from scratch (please find out the sed lines:

# Install dependencies
yum install -y libev-devel libtool-ltdl-devel libuuid-devel libcgroup-devel compat-msgpack-devel libarchive-devel binutils-devel python-devel python-setuptools gtest-devel python-virtualenv libxml2-devel uriparser-devel libidn-devel

# Fetch sources
rm -rf elliptics
mkdir elliptics
cd elliptics
git clone http://github.com/reverbrain/eblob.git eblob-0.23.14
git clone http://github.com/reverbrain/elliptics.git elliptics-2.26.11.1
cd elliptics-2.26.11.1
git submodule init
git submodule update
git clone http://github.com/reverbrain/swarm.git libswarm-3.5.2
git clone http://github.com/cocaine/cocaine-framework-python.git -b v0.11 cocaine-framework-python-0.11.1.6
git clone http://github.com/cocaine/cocaine-framework-native.git -b v0.11 cocaine-framework-native-0.11.1.6
git clone http://github.com/cocaine/cocaine-tools.git -b v0.11 cocaine-tools-0.11.7.1
git clone http://github.com/cocaine/cocaine-core.git -b v0.11 libcocaine-core2-0.11.3.1
cd libcocaine-core2-0.11.3.1
git submodule init
git submodule update
cd ../

mkdir -p rpmbuild/SOURCES

# Build handystats
wget http://repo.reverbrain.com/sources/handystats/handystats-1.11.3.tar.bz2
tar -xjf handystats-1.11.3.tar.bz2
mv handystats-1.11.3.tar.bz2 rpmbuild/SOURCES
rpmbuild --define "_topdir `pwd`/rpmbuild" -bb handystats-1.11.3/handystats-bf.spec
rpm -Uvh rpmbuild/RPMS/x86_64/handystats-1.11.3-1.el7.centos.x86_64.rpm

# build blackhole
wget http://repo.reverbrain.com/sources/libblackhole/libblackhole-0.2.4.tar.bz2
tar -xjf libblackhole-0.2.4.tar.bz2
mv libblackhole-0.2.4.tar.bz2 rpmbuild/SOURCES
rpmbuild --define "_topdir `pwd`/rpmbuild" -bb libblackhole-0.2.4/blackhole-bf.spec
rpm -Uvh rpmbuild/RPMS/x86_64/libblackhole-0.2.4-1.el7.centos.x86_64.rpm
rpm -Uvh rpmbuild/RPMS/x86_64/libblackhole-devel-0.2.4-1.el7.centos.x86_64.rpm

# Build eblob
tar -ca --exclude-vcs -f rpmbuild/SOURCES/eblob-0.23.14.tar.bz2 eblob-0.23.14
rpmbuild --define "_topdir `pwd`/rpmbuild" -bb eblob-0.23.14/eblob-bf.spec
rpm -Uvh rpmbuild/RPMS/x86_64/eblob-0.23.14-1.el7.centos.1.x86_64.rpm
rpm -Uvh rpmbuild/RPMS/x86_64/eblob-devel-0.23.14-1.el7.centos.1.x86_64.rpm

# Build cocaine-core
sed -i libcocaine-core2-0.11.3.1/cocaine-bf.spec -e 's/ msgpack-devel/ compat-msgpack-devel/g'
tar -ca --exclude-vcs -f rpmbuild/SOURCES/libcocaine-core2-0.11.3.1.tar.bz2 libcocaine-core2-0.11.3.1
rpmbuild --define "_topdir `pwd`/rpmbuild" -bb libcocaine-core2-0.11.3.1/cocaine-bf.spec
rpm -Uvh rpmbuild/RPMS/x86_64/libcocaine-core2-0.11.3.1-1.el7.centos.x86_64.rpm
rpm -Uvh rpmbuild/RPMS/x86_64/libcocaine-core2-devel-0.11.3.1-1.el7.centos.x86_64.rpm

# Build cocaine-tools
tar -ca --exclude-vcs -f rpmbuild/SOURCES/cocaine-tools-0.11.7.1.tar.bz2 cocaine-tools-0.11.7.1
rpmbuild --define "_topdir `pwd`/rpmbuild" -bb cocaine-tools-0.11.7.1/cocaine-tools-bf.spec

# Build cocaine-framework-python
tar -ca --exclude-vcs -f rpmbuild/SOURCES/cocaine-framework-python-0.11.1.6.tar.bz2 cocaine-framework-python-0.11.1.6
rpmbuild --define "_topdir `pwd`/rpmbuild" -bb cocaine-framework-python-0.11.1.6/cocaine-framework-python-bf.spec

# Build cocaine-framework-native
sed -i cocaine-framework-native-0.11.1.6/cocaine-framework-native-bf.spec -e 's/ msgpack-devel/ compat-msgpack-devel/g'
tar -ca --exclude-vcs -f rpmbuild/SOURCES/cocaine-framework-native-0.11.1.6.tar.bz2 cocaine-framework-native-0.11.1.6
rpmbuild --define "_topdir `pwd`/rpmbuild" -bb cocaine-framework-native-0.11.1.6/cocaine-framework-native-bf.spec
rpm -Uvh rpmbuild/RPMS/x86_64/cocaine-framework-native-0.11.1.6-2.el7.centos.x86_64.rpm
rpm -Uvh rpmbuild/RPMS/x86_64/cocaine-framework-native-devel-0.11.1.6-2.el7.centos.x86_64.rpm

# Build elliptics
tar -ca --exclude-vcs -f rpmbuild/SOURCES/elliptics-2.26.11.1.tar.bz2 elliptics-2.26.11.1
rpmbuild --define "_topdir `pwd`/rpmbuild" -bb elliptics-2.26.11.1/elliptics-bf.spec
rpm -Uvh rpmbuild/RPMS/x86_64/elliptics-client-2.26.11.1-1.el7.centos.x86_64.rpm
rpm -Uvh rpmbuild/RPMS/x86_64/elliptics-2.26.11.1-1.el7.centos.x86_64.rpm
rpm -Uvh rpmbuild/RPMS/x86_64/elliptics-devel-2.26.11.1-1.el7.centos.x86_64.rpm
rpm -Uvh rpmbuild/RPMS/x86_64/elliptics-client-devel-2.26.11.1-1.el7.centos.x86_64.rpm
bioothod commented 7 years ago

Elliptics rpm package should only depend on compat-msgpack-devel package, since newer distributions use msgpack 1+ version, which is not compatible with older version we use.

What was the package which depended on msgpack instead of compat-msgpack-devel?

odiszapc commented 7 years ago

This: https://github.com/cocaine/cocaine-core/blob/v0.11/cocaine-bf.spec And this: https://github.com/cocaine/cocaine-framework-native/blob/v0.11/cocaine-framework-native-bf.spec

According to documentation (http://doc.reverbrain.com/elliptics:server_eblob_tutorial) I need the above packages to build elliptics (See https://github.com/reverbrain/elliptics/blob/master/elliptics-bf.spec, Line 15,16):

BuildRequires:  libcocaine-core2-devel >= 0.11.2.0
BuildRequires:  cocaine-framework-native-devel >= 0.11.0.0

, but these two depend on msgpack-devel (i.e., see https://github.com/cocaine/cocaine-core/blob/v0.11/cocaine-bf.spec, line 21:

BuildRequires: msgpack-devel, libarchive-devel, binutils-devel

which is in conflict with already installed compat-msgpack-devel.

Fix me if I'm wrong.

bioothod commented 7 years ago

Yup, that's a problem, let us think a little on the proper fix. You can install packages from our repository so far, elliptics was built without cocaine support there.

odiszapc commented 7 years ago

Sure, installing from repo works fine for me. I just want to build it from scratch, and it also works with a liitle bit of sed magic.

odiszapc commented 7 years ago

Maybe (just maybe) the solution is to move cocaine to compat-msgpack-devel ?

bioothod commented 7 years ago

Yes, that is one of the possible ways. But it doesn't solve the case when there is already some application on your server which requires msgpack-devel, or if it is just installed.

odiszapc commented 7 years ago

Yes, that was my situation, and I was forced to remove msgpack-devel. What is the difference between msgpack and compat-msgpack?

Other solution (not sure about that) is to link msgpack statically avoiding RPM deps.

odiszapc commented 7 years ago

Offtopic: did you consider Google Protocol Buffers? It's stable for months

bioothod commented 7 years ago

msgpack 1.x is not API compatible with msgpack 0.x, we used to use older version, since elliptics uses it for many years, currently distributives started to switch to newer backward incompatible API.

You can not statically link msgpack since there is no appropriate library in the distributives.

Protocol buffers are "stable for months"? For MONTHS? It is not enough - elliptics exists for more than 7 years and there are users all over the world who use it since the very beginning of its life, it is virtually impossible to switch to this packing protocol. I never considered protobuf at the first place because of obscure dependency on google libraries which are required to build protoc compiler, which in turn is not present in every distribution in binary package. Also, proto3 is not API compatible with proto2, and distributions start moving to proto3. Also, protobuf is slower than msgpack and is not present for some languages (requires even more external dependencies by third parties).

odiszapc commented 7 years ago

Then it makes sense.