Closed bbarani closed 1 year ago
jcgraybill
commented
2 years ago It looks like Elasticsearch's DEB packages set up SysV or systemd hooks for running Elasticsearch. In the spirit of allowing OpenSearch to be a drop-in migration, let's be sure its DEB packages are set up the same way. https://www.elastic.co/guide/en/elasticsearch/reference/current/starting-elasticsearch.html#start-deb
mbreevoort
commented
2 years ago Using opendistro and waiting until the debian packages are available, does it make v1 2.0 this time?
gaiksaya
commented
2 years ago [Triage] We are working on the design documentation at this point. Please see #956
deknos
commented
2 years ago Hello, please consider to generate not only debian/rpm binary packages, but also source packages.
This will for one ensure that your packaging quality will be better, it also would enhance the possibilities that your software would be included in standard repositories of systems like redhat, debian, ubuntu, suse as they will only include packages where the have the source version of.
Please consider this. This would dramatically enhance your position as a project in the opensource community.
dblock
commented
2 years ago @deknos Do you have some more specifics about this for those like me not familiar with the different kinds of packages?
deknos
commented
2 years ago Sure! For Debian at least here: https://wiki.debian.org/Packaging/SourcePackage
What you do in general is, that you have an original software (like your source) and you build with debian tooling a debian source package which are the original tar.gz (just an example) and a dsc-file. with these you can build a debian binary package with dpkg-buildpackage.
for the question how to do this, there's a good tutorial by a former debian project leader: https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial.en.pdf
the only drawback is: debian in general wants that you also package your dependency properly, i do not know if there's some kind of workaround if you do not want to to this.
But also: when you did this for debian (source package, from which a debian binary package is built easily), building an rpm package is very easy because they adhere to similar principles.
if you want, you can contact me (dennis.knorr@gmx.net) and i could try to help with debian packaging, i packaged software for a company a few years back :)
dblock
commented
2 years ago Thanks @deknos. This is similar to how FreeBSD folks do it, or how OpenSearch was added to Homebrew, i.e. build from source using tooling on the host on which the software is ultimately going to reside. Generally I see two drawbacks to that approach: it would produce a different set of binaries depending on the environment, it doesn't produce signed binaries that users can rely on and verify the origin of (or we'd have to use the rpm process to build the binaries, then release the signed bits, but rpm doesn't exist on all platforms, and in this issue we chose to go directly to a binary package that includes signed bits built from source).
I think we'd spend our energy building support for things like MacOS or Windows before we build a second or a third way of distributing for Linux. I think though we would definitely welcome a community project just like what was done for FreeBSD and Homebrew and we could integrate it similarly to those (some source code is in opensearch-project, such as docs, and some code is not, such as the FreeBSD ports themselves).
Thanks for the offer to do the debian packaging work though, don't let me stop you from doing that! We'd gladly accept PRs here if they move us forward into the right direction. You could absolutely start contributing those things here, or build a personal repo, up to you and let us know how we can help.
theOperand
commented
2 years ago I think we'd spend our energy building support for things like MacOS or Windows before we build a second or a third way of distributing for Linux.
Does this mean that this issue is postponed indefinitely? A .deb package would be very useful, as many organizations (such as mine) that use Ubuntu servers but no Docker are still effectively stuck on the last non-SSPL Elasticsearch version.
dblock
commented
2 years ago It just means nobody is working on it. @peterzhuamazon care to update this ticket with a list of things you think one would need to do to make a .deb package so that someone can contribute?
bbarani
commented
2 years ago We are currently working on putting together a high level design and plan for both Windows and Deb distribution. We will publish the initial version of the plan very soon on this issue. Having said that, we don't have the release timeline for additional Linux distributions yet.
gruselglatz
commented
2 years ago How is the current State? Do you need some Help? There is already a Debian Build in the project iirc in the build.gradle?
marcinwisniowski
commented
2 years ago @bbarani Can you tell us any ETA of DEBs for OpenSearch? It's over a year that Ubuntu users are waiting to adopt OpenSearch in place of OpenDistro :)
bertvandepoel
commented
2 years ago I see this has moved to In-progress. Is there anything we can do to help with development or testing? We're upgrading from 20.04 to 22.04 and are concerned that our old pre-SSPL packages will at some point fail. So we have a lot of motivation to help out :sweat_smile:
peterzhuamazon
commented
2 years ago I see this has moved to In-progress. Is there anything we can do to help with development or testing? We're upgrading from 20.04 to 22.04 and are concerned that our old pre-SSPL packages will at some point fail. So we have a lot of motivation to help out 😅
setiah
commented
1 year ago Adding a related request from the forums - https://forum.opensearch.org/t/debian-ubuntu-installer-please/11442
peterzhuamazon
commented
1 year ago Hi @setiah @bbarani, now we have merged @mnin changes to have deb in python code, next step would be to bring it up to Jenkins.
Please advice on the plans to get it into a release form.
Thanks.
peterzhuamazon
commented
1 year ago Weird issue on Jenkins as local docker does not show such cases:
2022-12-14 19:25:12 INFO Execute debmake -f "OpenSearch Team" -e "opensearch@amazon.com" -i debuild -p opensearch -n -r 1 -u 2.4.1 in /tmp/tmptbsf26kp
Traceback (most recent call last):
File "/usr/bin/debmake", line 25, in <module>
import debmake
ModuleNotFoundError: No module named 'debmake'
2022-12-14 19:25:12 INFO Removing /tmp/tmptbsf26kp
Traceback (most recent call last):
File "./src/run_assemble.py", line 58, in <module>
sys.exit(main())
File "./src/run_assemble.py", line 49, in main
bundle.package(output_dir)
File "/var/jenkins/workspace/bundle-build-zhujiaxi@2/src/assemble_workflow/bundle.py", line 103, in package
self.min_dist.build(self.bundle_recorder.package_name, dest)
File "/var/jenkins/workspace/bundle-build-zhujiaxi@2/src/assemble_workflow/dist.py", line 85, in build
self.__build__(name, dest)
File "/var/jenkins/workspace/bundle-build-zhujiaxi@2/src/assemble_workflow/dist.py", line 127, in __build__
BundleLinuxDeb(self.filename, self.path, self.min_path).build(name, dest, self.archive_path, self.build_cls)
File "/var/jenkins/workspace/bundle-build-zhujiaxi@2/src/assemble_workflow/bundle_linux_deb.py", line 121, in build
subprocess.check_call(bundle_cmd, cwd=ext_dest, shell=True)
File "/usr/local/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'debmake -f "OpenSearch Team" -e "opensearch@amazon.com" -i debuild -p opensearch -n -r 1 -u 2.4.1' returned non-zero exit status 1.Apparently install python37 completely broke debmake on Ubuntu2004.
jgrammen-agilitypr
commented
1 year ago Doesn't Ubuntu 20.04 Focal ship python3.8 natively? so python 3.7 would be a downgrade from the stock version of python.
peterzhuamazon
commented
1 year ago More issue: deb-log.txt
peterzhuamazon
commented
1 year ago Doesn't Ubuntu 20.04 Focal ship python3.8 natively? so python 3.7 would be a downgrade from the stock version of python.
Our code base is based on 3.7 version, I just figured out a way to resolve the issue.
peterzhuamazon
commented
1 year ago Doesn't Ubuntu 20.04 Focal ship python3.8 natively? so python 3.7 would be a downgrade from the stock version of python.
Also, the image I used does not install 38 natively, so I can just install 37 and all the new packages that relies on python3 will just install without the need of 38.
peterzhuamazon
commented
1 year ago Tested 18.04 / 20.04 / 22.04 all have different issues. Would need to test 3.8 python and see if that is the cause. Thanks.
marcosfsch
commented
1 year ago I was able to run build.sh successfully using pyenv:
WORKDIR /usr/share/opensearch
RUN curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash
ENV PYENV_ROOT /usr/share/opensearch/.pyenv
ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH
RUN pyenv install 3.7.7I was able to run build.sh successfully using pyenv:
WORKDIR /usr/share/opensearch RUN curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash ENV PYENV_ROOT /usr/share/opensearch/.pyenv ENV PATH $PYENV_ROOT/shims:$PYENV_ROOT/bin:$PATH RUN pyenv install 3.7.7
Hi man, build.sh is not the issue as the problem now is with assemble.sh using debmake on Ubuntu2004 server. Thanks.
peterzhuamazon
commented
1 year ago More issue: deb-log.txt
Hi @mnin
I am currently on vacation so I dont have access to my dev env yet. I see your suggestions on skipping the lib check, which seems useful. I have transitioned my current process to @prudhvigodithi and he will take care of DEB in the next few weeks.
Would love to see if this issue is resolved. Thanks.
peterzhuamazon
commented
1 year ago New issue testing on a debian 10 with python 37 showing these:
dwz: Too few files for multifile optimization
objcopy: 'debian/opensearch/usr/lib/debug/.dwz/x86_64-linux-gnu/opensearch.debug': No such file
dh_dwz: objcopy --compress-debug-sections debian/opensearch/usr/lib/debug/.dwz/x86_64-linux-gnu/opensearch.debug returned exit code 1
make: *** [debian/rules:9: binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui failedI have confirmed that build have to happen at least on debian11/ubuntu2004 to bypass above errors. At least I reach this stage on both debian11/ubuntu2004 then fail the dep checks:
Tried to use @mnin suggestions on this in rules file:
override_dh_shlibdeps:
dh_shlibdeps --dpkg-shlibdeps-params=--ignore-missing-infoStill failed and have a rule error.
Still researching.
Thanks.
peterzhuamazon
commented
1 year ago Solved one:
sudo apt-get install libxrender1Solved two:
apt-get install libxtst6Solved three:
apt-get install libasound2
Finished running lintian.
I: upon return to the shell, current directory becomes /tmp/tmppyavbyxh
I: please execute "cd .." and inspect the build results.
2023-01-05 18:29:07 INFO Found deb file: None
2023-01-05 18:29:07 INFO Published /test/opensearch-build/deb/dist/opensearch/opensearch-2.4.1-linux-x64.deb.
2023-01-05 18:29:07 INFO Removing /tmp/tmppyavbyxh
2023-01-05 18:29:08 INFO Done.
It's there.
mnin
commented
1 year ago @peterzhuamazon thanks for your effort to fix these issues!
peterzhuamazon
commented
1 year ago Seems like more missing when running on Jenkins:
dpkg-shlibdeps: error: cannot find library libXi.so.6
apt-get install libxi6Try again
peterzhuamazon
commented
1 year ago Seems like the above is caused by python37, python38 by default is ok with this.
peterzhuamazon
commented
1 year ago Installed more pkgs and get close enough, not sure why there is suddenly a permission issue:
apt-get install libxi6 libgconf-2-4
N: 77 tags overridden (73 errors, 3 warnings, 1 info)
Finished running lintian.
I: upon return to the shell, current directory becomes /tmp/tmpy_8qwj3l
I: please execute "cd .." and inspect the build results.
2023-01-06 02:55:48 INFO Found deb file: None
2023-01-06 02:55:48 INFO Removing /tmp/tmpy_8qwj3l
Traceback (most recent call last):
File "/usr/lib/python3.7/shutil.py", line 566, in move
os.rename(src, real_dst)
PermissionError: [Errno 13] Permission denied: '/tmp/opensearch_2.4.1_amd64.deb' -> 'opensearch-2.4.1-linux-x64.deb'
During handling of the above exception, another exception occurred:Seems like the issue is due to user used to be root in my last test.
peterzhuamazon
commented
1 year ago Able to get jenkins to run full end-to-end on build to assemble to upload to s3 for deb on opensearch, dashboards standby.
peterzhuamazon
commented
1 year ago Also seems like we need to remove Multi-Arch: foreign as amd64 version should not be used for arm64 version?
Quote: https://wiki.debian.org/Multiarch/HOWTO
If a package is marked ‘Multi-Arch: foreign’, then it can satisfy dependencies of a package of a different architecture (e.g `debhelper:amd64` will satisfy a dependency on debhelper for any-architecture package).Some description might need changes as well:
new Debian package, version 2.0.
size 554485222 bytes: control archive=24731 bytes.
1604 bytes, 30 lines conffiles
245 bytes, 10 lines control
94339 bytes, 934 lines md5sums
471 bytes, 11 lines * postinst #!/bin/sh
32 bytes, 1 lines shlibs
73 bytes, 2 lines triggers
Package: opensearch
Version: 2.4.1
Architecture: amd64
Maintainer: OpenSearch Team <opensearch@amazon.com>
Installed-Size: 759860
Section: database
Priority: optional
Multi-Arch: foreign
Homepage: https://opensearch.org/
Description: OpenSearchBUG:
I: debuild
dpkg-buildpackage -us -uc -ui
dpkg-buildpackage: info: source package opensearch-dashboards
dpkg-buildpackage: info: source version 2.4.0
dpkg-buildpackage: info: source distribution UNRELEASED
dpkg-buildpackage: info: source changed by OpenSearch Team <opensearch@amazon.com>
dpkg-source --before-build .
dpkg-buildpackage: info: host architecture amd64
fakeroot debian/rules clean
dh clean
dh_auto_clean
dh_clean
dpkg-source -b .
dpkg-source: warning: no source format specified in debian/source/format, see dpkg-source(1)
dpkg-source: warning: source directory 'tmppqr2vfon' is not <sourcepackage>-<upstreamversion> 'opensearch-dashboards-2.4.0'
dpkg-source: info: using source format '1.0'
dpkg-source: info: building opensearch-dashboards in opensearch-dashboards_2.4.0.tar.gz
dpkg-source: info: building opensearch-dashboards in opensearch-dashboards_2.4.0.dsc
debian/rules build
dh build
dh_update_autotools_config
dh_autoreconf
dh_auto_configure
dh_auto_build
make -j8 "INSTALL=install --strip-program=true"
make[1]: Entering directory '/tmp/tmppqr2vfon'
mkdir -p /tmp/tmppqr2vfon/debian/opensearch
cp -a /tmp/tmppqr2vfon/etc /tmp/tmppqr2vfon/debian/opensearch/etc
cp -a /tmp/tmppqr2vfon/usr /tmp/tmppqr2vfon/debian/opensearch/usr
cp -a /tmp/tmppqr2vfon/var /tmp/tmppqr2vfon/debian/opensearch/var
make[1]: Leaving directory '/tmp/tmppqr2vfon'
dh_auto_test
create-stamp debian/debhelper-build-stamp
fakeroot debian/rules binary
dh binary
dh_testroot
dh_prep
dh_auto_install
make -j8 install DESTDIR=/tmp/tmppqr2vfon/debian/opensearch-dashboards AM_UPDATE_INFO_DIR=no "INSTALL=install --strip-program=true"
make[1]: Entering directory '/tmp/tmppqr2vfon'
mkdir -p /tmp/tmppqr2vfon/debian/opensearch
cp -a /tmp/tmppqr2vfon/etc /tmp/tmppqr2vfon/debian/opensearch/etc
cp -a /tmp/tmppqr2vfon/usr /tmp/tmppqr2vfon/debian/opensearch/usr
cp -a /tmp/tmppqr2vfon/var /tmp/tmppqr2vfon/debian/opensearch/varOSD is going to OS directory causing the pkg to be only 4KB size:
4.0K opensearch-dashboards-2.4.0-linux-x64.debAlso OS part the deb is having some issues to start:
● opensearch.service - OpenSearch
Loaded: loaded (/lib/systemd/system/opensearch.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sat 2023-01-07 01:40:08 UTC; 13s ago
Docs: https://opensearch.org/
Process: 10071 ExecStart=/usr/share/opensearch/bin/systemd-entrypoint -p ${PID_DIR}/opensearch.pid --quiet (code=exited, status=217/USER)
Main PID: 10071 (code=exited, status=217/USER)
Jan 07 01:40:08 ip systemd[1]: Starting OpenSearch...
Jan 07 01:40:08 ip systemd[10071]: opensearch.service: Failed to determine user credentials: No such process
Jan 07 01:40:08 ip systemd[10071]: opensearch.service: Failed at step USER spawning /usr/share/opensearch/bin/systemd-entrypoint: No such process
Jan 07 01:40:08 ip systemd[1]: opensearch.service: Main process exited, code=exited, status=217/USER
Jan 07 01:40:08 ip systemd[1]: opensearch.service: Failed with result 'exit-code'.
Jan 07 01:40:08 ip systemd[1]: Failed to start OpenSearch.
Changed to OSD folder and now it shows more reports related deps. Tho chrome might be removed for 2.5.0 using 2.4.x to test still need to resolve these:
dpkg-shlibdeps: warning: debian/opensearch-dashboards/usr/share/opensearch-dashboards/node_modules/re2/build/Release/re2.node contains an unresolvable reference to symbol _ZN2v814ObjectTemplate11NewInstanceENS_5LocalINS_7ContextEEE: it's probably a plugin
dpkg-shlibdeps: warning: 70 other similar warnings have been skipped (use -v to see them all)
dpkg-shlibdeps: warning: cannot find library libnss3.so needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: warning: cannot find library libnssutil3.so needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: warning: cannot find library libsmime3.so needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: warning: cannot find library libnspr4.so needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libatk-1.0.so.0 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libatk-bridge-2.0.so.0 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libcups.so.2 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libdrm.so.2 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libatspi.so.0 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libXcomposite.so.1 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libXdamage.so.1 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libXfixes.so.3 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libXrandr.so.2 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libgbm.so.1 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libxkbcommon.so.0 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libpango-1.0.so.0 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libcairo.so.2 needed by debian/opensearch-dashboards/usr/share/opensearch-dashboards/plugins/reportsDashboards/.chromium/headless_shell (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 libatspi2.0-dev libxcomposite-dev libxdamage1 libxfixes3 libxfixes-dev libxrandr2 libgbm-dev libxkbcommon-x11-0 libpangocairo-1.0-0 libcairo2 libcairo2-dev libnss3 libnspr4 libnspr4-dev
freeglut3 possible
need to also install aptly and gpg.
peterzhuamazon
commented
1 year ago Fixed OSD size:
318M opensearch-dashboards-2.4.0-linux-x64.debNeed proper maintainers scripts just like in RPM to make DEB run correctly.
peterzhuamazon
commented
1 year ago After a lot of research we will not sign DEB packages but sign APT repo InRelease files. As for the deb packages directly available for download, not through apt, we will provide detached .sig file just like rpm.
peterzhuamazon
commented
1 year ago OS deb now:
$ dpkg-deb -I opensearch-2.4.1-linux-x64.deb
new Debian package, version 2.0.
size 554486428 bytes: control archive=25738 bytes.
1687 bytes, 32 lines conffiles
418 bytes, 12 lines control
94409 bytes, 935 lines md5sums
2239 bytes, 63 lines * postinst #!/bin/sh
1040 bytes, 31 lines * preinst #!/bin/sh
774 bytes, 26 lines * prerm #!/bin/sh
32 bytes, 1 lines shlibs
73 bytes, 2 lines triggers
Package: opensearch
Version: 2.4.1
Architecture: amd64
Maintainer: OpenSearch Team <opensearch@amazon.com>
Installed-Size: 759872
Section: web
Priority: optional
Homepage: https://opensearch.org/
Description: An open source distributed and RESTful search engine
OpenSearch makes it easy to ingest, search, visualize, and analyze your data
For more information, see: https://opensearch.org/
License: Apache-2.0Since Ubuntu will point the default /bin/sh to dash instead of the normal bourne shell, I will have to force everything on bash so some of the scripts like the install demo config will not fail due to it is written in bash build-ins.
$ ll /usr/bin/sh
lrwxrwxrwx 1 root root 4 Dec 12 21:39 /usr/bin/sh -> dash*
$ ll /bin/sh
lrwxrwxrwx 1 root root 4 Dec 12 21:39 /bin/sh -> dash*$ sh /usr/share/opensearch/plugins/opensearch-security/tools/install_demo_configuration.sh -y -i -s
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755 **
**************************************************************************
/usr/share/opensearch/plugins/opensearch-security/tools/install_demo_configuration.sh: 9: Bad substitutionOS online:
$ sudo dpkg -i opensearch-2.4.1-linux-x64.deb
Selecting previously unselected package opensearch.
(Reading database ... 146329 files and directories currently installed.)
Preparing to unpack opensearch-2.4.1-linux-x64.deb ...
Running preinst script
Unpacking opensearch (2.4.1) ...
Setting up opensearch (2.4.1) ...
Running postinst script
### NOT starting on installation, please execute the following statements to configure opensearch service to start automatically using systemd
sudo systemctl daemon-reload
sudo systemctl enable opensearch.service
### You can start opensearch service by executing
sudo systemctl start opensearch.service
### Create opensearch demo certificates in /etc/opensearch/
See demo certs creation log in /var/log/opensearch/install_demo_configuration.log
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
Processing triggers for systemd (245.4-4ubuntu3.19) ...
$ cat /var/log/opensearch/install_demo_configuration.log
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755 **
**************************************************************************
OpenSearch Security Demo Installer
** Warning: Do not use on production or public reachable systems **
Basedir: /usr/share/opensearch
This script maybe require your root password for 'sudo' privileges
OpenSearch install type: rpm/deb on DISTRIB_ID=Ubuntu
OpenSearch config dir: /etc/opensearch
OpenSearch config file: /etc/opensearch/opensearch.yml
OpenSearch bin dir: /usr/share/opensearch/bin
OpenSearch plugins dir: /usr/share/opensearch/plugins
OpenSearch lib dir: /usr/share/opensearch/lib
Detected OpenSearch Version: x-content-2.4.1
Detected OpenSearch Security Version: 2.4.1.0
### Success
### Execute this script now on all your nodes and then start all nodes
### OpenSearch Security will be automatically initialized.
### If you like to change the runtime configuration
### change the files in ../../../config/opensearch-security and execute:
sudo "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/etc/opensearch/opensearch-security" -icl -key "/etc/opensearch/kirk-key.pem" -cert "/etc/opensearch/kirk.pem" -cacert "/etc/opensearch/root-ca.pem" -nhnv
### or run ./securityadmin_demo.sh
### To use the Security Plugin ConfigurationGUI
### To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin.
### (Ignore the SSL certificate warning because we installed self-signed demo certificates)
$ sudo systemctl start opensearch.service
$ sudo systemctl status opensearch.service
● opensearch.service - OpenSearch
Loaded: loaded (/lib/systemd/system/opensearch.service; disabled; vendor preset: enabled)
Active: active (running) since Sun 2023-01-08 06:04:54 UTC; 20s ago
Docs: https://opensearch.org/
Main PID: 19546 (java)
Tasks: 71 (limit: 18830)
Memory: 1.2G
CGroup: /system.slice/opensearch.service
└─19546 /usr/share/opensearch/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMe>
Jan 08 06:04:40 ip systemd[1]: Starting OpenSearch...
Jan 08 06:04:42 ip systemd-entrypoint[19546]: WARNING: A terminally deprecated method in java.lang.System has been called
Jan 08 06:04:42 ip systemd-entrypoint[19546]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.4.1.jar)
Jan 08 06:04:42 ip systemd-entrypoint[19546]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Jan 08 06:04:42 ip systemd-entrypoint[19546]: WARNING: System::setSecurityManager will be removed in a future release
Jan 08 06:04:43 ip systemd-entrypoint[19546]: WARNING: A terminally deprecated method in java.lang.System has been called
Jan 08 06:04:43 ip systemd-entrypoint[19546]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.4.1.jar)
Jan 08 06:04:43 ip systemd-entrypoint[19546]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Jan 08 06:04:43 ip systemd-entrypoint[19546]: WARNING: System::setSecurityManager will be removed in a future release
Jan 08 06:04:54 ip systemd[1]: Started OpenSearch.
$ curl https://localhost:9200 -u admin:admin --insecure
{
"name" : "ip",
"cluster_name" : "opensearch",
"cluster_uuid" : "M2nPp2cyS9edB6bpRlQYxg",
"version" : {
"distribution" : "opensearch",
"number" : "2.4.1",
"build_type" : "deb",
"build_hash" : "f2f809ea280ffba217451da894a5899f1cec02ab",
"build_date" : "2023-01-06T21:08:43.134090306Z",
"build_snapshot" : false,
"lucene_version" : "9.4.2",
"minimum_wire_compatibility_version" : "7.10.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
$ curl https://localhost:9200/_cat/plugins -u admin:admin --insecure
ip-172-31-11-175 opensearch-alerting 2.4.1.0
ip-172-31-11-175 opensearch-anomaly-detection 2.4.1.0
ip-172-31-11-175 opensearch-asynchronous-search 2.4.1.0
ip-172-31-11-175 opensearch-cross-cluster-replication 2.4.1.0
ip-172-31-11-175 opensearch-geospatial 2.4.1.0
ip-172-31-11-175 opensearch-index-management 2.4.1.0
ip-172-31-11-175 opensearch-job-scheduler 2.4.1.0
ip-172-31-11-175 opensearch-knn 2.4.1.0
ip-172-31-11-175 opensearch-ml 2.4.1.0
ip-172-31-11-175 opensearch-neural-search 2.4.1.0
ip-172-31-11-175 opensearch-notifications 2.4.1.0
ip-172-31-11-175 opensearch-notifications-core 2.4.1.0
ip-172-31-11-175 opensearch-observability 2.4.1.0
ip-172-31-11-175 opensearch-performance-analyzer 2.4.1.0
ip-172-31-11-175 opensearch-reports-scheduler 2.4.1.0
ip-172-31-11-175 opensearch-security 2.4.1.0
ip-172-31-11-175 opensearch-security-analytics 2.4.1.0
ip-172-31-11-175 opensearch-sql 2.4.1.0
$ curl https://localhost:9200/_cluster/health?pretty -u admin:admin --insecure
{
"cluster_name" : "opensearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"discovered_master" : true,
"discovered_cluster_manager" : true,
"active_primary_shards" : 1,
"active_shards" : 1,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
Is your feature request related to a problem? Please describe. No
Describe the solution you'd like Provide signed (X64)Debian artifacts for OpenSearch / OpenSearch Dashboards
Describe alternatives you've considered Generating the artifacts using the source code
Additional context Currently there are no DEB artifacts for OpenSearch / OpenSearch Dashboards
2526
3090