olivierlambert commented 6 years ago

We should be able to allow people to download directly some "extra" packages that can be really useful.

The only downside is that we'll need to synchronize any package update from CentOS to XCP-ng.

Monitoring

General

htop (prettier than top): no extra deps
iftop (network top): no extra deps
iperf (handy to check network issues): no extra deps
netdata (this RPM seems to work)

Security

wireguard (that would be VERY nice to avoid exposing XAPI publicly for "Cloud" setup)

UPS

apcupsd
nut

Misc

vim
mc

If you have ideas of stuff that you'll like, let us know here

mkninc commented 6 years ago

For UPS monitoring

apcupsd
nut

olivierlambert commented 6 years ago

@mkninc thanks for the feedback, can you check if those RPMs on CentOS 7 require external deps not already present in XCP-ng?

mkninc commented 6 years ago

===========================================================================================================
 Package                 Arch                   Version                         Repository            Size
===========================================================================================================
Installing:
 apcupsd                 x86_64                 3.14.14-5.el7                   epel                 305 k
Installing for dependencies:
 libusb                  x86_64                 1:0.1.4-3.el7                   base                  19 k

===========================================================================================================
 Package                   Arch                  Version                         Repository           Size
===========================================================================================================
Installing:
 nut                       x86_64                2.7.2-4.el7                     epel                1.6 M
Installing for dependencies:
 freeipmi                  x86_64                1.5.7-2.el7                     base                2.0 M
 libusb                    x86_64                1:0.1.4-3.el7                   base                 19 k
 nut-client                x86_64                2.7.2-4.el7                     epel                206 k

borzel commented 6 years ago

vim
mc ("graphical" file browser)

stormi commented 6 years ago

I love mc, totally agree with the suggestion :)

stormi commented 6 years ago

40 was asking for cryptosetup, too.

olivierlambert commented 6 years ago

Good catch, let's close the other issue to centralize requests here :+1:

adnae commented 6 years ago

Hi, one mistake ;) Cryptosetup is cryptsetup ;)

But I have one more package as request ... nrpe

That is nice for an external montoring

borzel commented 6 years ago

monitoring: check-mk-agent (and xinetd and maybe changes at the firewall rules for port 6556)

jcharaoui commented 6 years ago

yum-utils and yum-plugin-versionlock for easier package/repository management

stormi commented 6 years ago

We had added traceroute in #32 but that was in the old repo, so we need to add it again.

fetzerms commented 6 years ago

I usually install cryptsetup manually from the centos base repos on XS7 and it works fine.

Also:

monitoring: munin-node

gdelafond commented 6 years ago

Troobleshooting:

htop

iftop

iperf

tcpdump

nload

mtr

sysstat (iostat, mpstat, sar)

dstat

nethogs

Monitoring:

netdata

net-snmp-utils

nagios-plugins

Hardware RAID tools:

tw_cli
storcli

Editors

vim

emacs-nox

Security

fail2ban
ferm

Misc

hdparm
pv
lshw
dnsutils

oallart commented 6 years ago

locate - useful to ...locate things tmux - screen too, but tmux better for terminal sessions

yctn commented 6 years ago

bwm-ng small and simple console-based bandwidth monitor iptraf Interactive Colorful IP LAN Monitor iotop simple top-like I/O monitor shorewall(firewall)

PeS-hkfree commented 6 years ago

Hi, we are heavilly using nmon - nmon.sourceforge.net have binaries for RH/CentOS 7.2, works well.

stormi commented 6 years ago

The list is becoming overwhelming and I see no reason to reject any package from it (except if they require libs that would overwrite those of XCP-ng).

I'm considering a new approach, because I don't think maintaining a significant portion of the CentOS repos in ours will make sense in the long run. Actually two new approaches:

1) Add CentOS and EPEL repos to XCP-ng but limit them to a whitelist. This means that every additional package will have to be approved manually but without the need for us to maintain the package afterwards, leaving this to RedHat and CentOS people who already do a good job.

2) Add unrestricted CentOS and EPEL repos, using the yum-plugin-priorities plugin, with the highest priority given to XCP-ng packages. This would allow installing anything from CentOS and EPEL provided they don't try to overwrite a package from XCP-ng. I'm using this yum plugin in xcp-ng-build-env for similar reasons and it seems to be working well.

In both cases I would have to rethink the yum upgrade process a little bit because we would need to update the CentOS and EPEL repository files before the upgrade, but I think I can make it work quite easily with just an extra step to the procedure (= update xcp-ng-release before updating the other packages).

gdelafond commented 6 years ago

The list is becoming overwhelming and I see no reason to reject any package from it (except if they require libs that would overwrite those of XCP-ng).

True.

I'm considering a new approach. Actually two new approaches, because I don't think maintaining a significant portion of the CentOS repos in ours will make sense in the long run.

I agree.

1. Add CentOS and EPEL repos to XCP-ng but limit them to a whitelist. This means that every additional package will have to be approved manually but without the need for us to maintain the package afterwards, leaving this to RedHat and CentOS people who already do a good job.

It will require more work for the XCP-ng Team, but all packages could be tested (and upgrades with them installed too). Many question in this case:

will the list be fully open or be limited by criteria (wich ones)?
who will be responsible to update this list ? update frequency ?

2. Add unrestricted CentOS and EPEL repos, using the yum-plugin-priorities plugin, with the highest priority given to XCP-ng packages. This would allow installing anything from CentOS and EPEL provided they don't try to overwrite a package from XCP-ng. I'm using this yum plugin in xcp-ng-build-env for similar reasons and it seems to be working well.

It will be more difficult to reproduce problems as everybody will be able to install any packages, but people will be really free to do what they want. In this case, packages known to be working could be put on a list in the wiki, with a warning telling that others have not been tested ?

stormi commented 6 years ago

1. Add CentOS and EPEL repos to XCP-ng but limit them to a whitelist. This means that every additional package will have to be approved manually but without the need for us to maintain the package afterwards, leaving this to RedHat and CentOS people who already do a good job.

It will require more work for the XCP-ng Team, but all packages could be tested (and upgrades with them installed too). Many question in this case:

* will the list be fully open or be limited by criteria (wich ones)?

The criteria that I can think of right now: not overwriting (or depending on packages that would overwrite) existing XCP-ng packages

* who will be responsible to update this list ? update frequency ?

Me at first, then if people join to share the work this could probably be maintained collectively. Update frequency: at least once every big release, probably more often. It mainly depends on the test effort people are willing to provide.

2. Add unrestricted CentOS and EPEL repos, using the yum-plugin-priorities plugin, with the highest priority given to XCP-ng packages. This would allow installing anything from CentOS and EPEL provided they don't try to overwrite a package from XCP-ng. I'm using this yum plugin in xcp-ng-build-env for similar reasons and it seems to be working well.
It will be more difficult to reproduce problems as everybody will be able to install any packages, but people will be really free to do what they want. In this case, packages known to be working could be put on a list in the wiki, with a warning telling that others have not been tested ?

Yes

ManicoW commented 6 years ago

I have a specific request as the wanted package is not a tool, but a dependency for a tool.

I'm trying to install salt-minion in order to join an xcp-server to our saltstack master, which is used for hundred of servers. I follow this guide : https://repo.saltstack.com/#rhel

But salt has a few dependencies that are not satisfied :

python-backports-ssl_match_hostname , which can be installed from base centos without difficulties
systemd-python, which cannot be installed from base, as systemd-* packages are in the exclude list un yum centos repos configuration

As far as I see, It seems that systemd is not the same version as in centos classic distribution, so I think that the better solution is to add this package in xcp-ng repositories/iso. As the source package for systemd-python is systemd, I think that it can be generated with the whole systemd-* packages ?

update

As stormi said me on IRC, this packages is in fact available, in builddeps for 7.5, and will be in base in 7.6. So my question is already answered. Thanks to him

stormi commented 5 years ago

Adding deltarpm as mentioned in #120

stormi commented 5 years ago

nvme-cli, as mentioned in https://xcp-ng.org/forum/topic/908/smartmontools

xianglo commented 5 years ago

Netdata. For the love of data.

jotihojr commented 5 years ago

yum-cron, automatic security updates

adhawkins commented 5 years ago

joe - A simple text editor

stormi commented 5 years ago

Current wishlist, with additional information for each package.

Troubleshooting

package	provenance	added dependencies	action
htop	epel	none	installed by default (8.0)
iftop	epel	none	installed by default (8.0)
iperf	epel	none	added to repo (8.0)
iperf3	centos	none	added to repo (8.0)
traceroute	centos	none	added to repo (8.0)
~~tcpdump~~	already installed	none	-
nload	epel	none	added to repo (8.0)
mtr	centos	none	added to repo (8.0)
~~sysstat~~	already installed	none	-
dstat	centos	none	added to repo (8.0)
nethogs	epel	none
bwm-ng	epel	none
iptraf-ng	centos	none
iotop	centos	none	added to repo (8.0)
nmon	epel	none
pv	epel	none
fio	centos	several

Monitoring

package	provenance	added dependencies
netdata	needs packaging	?
nrpe	epel	nagios-common
check-mk-agent	epel	GeoIP,bind-libs,bind-utils,bind-libs-lite,bind-license
munin-node	epel	62 packages
net-snmp-utils	centos	net-snmp,net-snmp-agent-libs,net-snmp-libs
nagios-plugins	epel	nagios-common

Security

package	provenance	added dependencies	action
wireguard	needs packaging	?
cryptsetup	centos	cryptsetup-libs	installed by default (8.0)
fail2ban	epel	a lot
ferm	epel	none

Hardware RAID tools

package	provenance	added dependencies	action
tw_cli	needs packaging	?
storcli	needs packaging, a RPM exists on broadcom's website	none at runtime

UPS

package	provenance	added dependencies	action
apcupsd	epel	libusb
nut	epel	freeipmi,libtool-ltdl,libusb,nut-client

Other hardware-related tools **

package	provenance	added dependencies	action
~~hdparm~~	already installed	none	-
dc3dd	epel	none
nvme-cli	centos	none	added to repo (8.0)
lshw	centos	none	added to repo (8.0)

Editors

package	provenance	added dependencies	action
vim-enhanced	centos	gpm-libs,vim-common,vim-filesystem	added to repo (8.0)
emacs-nox	centos	8 packages
joe	epel	none	added to repo (8.0)

Misc

package	provenance	added dependencies	action
deltarpm	centos	none
dnsutils	needs packaging	?
mlocate	centos	none, but doesn't it add I/O overhead?
mc	centos	gpm-libs	added to repo (8.0)
socat	centos	none	added to repo (8.0)
tmux	centos	none	added to repo (8.0)
wpa_supplicant	centos	none	added to repo (8.0 updates)
yum-cron	centos	none	available in repo as part of yum suite, but I don't advise using it! (8.0)
yum-utils	centos	none	installed by default (8.0)
yum-plugin-versionlock	centos	none	available in repo as part of yum suite, but I don't advise using it! (8.0)

stormi commented 5 years ago

Suggested on the forum: socat, useful to open VNC access

gdelafond commented 5 years ago

For netdata, there are official packages now : https://packagecloud.io/netdata/netdata

gdelafond commented 5 years ago

dc3dd is usefull to wipe a disk (it is like dd if=/dev/zero but with live status & more options)

gdelafond commented 5 years ago

For netdata, there are official packages now : https://packagecloud.io/netdata/netdata

and netdata is present in epel repo too.

olivierlambert commented 5 years ago

Problem is that netdata official package doesn't have Xen stuff IMHO

stormi commented 4 years ago

A small comment since the last comment above isn't true anymore: we now provide netdata, either through Xen Orchestra (hosts will stream the data towards XO, that's the recommended setup because it doesn't write anything on host disks and uses little memory) or as a standalone installation (netdata-ui from our repositories).

rjt commented 4 years ago

Link Layer Discovery Protocol packages lldpd or lldpad. My hunch is that lldpd is better and provides lldpcli, but have not used them as much.

https://xcp-ng.org/forum/topic/2871/would-like-link-layer-discovery-protocol-on-xcp-hosts-any-downsides-that-might-come-up

stormi commented 4 years ago

lldpad is available as a CentOS package and has no additional external dependency so I could include it easily.
lldpd is available as an EPEL package and has no additional external dependency so I could include it easily too.

rjt commented 4 years ago

The OS performance tuning package "tuned" provides specific options. Would "virtual-host" optimize performance of xcp-ng like it does for KVM?

$ tuned-adm list
Available profiles:
- balanced                    - General non-specialized tuned profile
- desktop                     - Optimize for the desktop use-case
- hpc-compute                 - Optimize for HPC compute workloads
- latency-performance         - Optimize for deterministic performance at the cost of increased power consumption
- network-latency             - Optimize for deterministic performance at the cost of increased power consumption, focused on low latency network performance
- network-throughput          - Optimize for streaming network throughput, generally only necessary on older CPUs or 40G+ networks
- powersave                   - Optimize for low power consumption
- throughput-performance      - Broadly applicable tuning that provides excellent performance across a variety of common server workloads
- virtual-guest               - Optimize for running inside a virtual guest
- virtual-host                - Optimize for running KVM guests

Suspect tuned-adm still requires hand-holding when executed on CentOS as mentioned by this RHEL employee in Boosting CentOS server performance

stormi commented 4 years ago

I'm not sure that tuned tool would be adequate for dom0. dom0 is already supposed to be optimized for virtualization. If it isn't, that is something we should fix in dom0 directly, not rely on users running an external tool to tweak things and risk break it.

rjt commented 4 years ago

Yes, tuned would be strictly for testing to find out which parameters to investigate. So would it even make sense to have as a permanent member of a testing repository and never make it to a normal repository.

stormi commented 4 years ago

Yes, tuned would be strictly for testing to find out which parameters to investigate. So would it even make sense to have as a permanent member of a testing repository and never make it to a normal repository.

Then you can consider that it's already the case since you can install it with a single command:

yum install tuned --enablerepo=base,updates

This will pull several dependencies and I clearly discourage anyone from doing it on a production server.

stormi commented 4 years ago

* `lldpad` is available as a CentOS package and has no additional external dependency so I could include it easily.

* `lldpd` is available as an EPEL package and has no additional external dependency so I could include it easily too.

Actually lldpad is already present in XCP-ng and installed by default.

rjt commented 4 years ago

lldpcli in lldpd is much more useful for the client side ops.

  `lldpcli neighbors`

On Wed, Apr 29, 2020 at 9:23 AM Samuel VERSCHELDE notifications@github.com wrote:

lldpad is available as a CentOS package and has no additional external dependency so I could include it easily.

lldpd is available as an EPEL package and has no additional external dependency so I could include it easily too.

Actually lldpad is already present in XCP-ng and installed by default.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/xcp-ng/xcp/issues/56#issuecomment-621244062, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACX7FZQXUOBNLF4OGVKXQLRPAZ53ANCNFSM4FRAM2CA .

stormi commented 4 years ago

lldpad in XCP-ng is patched, with a patch named dont-add-xs-vif-to-lldpad.patch. Which makes me wonder if lldpd could need such patching in order not to cause any harm when used.

ghost commented 4 years ago

Hi,

please add:

    apcupsd
    nut

would be great so I could control my eaton ups :-)

stormi commented 4 years ago

I'm planning to do it... Sooner or later :)

ghost commented 4 years ago

OK cool thanks!

jibcage commented 4 years ago

initramfs-tools and/or dracut if possible.

nagilum99 commented 4 years ago

What's the benefit for XCP-ng? (What do you need/use it for?)

stormi commented 4 years ago

dracut is already present

adhawkins commented 4 years ago

Any chance of adding 'tuptime'? It's a python script for monitoring and reporting on uptime.

stormi commented 4 years ago

Any chance of adding 'tuptime'? It's a python script for monitoring and reporting on uptime.

Hi. Probably not, unless many users ask for it, because I can't find any package named tuptime in CentOS or EPEL repositories.

adhawkins commented 4 years ago

Any chance of adding 'tuptime'? It's a python script for monitoring and reporting on uptime.

Hi. Probably not, unless many users ask for it, because I can't find any package named tuptime in CentOS or EPEL repositories.

There are some packages here: https://copr.fedorainfracloud.org/coprs/frankcrawford/tuptime/

Not sure about their status.

Andy

xcp-ng / xcp

List of useful extra packages for dom0 #56

Monitoring

General

Security

UPS

Misc

40 was asking for cryptosetup, too.

update

Troubleshooting

Monitoring

Security

Hardware RAID tools

UPS

Other hardware-related tools **

Editors

Misc