search

jordansissel / fpm

Effing package management! Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity.
http://fpm.readthedocs.io/en/latest/
Other
11.16k stars 1.07k forks source link

How to extend systemd unit file with fpm? #1412

Open mark-rushakoff opened 7 years ago

mark-rushakoff commented 7 years ago

I don't see a way with fpm or pleaserun to add an arbitrary line to the systemd unit file. I've looked through the issues here on GitHub, I've tried various searches, and I've looked through the code.

In our specific case, we're using fpm to package influxdb, and over at https://github.com/influxdata/influxdb/issues/6068, we have a request to enable notifying systemd when the server is ready to handle requests. Two options for that in the unit file include setting ExecStartPost to a script that blocks until the service is ready, or setting Type=notify and modifying our code to write to that socket when it's ready. The pleaserun service template doesn't have a place for us to add an arbitrary line.

We would have the option to use the ".d drop-in directory" to extend the unit file, but that seems like incorrect packaging if we could emit the desired systemd unit file. If we were to use the drop-in directory, we would still need to place it in the right location, accounting for service name, symlinks, etc.

What next steps would you recommend?

jordansissel commented 7 years ago

@mark-rushakoff my feeling is that any service settings should belong to pleaserun. I'll have to think about how that might get done, but i"m open to ideas :)

Once that happens, fpm's pleaserun support can be updated to enable it.

jordansissel commented 7 years ago

@mark-rushakoff This may be off topic, but my general process is to make programs tolerate dependency outages -- having systemd wait can be useful but it can be a more resilient design to have a program (in the parent ticket, collectd) do that waiting -- otherwise you can still lose data or have undesirable outcomes if influxdb crashes or is restarted by the operator while a dependency is online.

That said, I trust your thoughts on whether or not this works for influxdb and its users.

I'm happy to have whatever is needed to pleaserun and fpm. Extending pleaserun's systemd support to add flags for ExecStartPost and also Type=notify should be pretty easy. If that's all you need, I can probably run with that and see how it goes.

mark-rushakoff commented 7 years ago

@jordansissel I think having both ExecStartPost and Type=notify (plus NotifyAccess) offers a good path for any service to handle growing start times.

The story for us and anyone in a similar situation looks like:

  1. From a clean install or with a relatively small amount of data to load, time-to-ready is short enough that it's a non-issue
  2. With a growing dataset, the process still starts quickly, but it takes time to scan hundreds of gigabytes of data before the process is ready to handle requests
  3. Now aware of the long time-to-ready, ExecStartPost is a good enough "quick fix" solution of a client polling until the service is ready
  4. With a temporary solution in place, we can take the time to properly architect coordination across all our subsystems to use sd_notify to properly indicate when the entire system is ready, as opposed to just when a particular endpoint is ready

In https://github.com/influxdata/influxdb/issues/6068 @phemmer mentioned

And the perfect solution would be if influxdb supported systemd socket passing.

I haven't yet wrapped my head around systemd socket passing or why it's a better solution than socket notification.

mark-rushakoff commented 7 years ago

It's just been brought to my attention that we are using a custom unit file already: https://github.com/influxdata/influxdb/blob/master/scripts/influxdb.service

We copy it in via the file system around here in the code: https://github.com/influxdata/influxdb/blob/c80c3636dd7bba6c3d0f3d6a1c321862da8f201a/releng/packages/fs/usr/local/bin/influxdb_packages.bash#L64-L109

So, I'm not sure what that means for this issue. For influxdb we no longer require further customization of the unit file via fpm or pleaserun. Maybe that feature would be useful to other fpm users, or maybe they would be better off taking the stock unit file generated by pleaserun and editing it by hand.

jordansissel commented 7 years ago

Cool! I'm happy y'all have this solved already, and I also agree this may still be useful feature for fpm/pleaserun.