search

Seagate / openSeaChest

Cross platform utilities useful for performing various operations on SATA, SAS, NVMe, and USB storage devices.
Other
479 stars 61 forks source link

Create official packages for Debian-based distros #104

Closed KOLANICH closed 1 year ago

vonericsen commented 1 year ago

Hi @KOLANICH, Can you give me a few more details on what you mean in this request?

KOLANICH commented 1 year ago

I mean the following can be implemented (in the order of increased effort and usability):

  1. scripts to generate *.deb package files for Debian-based distros
  2. publish the debs in releases
  3. publish a repo on GitHub Pages that can be consumed by apt
  4. upstream the packages for the tools into Debian official repos and maintain them. After some time this should make them appear in the repos of distros derived from Debian.
vonericsen commented 1 year ago

@KOLANICH, Thank you for elaborating! As of v23.03 we do have a .deb package available that was generated by the CI for this project. Currently that is only generated in the CI and not with a standalone script, so we can look into adding a script for anyone building from source.

We will look into the rest of this to see how we can work towards these things. I think that is a good list for us to reference as steps.

vonericsen commented 1 year ago

Answered a question from my previous comment. Debian has this useful page we can use as reference for the upstream of the package: https://wiki.debian.org/UpstreamGuide

alexmyczko commented 1 year ago

you are welcome: http://sid.ethz.ch/debian/openseachest/

vonericsen commented 1 year ago

Hi @alexmyczko, This is awesome to hear about! I do have some questions though to help me understand this a bit better.

From what I can tell this by the link, this is for Debian sid (unstable). Is this an official mirror, or in an official place where it will be pulled into a future debian release or update? If I were to install/setup the unstable repo sources, could I install this with apt today? I would like to try this as soon as I get enough time to setup a system or VM to see how this works.

If you were the one who created the package at that link, were there any issues you had or things we can address in openSeaChest to make things easier for the future?

When I was reading through the upstream guide I did take some notes on some things I thought we could improve and review:

KOLANICH commented 1 year ago

I saw a COPYING file is more common than license.md

COPYING is traditionally used with GPL-family licenses, and with other ones (except Unlicense, for which UNLICENSE is traditionally used) License is usually used.

alexmyczko commented 1 year ago

Hi @alexmyczko, This is awesome to hear about! I do have some questions though to help me understand this a bit better.

From what I can tell this by the link, this is for Debian sid (unstable).

It's a debian source package, that anyone can dget urlto.dsc then, dpkg-source -x the.dsc, cd dir*/ and debuild to create binary packages of your system, no matter if you have sid, debian, ubuntu, arm64 or what...

Is this an official mirror, or in an official place

nothing official

where it will be pulled into a future debian release or update? If I were to install/setup the unstable repo sources, could I install this with apt today? I would like to try this as soon as I get enough time to setup a system or VM to see how this works.

No, but instructions above should get you started...

If you were the one who created the package at that link, were there any issues you had or things we can address in openSeaChest to make things easier for the future?

Yes i created the package at that link. Well let me check that on Monday...

When I was reading through the upstream guide I did take some notes on some things I thought we could improve and review:

* better man pages. Currently I generate these with help2man, but I know this is just an easy way to get a man page, not necessarily the best way. This is also something that looks relatively easy to improve on.

help2man is fine, but the parameters given/parsing was wrong, should be easy to fix...

* Review that our makefiles meet the upstream requirements. I believe they do, but I know there are lots of areas to improve them already, so this could be a good time to review and make these improvements. I'm listing this since I did not see meson listed as a supported build system.

That worked fine... but what's Make about it? it's meson/ninja-build? 

* I saw the openPGP signing requirements for the package and plan to look into doing this for the future

There's no such requirements, forget about it...

* I saw a `COPYING` file is more common than license.md and this would be easy to create for the tarball. I also saw recommendation of including a changelog file. I think these are also reasonably easy to add to openSeaChest

...

alexmyczko commented 1 year ago 
W: openseachest: bad-whatis-entry [usr/share/man/man8/openSeaChest.8.gz]
W: openseachest: bad-whatis-entry [usr/share/man/man8/openSeaChest_Format.8.gz]
W: openseachest: no-manual-page [usr/bin/openSeaChest_FormatUnit]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_Basics.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_Configure.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_Erase.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_Firmware.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_GenericTests.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_Info.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_Logs.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_NVMe.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_PassthroughTest.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_PowerControl.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_SMART.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_Security.8.gz]
W: openseachest: useless-whatis-entry [usr/share/man/man8/openSeaChest_ZBD.8.gz]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_Basics.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_Configure.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_Erase.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_Firmware.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_GenericTests.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_Info.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_Logs.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_NVMe.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_PassthroughTest.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_PowerControl.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_SMART.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_Security.8.gz:2]
W: openseachest: wrong-manual-section 8 != 1 [usr/share/man/man8/openSeaChest_ZBD.8.gz:2]
vonericsen commented 1 year ago

@alexmyczko, Thank you for the feedback!

I will see if I can figure out what happened with help2man for now, but I think writing a man page will come out better in the end, so I will queue that up for the future.

That worked fine... but what's Make about it? it's meson/ninja-build?

We have makefiles as well as meson/ninja in the Make/gcc folder for the project. BUILDING_OLD.md talks about that method. I didn't see meson listed as a supported build system, which is the only reason I made this comment. Meson/ninja is the preferred method of building openSeaChest at this time, but there are some cases where GNU make may work better. For example, I could not use meson on the AIX box I had access to and could only use GNU make for that system.

alexmyczko commented 1 year ago

Telling help2man to use Section 8, suppress pointer to Texinfo manual, and give a whatis entry should be enough: help2man -s8 -N -n="drive utilities" openSeaChest_Info > openSeaChest_Info.8

I'm not very happy to have that many binaries with that large size:

-rwxr-xr-x 1 root root 4419624 Mar  4 22:48 openSeaChest_Basics
-rwxr-xr-x 1 root root 4244144 Mar  4 22:48 openSeaChest_Configure
-rwxr-xr-x 1 root root 4338304 Mar  4 22:48 openSeaChest_Erase
-rwxr-xr-x 1 root root 4154320 Mar  4 22:48 openSeaChest_Firmware
-rwxr-xr-x 1 root root 4273240 Mar  4 22:48 openSeaChest_FormatUnit
-rwxr-xr-x 1 root root 4341992 Mar  4 22:48 openSeaChest_GenericTests
-rwxr-xr-x 1 root root 4570776 Mar  4 22:48 openSeaChest_Info
-rwxr-xr-x 1 root root 4232136 Mar  4 22:48 openSeaChest_Logs
-rwxr-xr-x 1 root root 4323672 Mar  4 22:48 openSeaChest_NVMe
-rwxr-xr-x 1 root root 4553520 Mar  4 22:48 openSeaChest_PassthroughTest
-rwxr-xr-x 1 root root 4331608 Mar  4 22:48 openSeaChest_PowerControl
-rwxr-xr-x 1 root root 4169944 Mar  4 22:48 openSeaChest_Security
-rwxr-xr-x 1 root root 4782208 Mar  4 22:48 openSeaChest_SMART
-rwxr-xr-x 1 root root 4153656 Mar  4 22:48 openSeaChest_ZBD

and the capital lettering. would it be possible to put those into /usr/libexec/openseachest and just have one binary say openseachest that takes a parameter zbd or smart and would call the individual /usr/libexec/openseachest/binary ? That'd also save you to auto generate that many man pages of low quality, reducing your work to just one single manual page of higher quality?

about build systems: i was hoping for cmake, as that appears to be most popular, but your software, your choice

the description mentions a GUI, where is that?

vonericsen commented 1 year ago

@alexmyczko, Thanks for the command info. I am planning to work on it this week as I have already had a couple of internal bug reports I need to look into.

I'm not very happy to have that many binaries with that large size... and the capital lettering. would it be possible to put those into /usr/libexec/openseachest and just have one binary say openseachest that takes a parameter zbd or smart and would call the individual /usr/libexec/openseachest/binary ?

I have thought about changing this and have a document I started with ideas on doing this. It would remove the multiple binaries into a single tool with a required parameter to specify which sub-tool to run: ex: openSeaChest basics -d <handle> --shortDST --poll or openSeaChest smart -d <handle> --smartAttributes I can look into changing the camel-case to lowercase as well since I know this is a bit more effort to work with, especially when typing a lot for a CLI. I was not sure if this is something others would like and have discussed it only a little internally at this point, but I think it would simplify things for many people to only interact with a single binary, but that is a significant change that I wanted to make sure is actually desireable...so I'm happy to hear it is. I started poll #106 to see if I can get any other feedback on this since I think this could be a good long-term change to work on. You script idea could be a nice bridge between these for now too. I can look into that and use that as an example to help others understand this possible change.

about build systems: i was hoping for cmake, as that appears to be most popular, but your software, your choice

We were looking into CMake when we received #61 and so we took it and ran with it....I've really like meson although I know it is a newer project without the same level of support and popularity as CMake right now.

the description mentions a GUI, where is that?

openSeaChest does not have a GUI. That statement is referencing the SeaTools application on the Seagate website. I reviewed the text and can see why that may be confusing. I can look into how to change that a bit and make it more clear it is not part of this toolset, but another offering on Seagate's website. I've been thinking about rewriting the README.md file for a while now to try and clean it up a bit and this would be one of those things I can work on. Documentation is an area that seems to keep coming up and I think I can take some time to work on that.

I really appreciate all the feedback. This is exactly the kind of information I was wanting to know so we can continue improving this project for anyone who is using it.

KOLANICH commented 1 year ago

I propose the following - a single command line tool, hut the commands are plugins as shared libraries and a dynamic discoverer using some well-known libs that are abstraction layers around OS-specific API for doing those. Like I have done in my HydrArgs/TestAbs/FileTestSuite (C++ version).

meson The killer feature of CMake is its toolbox. There are some libs parsing CMake code in Python, around it I had made an interpreter, but it was made very rudimentary just to extract info from the files generated for CPack before CMake (for my metabuild system, BTW I used to have a repo with packages in GitLab CI before my metabuild system got broken) got the feature to export them as JSON. I guess to migrate to Meson it should contain an impl of CMake language in Python within itself, allowing interaction between CMake and Python code. This way allowing to use the batteries of CMake. The devil is that CMake has a lot of features implemented within its fore in C.

alexmyczko commented 1 year ago

http://sid.ethz.ch/debian/openseachest/2023/ looks much better...

E: openseachest: helper-templates-in-copyright
W: openseachest: bad-whatis-entry [usr/share/man/man8/openSeaChest.8.gz]

https://ftp-master.debian.org/new/openseachest_23.03.1-1.html

vonericsen commented 1 year ago

That is better. 😃 I am making progress on it, but it is slow at the moment due to other internal requests I'm also working on. I will keep tagging commits as I make more progress working towards this.

alexmyczko commented 1 year ago

rejected with

please also mention at least:
 openSeaChest-v23.03.1/subprojects/opensea-transport/include/of_nvmeIoctl.h
 openSeaChest-v23.03.1/subprojects/wingetopt/*
in your debian/copyright.
vonericsen commented 1 year ago

That is important to catch. I will make sure the license files are updated in the appropriate places.

alexmyczko commented 1 year ago

this time it looks better https://ftp-master.debian.org/new/openseachest_23.03.1-1.html

alexmyczko commented 1 year ago

@KOLANICH let’s close this issue?

alexmyczko commented 1 year ago

@vonericsen https://buildd.debian.org/status/package.php?p=openseachest

can i contact you by mail somehow?

KOLANICH commented 1 year ago

@KOLANICH let’s close this issue?

apt list openseachest Listing... Done

https://packages.debian.org/search?keywords=openseachest Sorry, your search gave no results

Not landed yet. So IMHO it is too early to close it before apt install openseachest started working for sid or experimental.

alexmyczko commented 1 year ago

@KOLANICH it was a matter of hours, now you can...

KOLANICH commented 1 year ago

:tada: Thanks to @vonericsen, @alexmyczko and everyone else involved into it. I guess we would like to put some badges for major distros from https://repology.org/project/openseachest/badges into the ReadMe.

vonericsen commented 1 year ago

Sorry for the long delay! My daughter was born not long after the last messages in this thread and I've been on leave. I'm really excited to come back and see openSeaChest in the repology list for debian and ubuntu among others! I pushed an update to the readme file so that this is now viewable on the main openSeaChest page. Thanks to @KOLANICH and @alexmyczko for your help with this issue!