Support for doctools, pkgtools & devtools?

[ghost]

I have a working copy of the devtools+pkgtools in a repository I created (https://github.com/hex7c/pkgtools). Is it possible I could fork this repository, and add the devtools, pkgtools, & doctools? If so, how should I structure the layout and where should the binaries be deposited? Also I've only tested the devtools+pkgtools on a recent Glibc, so if there are any issues with other libc's and systems, it would be nice to know.

Looking to hear feedback and hoping to contribute, thanks.

[takusuman]

First of all: Wow, does Heirloom's pkgtools work in modern Linux? That's amazing to hear, although the Pindorama project (in other words, we) have already started doing some work in a rewrite in Go called Motoko.

Is it possible I could fork this repository, and add the devtools, pkgtools, & doctools?

Well, that's an interesting question, to be honest.
I always thought in this repository being dedicated to Heirloom Toolchest, but it could be useful to add more utilities. So, yes, it would be good if you could add a working port of these utilities and complete the Heirloom project as itself, but maybe it could be in another directory such as extra/? Or even be mixed along with the "default system tools"? Or maybe still using the good old, original directory style that Ritter used back in the day, with everything in subdirectories? I don't know yet, it would be interesting to hear a suggestion. The only problem when messing with the distribution tarball directory structure would be breaking Copacabana's (and, possibly, other distributions) patches, I think.

By the way, yes, we can test it on musl via the Copacabana Linux distribution, but not on other UNIXes for now.

[ghost]

Wow, does Heirloom's pkgtools work in modern Linux?

Yup, at least for Glibc. If you have pkginfo, neofetch will detect the number of packages too!

[...] have already started doing some work in a rewrite in Go called Motoko.

That's really cool to hear, I'll need to check that out.

The only problem when messing with the distribution tarball directory structure would be breaking Copacabana's (and, possibly, other distributions) patches, I think.

Do you think I should create a new repository called heirloom-extra-ng,or maybe name it after the original names such as heirloom-doctools-ng, heirloom-devtools-ng, etc.? Or would you wish to create it (if you're even interested, that is)? I believe it'd be nice to maintain these tools not only for historical/documentary purposes but also potentially to actually get them usable in the modern day, at least somewhat.

I believe it'd be a really interesting project, and I'm also looking for something to contribute to.

If you want, I can work on restructuring the repositories with my edits to them to be closer to this one's structure, and I could try to upload it as a gzipped tarball or a new repository or something along the lines of those.

Thanks for the quick feedback :smile:

[takusuman]

Do you think I should create a new repository called heirloom-extra-ng,or maybe name it after the original names such as heirloom-doctools-ng, heirloom-devtools-ng, etc.? Or would you wish to create it (if you're even interested, that is)? I believe it'd be nice to maintain these tools not only for historical/documentary purposes but also potentially to actually get them usable in the modern day, at least somewhat.

I think we could modify the directory structure, throwing the current files into a heirloom-ng directory and then unpacking the doctools into a, let's say, doctools-ng directory and so on. The release tarballs itself could be generated by a script that packs every subdirectory separately, so we won't be breaking anything.

And sure, we're interested in the doctools for Copacabana! It's a good replacement for groff. Not so sure about pkgtools since we have Motoko being worked on, but since it's for historical reasons, I don't see a problem on it --- and, since Motoko aims to be at least compatible with the original pkgtools' """API""", we could use them as a "sanity test".

Thanks for the quick feedback 😄

In portuguese, we usually say "Não há de quê!", that would be an equivalent to "You're welcome!". So, não há de quê!

[takusuman]

By the way, I know it may sound silly, but we needed that someone helped writing the chroot(8) manual page. I'm not so good at writing. 😆

[ghost]

I think we could modify the directory structure, throwing the current files into a heirloom-ng directory and then unpacking the doctools into a, let's say, doctools-ng directory and so on.

Alright, so keep it all in one repository? If so, I'll fork and begin working on this and provide a pull request soon.

By the way, I know it may sound silly, but we needed that someone helped writing the chroot(8) manual page.

I'm definitely not the best at writing, but I sure will give it a try. I'll look at the other manual pages in the repo for inspiration and writing style & I'll include the man page in the same PR/repository. Note that I am not multilingual, so I can only write in Americanized English.

[ghost]

Note that I'll only include the doctools for now unless you otherwise specify. The other stuff wouldn't be used nearly as much I presume so I'll leave it out for now.

[takusuman]

Alright, so keep it all in one repository? If so, I'll fork and begin working on this and provide a pull request soon.

Yeah, it seems to be the best option for now. I'll write a simple script later for generating the release tarballs for every, let's say, "set" of tools --- idk a better name to call it, so let's call it "set" or now. 🤷🏽‍♂️

I'm definitely not the best at writing, but I sure will give it a try.

Thanks!

Note that I'll only include the doctools for now unless you otherwise specify. The other stuff wouldn't be used nearly as much I presume so I'll leave it out for now.

Alright

[ghost]

Also, a few last comments/concerns:

• Is there any style of commit messages I should follow?
• How should I add copyright info to files that have been edited by me?
• And should I change the copyright headers to the SPDX ID like other commits have? If so, is there an automated tool I can use or do I have to identify it myself?

[takusuman]

• Is there any style of commit messages I should follow?

We have been using a Conventional Commits-like standard, but tch69 have used a "<file being edited>: Description message" format, which is good too.

• How should I add copyright info to files that have been edited by me?

Well, you can just add your name above the original copyright holder, just like @arthurbacci did in this commit at pwd.c.

• And should I change the copyright headers to the SPDX ID like other commits have? If so, is there an automated tool I can use or do I have to identify it myself?

Yeah, it would help a lot, even for editing the files later (since it would reduce its size in a dozen of lines). Unfortunately, there's not a automated tool for doing this, I had to do it using sed and ex until I got the result that I needed --- and some files were even broken at the end, for instance see this commit at cpio/inflate.c. The only thing I remember doing right was deleting the lines between the patterns that started and ended the licence text. I'm not nearly as good with RegEx, so maybe I "overworked" this a lot.

[ghost]

Understood. Again, thanks for taking a look at this. I'll get started on this.

[ghost]

Update: If I move the current files & directories to a subfolder, Git DELETES all of the file history and replaces it with just the latest commit. Not the optimal solution, to say the least. I am going to see if I can merge the doctools into the current structure without conflicting anything. This will take more work than I expected.

[takusuman]

Update: If I move the current files & directories to a subfolder, Git DELETES all of the file history and replaces it with just the latest commit. Not the optimal solution, to say the least. I am going to see if I can merge the doctools into the current structure without conflicting anything. This will take more work than I expected.

In fact, it doesn't when you take a look in some Git hosting platform such as GitHub, it shows like it was renamed, at least by my experience.

[ghost]

In fact, it doesn't when you take a look in some Git hosting platform such as GitHub, it shows like it was renamed, at least by my experience.

Interesting. Think I should still continue with merging the sources instead of separating them into subdirectories?

[takusuman]

Interesting. Think I should still continue with merging the sources instead of separating them into subdirectories?

It would be better to separate them between directories, at least for packaging later.

[takusuman]

Oh, I also remembered: after done moving, you have to git add ..

[ghost]

Well, it appears I've discovered a group of people have already done the work. I believe it's better to point to their repository rather than re-inventing the wheel, as it's actively maintained. https://github.com/n-t-roff/heirloom-doctools

I just tested it; it does not work with GNU bison, however installing byacc and running export YACC; ./configure && make compiled it just fine.

Thoughts?

I did however make a few small changes; you can see them here: https://github.com/hex7c/heirloom-ng/commits?author=hex7c

• I created the man page for chroot(8), added a comment to chroot.c and chroot.8 indicating a potential future option (since GNU and *BSD has a default chroot command, being /bin/sh -i or $SHELL)
• Added a header to awk.g.y because it wouldn't compile on Arch Linux w/ glibc 2.36
• Added myself to copyright

If this sounds good, I'll close this issue and file a pull request.

[ghost]

Also, if there is anything else I can help with that doesn't belong in this issue (aside from big parts of C code; I'm not super good at it yet), don't hesitate to e-mail me because I really like the heirloom tools due to their heritage & history. (I've wanted to use them for a while on a custom install for a long time now. If you find any non-GNU stuff for ld and other binutils, let me know cause I wanna check it out.)

[ghost]

I'm going to look into modernizing the shell scripts with help from shellcheck. Many have legacy backticks & other issues that can be been with shellcheck, and I believe that since we are in 2022, legacy shell features shouldn't be used as there isn't much reason to, aside from supporting shells that are older than I am.

[takusuman]

Well, good morning. Sorry for the delay.

Well, it appears I've discovered a group of people have already done the work. I believe it's better to point to their repository rather than re-inventing the wheel, as it's actively maintained. https://github.com/n-t-roff/heirloom-doctools

Yeah, I remember those guys. I think it's the original Heirloom team that started maintaining doctools separately, but I'm not sure.

I just tested it; it does not work with GNU bison, however installing byacc and running export YACC; ./configure && make compiled it just fine.

Thoughts?

Are you talking about heirloom-ng or doctools? Well, at least in my machine (currently Ubuntu 22.04) and in Copacabana Linux, heirloom-ng can compile with bison -y as YACC. I hadn't tested with doctools on Copacabana yet.

I did however make a few small changes; you can see them here: https://github.com/hex7c/heirloom-ng/commits?author=hex7c

Sure, I'll be checking them out. :smile:

• I created the man page for chroot(8), added a comment to chroot.c and chroot.8 indicating a potential future option (since GNU and *BSD has a default chroot command, being /bin/sh -i or $SHELL)
• Added a header to awk.g.y because it wouldn't compile on Arch Linux w/ glibc 2.36
• Added myself to copyright

If this sounds good, I'll close this issue and file a pull request.

This sounds great!

[takusuman]

I'm going to look into modernizing the shell scripts with help from shellcheck. Many have legacy backticks & other issues that can be been with shellcheck, and I believe that since we are in 2022, legacy shell features shouldn't be used as there isn't much reason to, aside from supporting shells that are older than I am.

I think that it would be useful to modernize, but it would break support that Heirloom had for many systems before our fork, such as old NetBSD or even Solaris, for instance. Modernizing scripts can be painful too, since it takes a lot of time reading the syntax and replacing old standards with new ones.

[ghost]

Are you talking about heirloom-ng or doctools?

The doctools. It's a known bug in the GitHub issues section.

[...] but it would break support that Heirloom had for many systems before our fork, such as old NetBSD or even Solaris, for instance.

Understood, I'll revert the commits that made changes to the shell scripts.

[ghost]

Also, on Debian 11.x on my end, there are some issues with compiling so once I successfully compile heirloom-ng I'll then file the PR.

[ghost]

Ends up the compile issues were due to me not disabling the static library options. Closing issue & opening PR.