search

rocky / remake

Enhanced GNU Make - tracing, error reporting, debugging, profiling and more
http://bashdb.sf.net/remake
GNU General Public License v3.0
783 stars 73 forks source link

No remake for GNU make 4.0 #12

Open mohawk2 opened 9 years ago

mohawk2 commented 9 years ago

Would you be interested in a PR updating remake to GNU make 4.0 and 4.1? (With suitable tags post facto)

rocky commented 9 years ago

Of course, help is always appreciated. But I am not sure I understand what's wrong.

edfaafbf09b7a5c94d84b8fe08720f8e00be6ebe is a merge of changes to bring this code compatible with make 4.1. Is what you propose to do is have the code for the unmodified GNU make 4.0 and 4.1 around in the tree as well as a branch just so folks have that as a reference?

mohawk2 commented 9 years ago

Ah ha, I don't know why I didn't see / remember that under https://sourceforge.net/projects/bashdb/files/remake/ - probably because I found 4.1 to make things worse for the situation I'm trying to debug (in perl's PDL), so I got the 3.82 one.

I do think that a good history strategy would be to have branches something like this:

And rebasing the dbg-* branch on a "rolling" basis against make-master of your changes, tagging the head commit of that with each release as suitable. This would even make it possible to have eg make-3.82-dbg0.91 (since some folks might want to debug 3.82 but using the latest of your rather good -x command).

If you think this would be something that's a good idea, let me know and I'll have a go at realising it.

rocky commented 9 years ago

The current organization came from yarikoptic, although I may have messed it up. Care to comment @yarikoptic ?

yarikoptic commented 9 years ago

uff... was it just yesterday, I might have remembered detail, but now... let's investigate, here is how "big picture" looks like http://www.onerussian.com/tmp/remake-big-picture.svg

I just remember that it was a bit messy since we were bringing two separate trees (make / remake) into a single one trying I believe to get some "historical merges" in... but looking forward to have future simple merges from upstream. So in general -- having make-master to track make, and then master track remake sounds most logical. And it is something we probably kinda had. But may be easier would be just to dump "manual" tracking of make with make-master branch, and just rely that there would be e.g. "make" remote pointing to make's git (I have git://git.savannah.gnu.org/make.git as upstream-make ATM). Sure thing, if there is need to support previous version trees -- those dbg-* branch could come

but atm, after all my initial "organization" I believe rocky still reverted to simple manual injection of next versions of make (as in e6f0d0d13883ac2efe4caaddda9fbddf25c6ea8c) so whatever was done before is irrelevant and might need another round of detangling to separate cleanly branch of upstream make changes from remake changes. Whatever strategy you choose which sticks -- would be the best, apparently my previous one didn't ;)

mohawk2 commented 9 years ago

I'd like to make things simple (especially compared to that diagram :-) - this would involve a one-time history-rewrite, with all the tags (that are considered to matter) still pointing at correct places. If you guys are interested, I'll fork the repo and get chopping, and update on this issue. It will include notes on how to maintain it going forward! If however you don't want to chop the history, it may be less valuable but still possible to do this.

Let me know your thoughts.

rocky commented 9 years ago

Here's a suggestion. Why don't you fork the code and organize it the way you think is best? Simple is good! Then we have something concrete to discuss.

Thanks.

mohawk2 commented 9 years ago

So far I have in my repo a mirror of the current GNU make (in make-master), and a rebase of remake up to but not including the 3-way merge("Just binding all repositories (CVS of make, old CVS of remake, and git of remake) togehter"), against the 3.80 release point of GNU make: https://github.com/mohawk2/remake/branches/all

I am currently rebasing that set of changes against the 3.81 release point of GNU make, with the goal of it at least compiling before I continue the rebase. I am finding it heavy going with the second commit because all the autoconf stuff is out of date, but I am working around that.

A current actual problem is the change of API to parse_file_seq, which added a floc parameter. Did that change persist? More recent versions of read.c seem to have undone that addition. Please help :-)

rocky commented 9 years ago

In short, I think not.

I don't see a floc parameter on either the remake 3.81+dbg or 4.1+dbg released versions. floc is an additional "file location" parameter used in remake only to assist debugging and tracing to explain where we are. Those parameters are not in the GNU make sources.

If you can point me to a sha where this is around, I could probably explain with more confidence.

My guess is that I added that temporarily thinking I'd need it only to later find not.

mohawk2 commented 9 years ago

This is a/the SHA with it in: 8dc0108ebdccda779a1e864b74a69ec49216a67f

I will proceed by simply erasing it from history! That's going to happen with the various merges over from the proto-3.81.

If by the way you feel like this is a process that you're not going to incorporate into the "new" history of remake, please say so. It is quite a bit of work and if it's not going to be used going forward, it is a waste of time.

rocky commented 9 years ago

With regards to the effort and wasting your time: I have been in your position on other projects and fully appreciate the difficulty and how awful it feels when one puts this much work digging into "mud" (or some other euphemism) and not having it be used. Trust me, if I thought this was a waste of time I would have said so.

I've looked at your fork so far. The branches so far make sense to me and I think/hope I can maintain that structure. I'm not quite sure of the mechanics of folding this back in, but I'm sure it can be done.

@yarikoptic : your thoughts please?

Finally, with respect to 8dc0108 : As you noted this part of a huge first set of patches that went into proto 3.81. Personally, I'm more interested in getting a structure that makes sense and can easily be maintained with someone with as feeble a brain as I have than preserving history. In fact erasing all the false starts might be a good idea.

mohawk2 commented 9 years ago

My plan is to make sure each rebase-conflicting commit compiles and passes tests on the way through this one, which will here require chopping out the parse_file_seq change. Once that rebase is complete, and its head compiles/passes, then I'll merge in the remaining remake commits, rebasing against each version of remake as appropriate. I might need to bring in the final state of the autoconf files early, since I'm not planning to downgrade my autotools installs.

Then I'll be looking at a big compression activity to reduce the number of commits down a lot, to make them a coherent set of changes.

rocky commented 9 years ago

This sounds good. Thanks for the information. Thanks for undertaking it.

(With respect to my remark "I have been in your position" see https://sourceforge.net/p/ruby-debugger-runtime/code/ci/master/tree/patches/ is a project where I've had to do something similar.)

mohawk2 commented 9 years ago

That's an impressive amount of debugger-writing! :+1:

mohawk2 commented 9 years ago

My latest idea is to cancel the fairly agonising process of rebasing all the false starts, and instead to squash all the earlier releases down into one commit each. The end-state will be:

That one "all of remake" commit could be broken into more than one, but it will be done (if at all) on a logical basis rather than a historical one, for easier maintenance going forward.

This is a tad radical, and will involve "losing" all the false starts, but it's something that can be completed relatively easily and I think from the above you're comfortable with it?

rocky commented 9 years ago

I'm trying to think of what the end result should be. Something like:

Possibly remake-3-81 but I suppose I don't care about that any more. The debian branch was inadvertently merged into other branches. I can't speak for yarikoptic if he wants the history there.

Merging everything within a major release (3.82 or 4.1) is a bit drastic, but possible. Again I'd like to hear from yarikoptic .

mohawk2 commented 9 years ago

Another, separate, possibility, is to flatten the currently spaghetti-like history into one stream of commits from GNU 3.80 to present day, and leave that on a branch for historical purposes - I'm currently going through this. Then make a single commit branch against latest GNU, and keep rebasing that along GNU as it updates.

rocky commented 9 years ago

Something like a picture would help me understand what you have in mind. Here's what I was trying to convey as a possibility:

gnu-make------+------+-------------------+-----------------+-------------+---------------->
       |     (3-80)  (3-81)            (3-82)            (4-0)         (4-1)
       |      |       |                  |                               |
       |      |       |                  |                               |
       |      |       |                  |                               |    
       |      |       |                  + remake-3.82--->               |              (4-1+09)  (4-1+091)
       |      |       + remake-3-81-->                                   +-remake-4-1----------------->
       |      |                                                                         
master +      +-------------------------------------------------------------------------+------->

Above, rows are branches, e.g. gnu-make, master, remake-3-81, remake-3-82, and remake-4-1. Tags are listed in parenthesis, like (3-80), (4-1+091).

Does this make sense?

mohawk2 commented 9 years ago

Yes. However that is not what I propose. The above is sort of what you've done up till now, but it's a bit untidy both aesthetically and conceptually. I'm just finishing going through and flattening out the various merges. I don't really see any reason to have branches other than:

Branch Description
make-master GNU make
dbg an evolving body of code, based on relatively few commits, constantly rebased against make-master as that changes
dbg-releases each full release gets snapshotted from dbg onto here in some meaningful way
dbg-historical with the full history up to this point except with merges flattened into more or less chronological order with a very big hammer

Speaking of a very big hammer, I am approaching the debian branch tangle. If it looks difficult to merge in, I'll leave it out, since if you want it in, I can go back and add it. Please let me know your wishes on that.

mohawk2 commented 9 years ago

The debian stuff is in. Here are the tags I have updated:

remake tags updated to new scheme:
release_0_2 390e9c2
release_0_3 52c8e1e
release_0_4 5ae29e2
release_0_5 57ce053
release_0_60 6e6be98
release_0_61 5e69b5f
release_0_62 d6ca7a7
release_fleet c9d4b58
remake_3_81_branch_start 6fdc5ad
release-3.81+dbg-0.1-rebased 1187d4bb
release-3.81+dbg-0.1 1187d4b
release-3.81+dbg-0.2 b38e31c3
release-3.81+dbg-0.3 19494fd6
release-3.82+dbg-0.4 649d57a2
release-0.5 10e36d5c
release-0.6 ebfb09b4
release-3.82+dbg-0.7 50b4749b
release-3.82+dbg-0.8 1a5e035
release-3.82+dbg-0.9 9024f1be
remake-4.1+dbg-0.91 8a586b2b3e

I have pushed dbg-historical and dbg, which is one commit of current remake master against GNU 4.1. I very quickly tried rebasing that against GNU master, and there were a couple of conflicts, so I left that for now.

Tell me what you guys think.

rocky commented 9 years ago

Yes. However that is not what I propose. The above is sort of what you've done up till now, but it's a bit untidy both aesthetically and conceptually. I'm just finishing going through and flattening out the various merges. I don't really see any reason to have branches other than: ...

Big sigh. I think there is a misunderstanding of how development of the project works and I regret not having communicated this better earlier.

GNU Make has its own history and it is now even in git. (When yarick started, GNU Make was using CVS). When I started remake, I tracked some changes of GNU Make periodically; I don't anymore.

Let me explain this. Between GNU Make releases 3.80 through release 3.82, it was about 4 years between each release. The changes in that 8-year time period were mostly of a bug-fix nature. I was developing the code to do the tracing, comment-processing, and providing a debugger. For me to work off the last stable version from say 2-3 years ago ignoring patches since then was silly. So initially, and messily, some of the earlier releases represented GNU Make up to that point plus my patches. It was kind of a selling point for remake, because you got the last released GNU Make plus its patches plus my patches.

Then GNU Make 4.0 came around. Things were slightly incompatible. And what I had to say was pretty much done. So for release 3.82+dbg and 4.1+dbg I work off of the last stable release. If it so happens that there is a specific patch or set of patches from the last release of GNU Make, I can apply that in the remake branch. But even here we aren't tracking inter-release changes to GNU Make anymore.

Given the above, the master branch should not be GNU Make; again, if you want that, use the git for GNU Make. The master branch should be for remake. Until such time as this code gets merged into GNU Make (which is a separate undertaking of itself), this git repo is for remake. Furthermore I think it would be irresponsible to represent the master branch as what goes on in GNU Make unless there were some automated mechanism for mirroring GNU Make.

Even if the outline of the branching is complicated or ugly, it matches how I think development will work. For git to be useful and for me not to botch things up again, it has to match how development works. So to reiterate. For now and the foreseeable future, we'll start with a release of GNU Make (in the early days it was an inter-release of GNU Make) and make a largish patch off of that.

That's why the structure should be:

make      +---|---------+-------+ ...
381+dbg   |   +---+     |       |
382+dbg   |       |     +----+  |
41+dbg    |       |          |  +-------+------+ ...
          |       |          |          |      |
...       |       |          |          |      |
master    +-------+----------+----------+------+ ....
                (381+dbg7)  (382+dbg8)         (42+dbg91)

It is perfectly fine to squash changes in the GNU Make branch between branches or splits to remake off of that. Those inter-release changes are not important for us but can be found in the GNU Make project. Squashing the changes in remake while they may be tolerable are a bit unconventional.

I hope this clears things up. Again, I apologize for not making this clearer before.

rocky commented 9 years ago

Just had a chance to look at https://github.com/mohawk2/remake and https://github.com/mohawk2/remake/network . Github doesn't list the name of the branches, but I can make out what they are based on commits and tags.

Shouldn't these all branch off of some version of GNU Make since that's where they started? Other than that things are great.

The debian stuff is in. Here are the tags I have updated: ...

Again all looks fine but shouldn't there be tags for each of the official GNU make releases?