search

joncampbell123 / dosbox-x

DOSBox-X fork of the DOSBox project
GNU General Public License v2.0
2.75k stars 381 forks source link

Cleaning and improving DOSBox-X's repository facade? #1918

Open aybe opened 4 years ago

aybe commented 4 years ago

Is your feature request related to a problem? Please describe.

When looking at home page, there's really a lot of files in /.

At the present time, there are exactly 115 files.

Describe the solution you'd like

Couldn't:

I'm pretty sure this could be reduced drastically, resulting in more elegant repository.

But of course, I can't pronounce myself about it, especially when it comes to:

Initial thoughts

Repository files

Entry Proposal
~!DosBox-X~ can't this RISC stuff go to wip?
.vscode my fault, this could be written in docs instead
~NOTES-TESTING-LOG~ can't this become a subdir of NOTES ?
~OLD-REFERENCE~ move to docs ?
~PLANS~ move to docs ?
~Ref-FM-Towns-Bochs~ put in docs ?
~chk~ 7 years old, delete ?
~doc-collection~ move to docs ?
experiments merge with wip ?
optimization-1 move to docs or wip ?
patch-integration move to contrib ?
pc98-testme-1 2 years old, delete ?
ref move to docs ?
scripts 7 years old, delete ?
snapshots move to NOTES\LOGS ?
vs2015 rename to windows or vs ?
most .md files README could point to a subdirectory where all docs are grouped ?
*.build/config/sh - these could go to scripts for clarity but will need modifications
- not sure as CHDIR-ing for these might be extremely boring for people using them
- might not be convenient for frequent contributors

Etc, in short, have a better front page for DOSBox-X repository.

Branches

https://github.com/joncampbell123/dosbox-x/branches

Active branches

Branch Date Proposal
master Updated 1 hour ago by Wengier it's fine
develop Updated 3 hours ago by Wengier it's fine
gh-pages Updated 4 hours ago by Wengier it's fine
translation Updated 16 days ago by aybe useless, deleted it

Stale branches

Branch Date Proposal
master-sdl2 Updated 3 years ago by joncampbell123 already in master ?
develop-dynamic-core Updated 3 years ago by joncampbell123 already in master ?
xbrz Updated 2 years ago by joncampbell123 already in master ?
sdlmain_refactor Updated 2 years ago by joncampbell123 #843 Merged
master-ptrace-core Updated 2 years ago by joncampbell123 already in master ?
master-win-ci Updated 2 years ago by joncampbell123 already in master ?
feature-sdl-entry-20181230 Updated 2 years ago by joncampbell123 already in master ?
feature-win7-taskbar-thumbnail-clip-20181231 Updated 2 years ago by joncampbell123 already in master ?
cleanup Updated 2 years ago by joncampbell123 already in master ?
private Updated 2 years ago by joncampbell123 already in master ?
Allofich-fix Updated 15 months ago by Allofich already in master ?
Wengier-updated Updated 4 months ago by joncampbell123 already in master ?
updated Updated 4 months ago by Wengier already in master ?

So yeah, there could be 3 branches at all for the repository.

Also, but this is a breaking change,

Why isn't develop the default branch as committing directly to master is a bit extreme in my opinion ?

EDIT

Also #1898.

aybe commented 4 years ago

Oops, did forget a few branches, updated first post accordingly.

Wengier commented 4 years ago

I agree with the general idea to clean up the files in the repository to make things more organized, while not breaking existing code. Indeed, @rderooy did some file cleanup recently, but more can be done of course.

Allofich commented 4 years ago

One thing to be careful about if you rename or move a file, is that file history is retained. Moving/renaming folders or files in GitHub will lose the file history.

https://stackoverflow.com/questions/2314652/is-it-possible-to-move-rename-files-in-git-and-maintain-their-history https://github.community/t/renaming-folder-within-a-repo-loses-file-history/1752/4

People at the above links say it is possible to keep file history by using the command line, not GitHub. I also see someone saying it is not possible, though. Seems like a good idea to confirm on your own fork that the file history is still there before committing anything to master.

joncampbell123 commented 4 years ago

If you move a file, you have to do it in one commit or Git will record it as the deletion of one file and the creation (addition) of another.

If you do it in one commit (and make no modifications), the command line Git tool is smart enough to record that it moved and will say so on the command line.

aybe commented 4 years ago

That's a very good point.

After reading these links and testing for a bit, this is the situation:

Command line

cmd_2020-10-04_08-19-35

SourceTree

It's there but not readily visible, first you have to select file then Log Selected:

SourceTree_2020-10-04_08-19-09

Then you must Follow renamed files:

SourceTree_2020-10-04_08-14-08

So it is possible but hidden...

Git Gui

Does it, sort of:

wish_2020-10-04_08-23-46

GitKraken

Doesn't:

gitkraken_2020-10-04_08-26-39

SmartGit

Does it, but it's not easily visible as well:

smartgit_2020-10-04_08-28-28

GitHub Desktop

Doesn't:

GitHubDesktop_2020-10-04_08-34-00

Conclusion

It's possible to view history for a file but only using git log --follow and providing that your CVS software GUI supports it.

I think for the files we're talking it's mostly okay, but say, for sources, it wouldn't be so cool.

As a side note, a readme file could be put in the destination folder to tell about the files with changed history, but well...

@joncampbell123 So what do we do ? Start with a clean slate for some of the files ? 😂

Allofich commented 4 years ago

@joncampbell123 So what do we do ? Start with a clean slate for some of the files ? 😂

If I understood @joncampbell123 correctly, I guess as long as you move/rename files without modifying their contents and send the moves/renames as commits through the command line, the history will be retained. Although, I didn't quite understand what he meant by doing it a file move in "one commit" (I don't understand how you would divide a file move across multiple commits). @joncampbell123, please correct me if I misunderstood something.

Edit: Ah, by do it in one commit, you probably meant don't delete the file from the source tree (such as by temporarily moving it out of the source tree to your local hard drive) as one commit, then put it in a new place in the source tree as a second commit.

aybe commented 4 years ago

@Allofich

I don't understand what you mean.

I've tried both ways, by moving files using Windows Explorer and by using git mv, both produced exactly the same result:

As people pointed out on the links you've put before, even though git sees a moves, it also sees a new file, which is weird ofc.

Allofich commented 4 years ago

@aybe

I just tried as well, and see the same thing. As a test, I tried moving CHANGELOG to the src folder. Both by using Windows Explorer and git mv CHANGELOG src/CHANGELOG, and pushing as a commit, it was recognized as a file rename in the commit, but in GitHub the history of the file is gone.

So unfortunately I do not know if there is an easy way to keep the file history or not, besides git log --follow.

rderooy commented 4 years ago

I had the same issue when I moved some files to contrib in the last cycle. I also looked it up and found the same answers. Basically you lose the history, which is rather annoying.

On Sun, Oct 4, 2020 at 9:38 AM Allofich notifications@github.com wrote:

@aybe https://github.com/aybe

I just tried as well, and see the same thing. As a test, I tried moving CHANGELOG to the src folder. Both by using Windows Explorer and git mv CHANGELOG src/CHANGELOG, and pushing as a commit, it was recognized as a file rename in the commit, but in GitHub the history of the file is gone.

So unfortunately I do not know if there is an easy way to keep the file history or not.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/joncampbell123/dosbox-x/issues/1918#issuecomment-703216313, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEHBR2TU4WZWV3Q54FVE2X3SJAQ6HANCNFSM4SDLPY5A .

aybe commented 4 years ago

Simply, there isn't.

The only facility that a CVS software GUI might provide for a history on steroids is through git log --follow.

But as you can see on my little experiment, some do it but it's not readily visible, many mouse clicks are needed.

Just remembered than Raymond Chen holds a serie on Git, though it's quite convoluted, it appears to be possible after all !!!

How to duplicate a file while preserving git line history

(truncated)

git blame foods

^62ef37c (Alice 2019-09-19 07:00:00 -0700 1) apple
335acb1b (Bob   2019-09-19 07:00:01 -0700 2) orange

git blame foods-new

^62ef37c foods (Alice 2019-09-19 07:00:00 -0700 1) apple
335acb1b foods (Bob   2019-09-19 07:00:01 -0700 2) orange

Just tried it and yes, it does work !!!

Now we just need to make a script to automate the process...

joncampbell123 commented 4 years ago

The way I do it is like this:

  1. Rename/move the file
  2. git add new file path
  3. git commit
  4. Git notes the file moved, doesn't count it as delete and add
ThePillenwerfer commented 4 years ago

I don't know about the technicalities of GitHub but some housekeeping is definitely in order as cloning brings down a load if rubbish as well as what's needed. Those of us how are far from experts can't see the wood for the trees.

aybe commented 4 years ago

@joncampbell123 it moves the file but history is still only visible through git log --follow

aybe commented 4 years ago

About the trick I posted, I doubled checked, not visible in GH, in fact results are exactly the same, GH sees a new file only:

chrome_2020-10-04_22-31-34

chrome_2020-10-04_22-31-56

So we're back to step zero, no history.

Ironically I now realize that some files slipped through the process and history was lost:

Here you see the file move but no way to see what it was before: https://github.com/joncampbell123/dosbox-x/commit/679c4d89c888f56b7138ad02493f92088bba4e41

Unless you do:

git log --all --full-history -- README.Windows

commit 679c4d89c888f56b7138ad02493f92088bba4e41
Author: Wengier <wengierwu@yahoo.com>
Date:   Sun Aug 2 00:35:59 2020 -0400

    Update and rename README.Windows to README.development-in-Windows

commit 6e08eb3bf0a95612e2c86950b58d9975d4cf0417
Merge: 9b994c7a4 473831cb5
Author: Jonathan Campbell <6245486+joncampbell123@users.noreply.github.com>
Date:   Wed Nov 13 16:32:50 2019 -0800

    Merge pull request #1348 from joncampbell123/vsfix

    Completed fixes for solution.

Then you get this: https://github.com/joncampbell123/dosbox-x/commit/6e08eb3bf0a95612e2c86950b58d9975d4cf0417

So yeah, history, but only through command-line.

To sum it up,

GH could have implemented git log --follow for a follow history but they didn't, like the majority of other software.

The only viable solution I see is to add a README.HISTORY next to new files location and tell about these special cases. And if user struggles about it, then eventually, he'll realize there's a README next to it that might be worth checking out !

I mean, there's no point in trying to fight with the system, it's possible but hasn't been implemented and probably never will.

There is a chrome extension exactly for that but well ... it's even more awkward than a README. (btw, it's broken, from 2018, obvious, GH has changed a lot since then)

aybe commented 4 years ago

I just had the following idea but I don't know how to achieve it:

If that's possible, then we should have an equivalent history.

Of course, dates will be different for destination file but at least we'll have a sliced history.

What do you think ?

Wengier commented 4 years ago

@aybe It is certainly an interesting idea, but generating one patch per diff for every single commit of every single affected file may take a significant amount of time and space.

In any case, I wonder why Git still does not support this natively, even though apparently a lot of projects are using it.

aybe commented 4 years ago

Git supports it but it's not clearly visible and software may or may not, but the vast majority being not.

I was sketching a promising batch but then got stuck with git apply that needs a relative path without trailing slash... boring.

Here it is if you want to improve it:

@echo off

if [%1]==[] (
    echo Missing parameter: source path
    goto usage
)

if [%2]==[] (
    echo Missing parameter: target path
    goto usage
)

rem sanity check
if not exist %1 (
    echo Source path does not exist
    goto failure
)
if exist %2 (
    echo Target path must not exist
    goto failure
)

rem get current branch
for /f "tokens=* usebackq" %%b in (`git branch --show-current`) do (
    set branch=%%b
)
echo Branch is "%branch%"
echo.

rem get commit count for file   
for /f "tokens=* usebackq" %%c in (`git rev-list --count %branch% %1`) do (
    set ccount=%%c
)
echo Source is "%~1" with %ccount% commits
echo.

rem create target path (but don't create file for first patch)
if not exist "%~dp2" mkdir "%~dp2"
echo Target is "%~2"
echo.

rem create patches
git format-patch -%ccount% %1 >patches.txt
echo Generated %ccount% patches for %1:
for /f "delims=" %%d in (patches.txt) do (
    echo    %%d
)
echo.

rem apply patches
echo Applying patches...
for /f "delims=" %%d in (patches.txt) do (
    rem git apply needs stupid relative dir with no trailing /
    rem todo commit
)
echo.

rem todo clean crap

goto success

:usage
echo Usage: %~n0 "source path" "target path"
call :cleanup
goto:eof

:success
echo Operation successful
call :cleanup
goto:eof

:failure
echo An error occurred, aborting
call :cleanup
goto:eof

:cleanup
set branch=
set ccount=
del patches.txt
goto:eof
Wengier commented 4 years ago

@aybe I have never used git apply myself, honestly. Someone more familiar with git may want to help.

aybe commented 4 years ago

Basically it should be git apply --directory=dir patch since we patch somewhere else than where file was. Problem is, dir should be relative and without a trailing slash, that's all what's missing.

I've tried manually and it works.

After of course we just have to add git add and git commit so it's the easiest. And to make it really nice I'd like to fetch author, message and maybe original hash too so it's really clean. Then finally, delete the old file.

To sum it up, main issue is more possibilities in batch language than git.

Wengier commented 4 years ago

@aybe If I understand it correctly, the git apply command will be applied to the target path with the patch files already generated. So what is wrong with something like git apply -p2 --directory=%2 %%d. But perhaps it is designed to work slightly different, and please let me know in such case. Thanks!

aybe commented 3 years ago

You've made a point here, target was target path but if it'd become targetdir then problem is almost solved, should be tried!

Wengier commented 3 years ago

@aybe I am not quite sure about the difference between target path and targetdir you intended to mean, but is there any progress on this one? Once it works I think it is almost time to do the actual cleanup.

Wengier commented 3 years ago

@aybe In case you are still struggling with the target path vs targetdir, can you please explain what the difference between them is? Perhaps an example will help me understand the difference better.

aybe commented 3 years ago

As said before, the frustrating issue in that is the limits of string manipulation you can do in batch.

Passing targetdir would remove the need to build a non-prepended slash dir-only path using target path 🤪, which is the only form that git apply accepts for --directory.

I tried somehow to play with EnableDelayedExtensions but then you can't assign that to a variable, and then it doesn't work when passed directly...

aybe commented 3 years ago

Now using PowerShell, works for .\!DosBox-X\!Boot,feb:

GitHubDesktop_2020-10-09_04-46-12

So we have original title, email and even date!!!

param (
    [string]$source = '.\!DosBox-X\!Boot,feb',
    [string]$target = '.\docs\RISC\!Boot,feb'
)
cls
"Source is {0}" -f $source
"Target is {0}" -f $target

# get current branch
$branch = & git branch --show-current | Out-String
$branch = $branch.Trim()
"Branch is {0}" -f $branch

# get commit count for file
$length = & git rev-list --count $branch $source | Out-String
$length = $length.Trim()
"Commit count is {0}" -f $length

# make patches
"Generating patches..."
$patchs = & git format-patch -$length $source | Out-String
$patchs = $patchs.Trim()
foreach ($patch in $patchs)
{
    $patch
}

# create target directory
$tgtdir = Split-Path $target | Out-String
$tgtdir = $tgtdir.Trim()
"Creating target directory {0}" -f $tgtdir
New-Item $tgtdir -ItemType Directory -Force | Out-Null

# apply patches, pass good syntax, add historical data, commit, cleanup
"Applying patches..."
$gitdir = $tgtdir.Substring(2).Replace("\","/")
$gitdir = $gitdir.Trim()
$dirlen = $gitdir.Split("/").Count
foreach ($patch in $patchs)
{
    & git apply --directory=$gitdir -p $dirlen "$patch"

    $hash = (Get-Content $patch)[0].Substring(5, 40)
    $mail = (Get-Content $patch)[1].Substring(6)
    $date = (Get-Content $patch)[2].Substring(6)
    $text = (Get-Content $patch)[3].Substring(9)
    $data = "NOTE: auto-magically re-imported by HAL 9000`r`nHASH: {0} (original)" -f $hash

    & git add $target
    & git commit -m "$text" -m "$data" --date="$date" --author="$mail"

    Remove-Item $patch
}

I was about to celebrate but then now git complains there's a stupid question mark in file but it's impossible...

There's no question mark in file name, that's impossible!!! 😡

Source is .\PLANS\General TODO.txt
Target is .\docs\PLANS\General TODO.txt
Branch is aybe-cleanup
Commit count is 7
Generating patches...
0001-Plenty-of-blather.-Got-some-good-ideas.patch
0002-add-reminder.patch
0003-more-plans.patch
0004-Note-to-self.patch
0005-Update-General-TODO.txt.patch
0006-Update-General-TODO.txt.patch
0007-Update-General-TODO.txt.patch
Creating target directory .\docs\PLANS
Applying patches...
git.exe : error: can't open patch '0001-Plenty-of-blather.-Got-some-good-ideas.patch?
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:41 char:5
+     & git apply --directory=$gitdir -p $dirlen "$patch"
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (error: can't op...od-ideas.patch?:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

0002-add-reminder.patch?
0003-more-plans.patch?
0004-Note-to-self.patch?
0005-Update-General-TODO.txt.patch?
0006-Update-General-TODO.txt.patch?
0007-Update-General-TODO.txt.patch': No such file or directory
Get-Content : Illegal characters in path.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:43 char:14
+     $hash = (Get-Content $patch)[0].Substring(5, 40)
+              ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Get-Content], ArgumentException
    + FullyQualifiedErrorId : ItemExistsArgumentError,Microsoft.PowerShell.Commands.GetContentCommand

Get-Content : Cannot find path 'C:\Users\Aybe\source\repos\dosbox-x\0001-Plenty-of-blather.-Got-some-good-ideas.patch
0002-add-reminder.patch
0003-more-plans.patch
0004-Note-to-self.patch
0005-Update-General-TODO.txt.patch
0006-Update-General-TODO.txt.patch
0007-Update-General-TODO.txt.patch' because it does not exist.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:43 char:14
+     $hash = (Get-Content $patch)[0].Substring(5, 40)
+              ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Get-Content], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

Cannot index into a null array.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:43 char:5
+     $hash = (Get-Content $patch)[0].Substring(5, 40)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

Get-Content : Illegal characters in path.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:44 char:14
+     $mail = (Get-Content $patch)[1].Substring(6)
+              ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Get-Content], ArgumentException
    + FullyQualifiedErrorId : ItemExistsArgumentError,Microsoft.PowerShell.Commands.GetContentCommand

Get-Content : Cannot find path 'C:\Users\Aybe\source\repos\dosbox-x\0001-Plenty-of-blather.-Got-some-good-ideas.patch
0002-add-reminder.patch
0003-more-plans.patch
0004-Note-to-self.patch
0005-Update-General-TODO.txt.patch
0006-Update-General-TODO.txt.patch
0007-Update-General-TODO.txt.patch' because it does not exist.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:44 char:14
+     $mail = (Get-Content $patch)[1].Substring(6)
+              ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Get-Content], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

Cannot index into a null array.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:44 char:5
+     $mail = (Get-Content $patch)[1].Substring(6)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

Get-Content : Illegal characters in path.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:45 char:14
+     $date = (Get-Content $patch)[2].Substring(6)
+              ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Get-Content], ArgumentException
    + FullyQualifiedErrorId : ItemExistsArgumentError,Microsoft.PowerShell.Commands.GetContentCommand

Get-Content : Cannot find path 'C:\Users\Aybe\source\repos\dosbox-x\0001-Plenty-of-blather.-Got-some-good-ideas.patch
0002-add-reminder.patch
0003-more-plans.patch
0004-Note-to-self.patch
0005-Update-General-TODO.txt.patch
0006-Update-General-TODO.txt.patch
0007-Update-General-TODO.txt.patch' because it does not exist.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:45 char:14
+     $date = (Get-Content $patch)[2].Substring(6)
+              ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Get-Content], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

Cannot index into a null array.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:45 char:5
+     $date = (Get-Content $patch)[2].Substring(6)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

Get-Content : Illegal characters in path.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:46 char:14
+     $text = (Get-Content $patch)[3].Substring(9)
+              ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Get-Content], ArgumentException
    + FullyQualifiedErrorId : ItemExistsArgumentError,Microsoft.PowerShell.Commands.GetContentCommand

Get-Content : Cannot find path 'C:\Users\Aybe\source\repos\dosbox-x\0001-Plenty-of-blather.-Got-some-good-ideas.patch
0002-add-reminder.patch
0003-more-plans.patch
0004-Note-to-self.patch
0005-Update-General-TODO.txt.patch
0006-Update-General-TODO.txt.patch
0007-Update-General-TODO.txt.patch' because it does not exist.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:46 char:14
+     $text = (Get-Content $patch)[3].Substring(9)
+              ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Get-Content], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

Cannot index into a null array.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:46 char:5
+     $text = (Get-Content $patch)[3].Substring(9)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

git.exe : fatal: pathspec '.\docs\PLANS\General TODO.txt' did not match any files
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:49 char:5
+     & git add $target
+     ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (fatal: pathspec...match any files:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

On branch aybe-cleanup
Untracked files:
  (use "git add <file>..." to include in what will be committed)
    0001-Plenty-of-blather.-Got-some-good-ideas.patch
    0002-add-reminder.patch
    0003-more-plans.patch
    0004-Note-to-self.patch
    0005-Update-General-TODO.txt.patch
    0006-Update-General-TODO.txt.patch
    0007-Update-General-TODO.txt.patch
    move-with-hist.ps1

nothing added to commit but untracked files present (use "git add" to track)
Remove-Item : Illegal characters in path.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:52 char:5
+     Remove-Item $patch
+     ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Remove-Item], ArgumentException
    + FullyQualifiedErrorId : ItemExistsArgumentError,Microsoft.PowerShell.Commands.RemoveItemCommand

Remove-Item : Cannot find path 'C:\Users\Aybe\source\repos\dosbox-x\0001-Plenty-of-blather.-Got-some-good-ideas.patch
0002-add-reminder.patch
0003-more-plans.patch
0004-Note-to-self.patch
0005-Update-General-TODO.txt.patch
0006-Update-General-TODO.txt.patch
0007-Update-General-TODO.txt.patch' because it does not exist.
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:52 char:5
+     Remove-Item $patch
+     ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\Aybe\s...-TODO.txt.patch:String) [Remove-Item], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand

Here is a file where it complains about non-existent question mark:

https://github.com/joncampbell123/dosbox-x/blob/master/PLANS/General%20TODO.txt

So if you guys wanna try, am out of ideas.

Wengier commented 3 years ago

@aybe Great efforts! But I guess something seems to be wrong with line 39 of the script:

foreach ($patch in $patchs)

When I changed it to:

foreach ($patch in $($patchs -split "`r`n"))

I got the following result:

Source is .\PLANS
Target is .\docs\PLANS
Branch is master
Commit count is 9
Generating patches...
0001-Plenty-of-blather.-Got-some-good-ideas.patch
0002-add-reminder.patch
0003-more-plans.patch
0004-Note-to-self.patch
0005-new-idea.patch
0006-more.patch
0007-Update-General-TODO.txt.patch
0008-Update-General-TODO.txt.patch
0009-Update-General-TODO.txt.patch
Creating target directory .\docs
Applying patches...
0001-Plenty-of-blather.-Got-some-good-ideas.patch:1667: trailing whitespace.

0001-Plenty-of-blather.-Got-some-good-ideas.patch:1694: trailing whitespace.
VII.  The registers
0001-Plenty-of-blather.-Got-some-good-ideas.patch:1734: trailing whitespace.
Well, a month overdue, but its finally here.  This guide will attempt to
0001-Plenty-of-blather.-Got-some-good-ideas.patch:1737: trailing whitespace.
program the EMU8000 on a low-level.  And I succeeded.  This is a compilation
0001-Plenty-of-blather.-Got-some-good-ideas.patch:1742: trailing whitespace.
reading an AWE32 in demos thread on comp.sys.ibm.pc.demos, I realized that the
warning: squelched 115 whitespace errors
warning: 120 lines add whitespace errors.
[master e1c2717b7] [PATCH 1/9] Plenty of blather. Got some good ideas.
 Author: Jonathan Campbell
 Date: Mon Jan 20 19:22:14 2014 -0800
 3 files changed, 8 insertions(+), 341 deletions(-)
 delete mode 100644 docs/PLANS/Sound Master II emulation/SM2_V_SB.DOC
 delete mode 100644 docs/PLANS/VMX core (using Linux KVM virtualization driver).txt
[master 113fb4d97] [PATCH 2/9] add reminder
 Author: Jonathan Campbell
 Date: Tue Jan 21 20:40:57 2014 -0800
 1 file changed, 2 insertions(+)
0003-more-plans.patch:27: trailing whitespace.
    pause emulation and allow you to examine/play with the VGA palette, DAC,
warning: 1 line adds whitespace errors.
[master 693c147ee] [PATCH 3/9] more plans
 Author: Jonathan Campbell
 Date: Wed Jan 22 14:11:21 2014 -0800
 1 file changed, 32 insertions(+)
[master 8384c783b] [PATCH 4/9] Note to self.
 Author: Jonathan Campbell
 Date: Thu Apr 17 20:29:23 2014 -0700
 1 file changed, 17 insertions(+)
0005-new-idea.patch:29: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
[master 41aa03148] [PATCH 5/9] new idea
 Author: Jonathan Campbell
 Date: Sun Apr 27 03:01:42 2014 -0700
 1 file changed, 13 insertions(+)
 create mode 100644 docs/PLANS/VMX core (using Linux KVM virtualization driver).txt
0006-more.patch:17: trailing whitespace.

0006-more.patch:18: trailing whitespace.

0006-more.patch:19: trailing whitespace.

0006-more.patch:20: trailing whitespace.

0006-more.patch:21: trailing whitespace.

warning: squelched 260 whitespace errors
warning: 265 lines add whitespace errors.
[master 24035089f] [PATCH 6/9] more
 Author: Jonathan Campbell
 Date: Sun Feb 8 14:06:21 2015 -0800
 1 file changed, 264 insertions(+)
 create mode 100644 docs/PLANS/Sound Master II emulation/SM2_V_SB.DOC
[master a10f7c624] [PATCH 7/9] Update General TODO.txt
 Date: Mon Jun 15 22:45:34 2020 -0400
 1 file changed, 3 insertions(+), 3 deletions(-)
[master 7fdb1d81a] [PATCH 8/9] Update General TODO.txt
 Date: Mon Jun 15 23:15:18 2020 -0400
 1 file changed, 7 insertions(+), 5 deletions(-)
[master 02b7d96ae] [PATCH 9/9] Update General TODO.txt
 Date: Sun Aug 2 13:27:01 2020 -0400
 1 file changed, 8 insertions(+), 5 deletions(-)

Can you check it out?

Wengier commented 3 years ago

@aybe As a demonstration the directory PLANS was moved to docs/PLANS using the tool in the repository. Can you check it out as well?

aybe commented 3 years ago

Ah yes, that makes sense. You've seen that I do $branch = $branch.Trim() many times, that's because it's not powershell 7.1 so no NoNewLine feature. The fix is to do $branch = $branch.Trim() everywhere so you always have a single line.

aybe commented 3 years ago

Forgot to mention, what's weird is that it does work out of the box for other files without any changes you've mentioned.

aybe commented 3 years ago

I've updated the script and enhanced it a bit, now file gets deleted at end:

https://github.com/joncampbell123/dosbox-x/blob/develop/move-with-hist.ps1

But it'll still fail on some binary patches:

.\move-with-hist.ps1 '.\Ref-FM-Towns-Bochs\FMBochs(2).zip' '.\docs\FM-Towns\Ref-FM-Towns-Bochs\FMBochs(2).zip'

Source is .\Ref-FM-Towns-Bochs\FMBochs(2).zip
Target is .\docs\FM-Towns\Ref-FM-Towns-Bochs\FMBochs(2).zip
Branch is develop
Commit count is 1
Generating patches
0001-more.patch
Creating target directory .\docs\FM-Towns\Ref-FM-Towns-Bochs
Applying patches
git.exe : error: git diff header lacks filename information when removing 3 leading pathname components (line 14)
At C:\Users\Aybe\source\repos\dosbox-x\move-with-hist.ps1:48 char:5

There's a fix I've tried but that didn't work I've commented out: https://github.com/joncampbell123/dosbox-x/blob/develop/move-with-hist.ps1#L43

Wengier commented 3 years ago

@aybe I think the script generally works, and I did not see the error you posted, but at least two (mostly minor) issues need to be fixed:

1: Right now you have to specify source and target paths with .\<path-name>. Simply <path-name> (e.g. Ref-FM-Towns-Bochs instead of .\Ref-FM-Towns-Bochs) will not work as expected.

  1. I think there should be a -r option for git rm, or it may show errors that -r is required for recursive removal.

Almost there I think.

aybe commented 3 years ago

If you haven't seen the error then for sure you have a different git config than mine I guess.

For the paths I just use them as they're provided by PowerShell auto-complete feature.

And why would you want recursive, it's deleting a single file at a time?

Yes, pretty close, writing a script for moving entire directories should be very easy after that!

Wengier commented 3 years ago

@aybe Now the script has the ability to ask the user to enter the source and target paths, but the user may not enter .\ as part of the paths, so I guess it is better to address them as well so that it will work better in such cases.

Also, I am not sure if it is deleting a single file at a time, but I kept getting the following errors if the -r option is not added to the command:

Removing original file ./Ref-FM-Towns-Bochs
fatal: not removing './Ref-FM-Towns-Bochs' recursively without -r

You can check it out too.

aybe commented 3 years ago

The script is intended to move a single file at a time and I don't see what's wrong with the syntax, it's the default one. What should be fixed is getting these patches with blobs to work by figuring out why it fails.

Wengier commented 3 years ago

@aybe Any fix for it yet? We can start to use it once it is confirmed to work.

aybe commented 3 years ago

Sorry no, currently I lack time to look at it ...

aybe commented 3 years ago

@Wengier

I've improved it a bit and added moving dirs entirely: https://github.com/joncampbell123/dosbox-x/blob/migration/move-with-hist-bulk.ps1

But still some errors: https://github.com/joncampbell123/dosbox-x/blob/migration/move-with-hist-bulk.errors.1.txt https://github.com/joncampbell123/dosbox-x/blob/migration/move-with-hist-bulk.errors.2.txt

And yeah, I'm quite stupid, I shouldn't have pushed to develop, it's not ready yet 😱! Not sure what to do if to undo these changes or, thing is they're intertwined with changes to scripts... What's sure is that next time I won't commit again to develop...

I just reverted things

Lesson learned, I'll let it sit for a bit until those errors are figured out.

Wengier commented 3 years ago

@aybe You are certainly more familiar with the Windows PowerShell in general. Please keep up the efforts and let us know if there is progress. Thanks!

aybe commented 3 years ago

@Wengier

I've upgraded the scripts a bit in a hope to pin-point where the error comes from but failed miserably. Everything I've found about it didn't work...

powershell_ise_2020-10-17_09-06-58

https://github.com/joncampbell123/dosbox-x/tree/migration https://github.com/joncampbell123/dosbox-x/blob/migration/move-with-hist.ps1 https://github.com/joncampbell123/dosbox-x/blob/migration/move-with-hist-bulk.ps1

Ironically, I realize that move-with-hist.ps1 also works against directories, or almost; it will generate patches for all file in directory but then you get an extremely weird error: git : error: unknown switch0'` But there's no zero at all, very puzzling.

It's best for the time being to keep move-with-hist-bulk.ps1 as the directory mover then eventually we'll merge.

EDIT: Actually no, not familiar at all with it, first time I've used it is to write this very script ! 😂

aybe commented 3 years ago

And to sum it up,

Basically it almost works, except for a few cases for which git returns weird error messages and for which online help hasn't been of much so far.