search

K2InformaticsGmbH / oranif

Oracle OCI driver using dirty NIF
Apache License 2.0
4 stars 2 forks source link

commit history cleanup #93

Closed c-bik closed 5 years ago

c-bik commented 5 years ago

Current repo history size:

$ git clone --mirror https://github.com/k2informaticsgmbh/oranif
Cloning into bare repository 'oranif.git'...
remote: Enumerating objects: 469, done.
remote: Counting objects: 100% (469/469), done.
remote: Compressing objects: 100% (257/257), done.
remote: Total 4253 (delta 226), reused 436 (delta 212), pack-reused 3784
Receiving objects: 100% (4253/4253), 88.16 MiB | 1.15 MiB/s, done.
Resolving deltas: 100% (2961/2961), done.

$ du -sh oranif.git/
89M     oranif.git/
c-bik commented 5 years ago

Large files not related to source (was temporary used in preperation of CI/CT integration) are found in:

$ git rev-list --objects --all \
    | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
    | sed -n 's/^blob //p' \
    | grep -vF --file=<(git ls-tree -r HEAD | awk '{print $3}') \
    | sort --numeric-sort --key=2 \
    | cut -c 1-12,41- \
    | $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest

5f6286496a2f   81KiB test/install/oracle/db-sample-schemas-12.2.0.1/order_entry/oe_p_hu.sql
9b6e699e7634  282KiB test/install/oracle/db-sample-schemas-12.2.0.1/product_media/keyboard.mpa
517ca9672a08  318KiB test/install/oracle/db-sample-schemas-12.2.0.1/product_media/monitor.rm
5dacafe8af7d  952KiB test/install/oracle/db-sample-schemas-12.2.0.1/product_media/dimm1.rm
990ef7da1a56  1.2MiB test/install/oracle/db-sample-schemas-12.2.0.1/sales_history/dem1v3.dat
908dcb42d760  2.4MiB test/install/oracle/db-sample-schemas-12.2.0.1/sales_history/sh_prod.dat
891a1ef96e72  4.8MiB test/install/oracle/db-sample-schemas-12.2.0.1/sales_history/sale2v3.dat
2aa2c21028b7  6.8MiB test/install/oracle/db-sample-schemas-12.2.0.1/sales_history/sh_cust.dat
a0fccfb8702a  8.9MiB test/install/oracle/db-sample-schemas-12.2.0.1/order_entry/PurchaseOrders.dmp
d96727e2c01e   13MiB test/install/oracle/db-sample-schemas-12.2.0.1/sales_history/cust1v3.dat
caeba6ba69c8   40MiB test/install/oracle/db-sample-schemas-12.2.0.1/sales_history/sale1v3.dat
8044a427be9a   51MiB test/install/oracle/oracle-instantclient19.3-basic-19.3.0.0.0-1.x86_64.rpm
0db3f7b6a3ed   53MiB test/install/oracle/db-sample-schemas-12.2.0.1/sales_history/sh_sales.dat

After purging test/install with java -jar bfg-1.13.0.jar --delete-folders install oranif.git

$ du -sh oranif.git
464K    oranif.git
c-bik commented 5 years ago

July 1 : First attempt

$ java -jar bfg-1.13.0.jar --delete-folders install oranif.git
Using repo : /tmp/oranif-cleanup/oranif.git

Found 36 objects to protect
Found 1 tag-pointing refs : refs/tags/0.2.0
Found 77 commit-pointing refs : HEAD, refs/heads/cbik-eunit_cleanup, refs/heads/common-test, ...

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

* commit c03440e2 (protected by 'HEAD')

Cleaning
--------

Found 792 commits
Cleaning commits:       100% (792/792)
Cleaning commits completed in 1,364 ms.

Updating 20 Refs
----------------

        Ref                  Before     After
        ----------------------------------------
        refs/pull/60/head  | b6ab5869 | 52f6c8b2
        refs/pull/62/head  | 664c4b27 | 6ad8bafd
        refs/pull/64/head  | 969d4413 | 8fb6feff
        refs/pull/66/head  | 237eb36d | 90aeaf55
        refs/pull/67/head  | 68080b55 | 7f110a2b
        refs/pull/68/head  | fc159d64 | a5c67086
        refs/pull/70/head  | 7a33387c | ffaa6479
        refs/pull/71/merge | db65c2aa | d27be138
        refs/pull/72/head  | 2b31fcb7 | b402b806
        refs/pull/73/head  | 71a095d7 | 632761c1
        refs/pull/74/head  | e1eba552 | 864a4054
        refs/pull/75/head  | 11f3c63c | 97340794
        refs/pull/77/head  | 1e1ebd03 | 643ac8ce
        refs/pull/78/head  | 10b33243 | fb0d3425
        refs/pull/79/head  | de28c97e | 8b29e33b
        ...

Updating references:    100% (20/20)
...Ref update completed in 283 ms.

Commit Tree-Dirt History
------------------------

        Earliest                                              Latest
        |                                                          |
        ..............................DDDDDDDDDDD.mmm..Dmmmmmmmmmm..

        D = dirty commits (file tree fixed)
        m = modified commits (commit message or parents changed)
        . = clean commits (no changes to file tree)

                                Before     After
        -------------------------------------------
        First modified commit | cae2a9b9 | b4edcfbf
        Last dirty commit     | d2b16a35 | a3e54748

In total, 203 object ids were changed. Full details are logged here:

        /tmp/oranif-cleanup/oranif.git.bfg-report/2019-07-01/12-51-21

BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive

/tmp/oranif-cleanup$ cd oranif.git

/tmp/oranif-cleanup/oranif.git$ git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 3597, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3555/3555), done.
Writing objects: 100% (3597/3597), done.
Total 3597 (delta 2642), reused 790 (delta 0)

/tmp/oranif-cleanup/oranif.git$ git push --force
Counting objects: 571, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (328/328), done.
Writing objects: 100% (571/571), 134.15 KiB | 6.71 MiB/s, done.
Total 571 (delta 355), reused 385 (delta 236)
remote: Resolving deltas: 100% (355/355), completed with 38 local objects.
To https://github.com/k2informaticsgmbh/oranif
! [remote rejected] refs/pull/10/head -> refs/pull/10/head (deny updating a hidden ref)
! [remote rejected] refs/pull/14/head -> refs/pull/14/head (deny updating a hidden ref)
! [remote rejected] refs/pull/15/head -> refs/pull/15/head (deny updating a hidden ref)
! [remote rejected] refs/pull/15/merge -> refs/pull/15/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/2/head -> refs/pull/2/head (deny updating a hidden ref)
! [remote rejected] refs/pull/24/head -> refs/pull/24/head (deny updating a hidden ref)
! [remote rejected] refs/pull/25/head -> refs/pull/25/head (deny updating a hidden ref)
! [remote rejected] refs/pull/26/head -> refs/pull/26/head (deny updating a hidden ref)
! [remote rejected] refs/pull/3/head -> refs/pull/3/head (deny updating a hidden ref)
! [remote rejected] refs/pull/31/head -> refs/pull/31/head (deny updating a hidden ref)
! [remote rejected] refs/pull/32/head -> refs/pull/32/head (deny updating a hidden ref)
! [remote rejected] refs/pull/33/head -> refs/pull/33/head (deny updating a hidden ref)
! [remote rejected] refs/pull/39/head -> refs/pull/39/head (deny updating a hidden ref)
! [remote rejected] refs/pull/41/head -> refs/pull/41/head (deny updating a hidden ref)
! [remote rejected] refs/pull/47/head -> refs/pull/47/head (deny updating a hidden ref)
! [remote rejected] refs/pull/48/head -> refs/pull/48/head (deny updating a hidden ref)
! [remote rejected] refs/pull/50/head -> refs/pull/50/head (deny updating a hidden ref)
! [remote rejected] refs/pull/52/head -> refs/pull/52/head (deny updating a hidden ref)
! [remote rejected] refs/pull/54/head -> refs/pull/54/head (deny updating a hidden ref)
! [remote rejected] refs/pull/56/head -> refs/pull/56/head (deny updating a hidden ref)
! [remote rejected] refs/pull/59/head -> refs/pull/59/head (deny updating a hidden ref)
! [remote rejected] refs/pull/60/head -> refs/pull/60/head (deny updating a hidden ref)
! [remote rejected] refs/pull/62/head -> refs/pull/62/head (deny updating a hidden ref)
! [remote rejected] refs/pull/63/head -> refs/pull/63/head (deny updating a hidden ref)
! [remote rejected] refs/pull/64/head -> refs/pull/64/head (deny updating a hidden ref)
! [remote rejected] refs/pull/66/head -> refs/pull/66/head (deny updating a hidden ref)
! [remote rejected] refs/pull/67/head -> refs/pull/67/head (deny updating a hidden ref)
! [remote rejected] refs/pull/68/head -> refs/pull/68/head (deny updating a hidden ref)
! [remote rejected] refs/pull/7/head -> refs/pull/7/head (deny updating a hidden ref)
! [remote rejected] refs/pull/7/merge -> refs/pull/7/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/70/head -> refs/pull/70/head (deny updating a hidden ref)
! [remote rejected] refs/pull/71/head -> refs/pull/71/head (deny updating a hidden ref)
! [remote rejected] refs/pull/71/merge -> refs/pull/71/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/72/head -> refs/pull/72/head (deny updating a hidden ref)
! [remote rejected] refs/pull/73/head -> refs/pull/73/head (deny updating a hidden ref)
! [remote rejected] refs/pull/74/head -> refs/pull/74/head (deny updating a hidden ref)
! [remote rejected] refs/pull/75/head -> refs/pull/75/head (deny updating a hidden ref)
! [remote rejected] refs/pull/77/head -> refs/pull/77/head (deny updating a hidden ref)
! [remote rejected] refs/pull/78/head -> refs/pull/78/head (deny updating a hidden ref)
! [remote rejected] refs/pull/79/head -> refs/pull/79/head (deny updating a hidden ref)
! [remote rejected] refs/pull/8/head -> refs/pull/8/head (deny updating a hidden ref)
! [remote rejected] refs/pull/80/head -> refs/pull/80/head (deny updating a hidden ref)
! [remote rejected] refs/pull/85/head -> refs/pull/85/head (deny updating a hidden ref)
! [remote rejected] refs/pull/85/merge -> refs/pull/85/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/86/head -> refs/pull/86/head (deny updating a hidden ref)
! [remote rejected] refs/pull/87/head -> refs/pull/87/head (deny updating a hidden ref)
! [remote rejected] refs/pull/87/merge -> refs/pull/87/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/88/head -> refs/pull/88/head (deny updating a hidden ref)
! [remote rejected] refs/pull/9/head -> refs/pull/9/head (deny updating a hidden ref)
! [remote rejected] refs/pull/90/head -> refs/pull/90/head (deny updating a hidden ref)
! [remote rejected] refs/pull/90/merge -> refs/pull/90/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/91/head -> refs/pull/91/head (deny updating a hidden ref)
! [remote rejected] refs/pull/91/merge -> refs/pull/91/merge (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/k2informaticsgmbh/oranif'

Contacted github support.

c-bik commented 5 years ago

Correspondences with github support till today:

Jul 2, 13:02 AM UTC Github : In order for you to push up the changes we will need to either delete those pull requests or we can dereference them, which will keep the conversations in tact but 404 the files changes tab.

Jul 2, 5:40 AM UTC Us: if you delete/dereference the pull requests the branches these PRs made from will be left intact. If that is correct understanding, please dereference the PRs. We can then easily re-PR from the same branches after cleanup. PRs with branch name info and conversation should be enough to create new PRs. As for the dereferenced PRs we can live with 404 on changed files / diff, we will probably close them (the open ones) anyway afterwards

Jul 3, 3:58 AM UTC Github : I have now dereferenced those Pull Requests. Let me know how you go.

c-bik commented 5 years ago

July 3 : Second Attempt

~/oranif-cleanup$ java -jar bfg-1.13.0.jar --delete-folders install oranif.git
Using repo : /home/bikram/oranif-cleanup/oranif.git

Found 36 objects to protect
Found 1 tag-pointing refs : refs/tags/0.2.0
Found 17 commit-pointing refs : HEAD, refs/heads/common-tests, refs/heads/eunit_cleanup, ...

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

* commit 3f9af7c9 (protected by 'HEAD')

Cleaning
--------

Found 487 commits
Cleaning commits:       100% (487/487)
Cleaning commits completed in 694 ms.

Updating 3 Refs
---------------

        Ref                               Before     After
        -----------------------------------------------------
        refs/heads/karl-eunit-cleanup-2 | bf91466d | d6998997
        refs/pull/87/head               | bf91466d | d6998997
        refs/pull/87/merge              | 3a293e87 | ef98cac2

Updating references:    100% (3/3)
...Ref update completed in 85 ms.

Commit Tree-Dirt History
------------------------

        Earliest                                              Latest
        |                                                          |
        ..............DDDDDDDDDDDDDDDmm.m.m...Dmmmm..mmmmmmmmmmmmmmm

        D = dirty commits (file tree fixed)
        m = modified commits (commit message or parents changed)
        . = clean commits (no changes to file tree)

                                Before     After
        -------------------------------------------
        First modified commit | cae2a9b9 | b4edcfbf
        Last dirty commit     | d2b16a35 | a3e54748

In total, 192 object ids were changed. Full details are logged here:

        /home/bikram/oranif-cleanup/oranif.git.bfg-report/2019-07-03/10-44-11

BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive

~/oranif-cleanup/oranif.git$ git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 1772, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1742/1742), done.
Writing objects: 100% (1772/1772), done.
Total 1772 (delta 1230), reused 461 (delta 0)

bikram@WKS006:~/oranif-cleanup/oranif.git$ git push --force
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 894 bytes | 894.00 KiB/s, done.
Total 3 (delta 1), reused 2 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To https://github.com/k2informaticsgmbh/oranif
! [remote rejected] refs/pull/87/head -> refs/pull/87/head (deny updating a hidden ref)
! [remote rejected] refs/pull/87/merge -> refs/pull/87/merge (deny updating a hidden ref)
! [remote rejected] refs/pull/91/merge -> refs/pull/91/merge (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/k2informaticsgmbh/oranif'

Reported back to github support

c-bik commented 5 years ago

Github Response

Looks like you'll need to delete these branches as those Pull Requests are still open.

c-bik commented 5 years ago

Third Attempt

$ java -jar bfg-1.13.0.jar --delete-folders install oranif.git

Using repo : /home/bikram/git-cleanup/oranif.git

Found 36 objects to protect
Found 1 tag-pointing refs : refs/tags/0.2.0
Found 15 commit-pointing refs : HEAD, refs/heads/common-tests, refs/heads/eunit_cleanup, ...

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

 * commit a622f838 (protected by 'HEAD')

Cleaning
--------

Found 520 commits
Cleaning commits:       100% (520/520)
Cleaning commits completed in 645 ms.

Updating 1 Ref
--------------

        Ref                 Before     After
        ---------------------------------------
        refs/pull/87/head | 327789be | fbd5af52

Updating references:    100% (1/1)
...Ref update completed in 63 ms.

Commit Tree-Dirt History
------------------------

        Earliest                                              Latest
        |                                                          |
        .............DDDDDDDDDDDDDDDm.m.m...Dmmm..mmmmmmmmmmmmmmmmmm

        D = dirty commits (file tree fixed)
        m = modified commits (commit message or parents changed)
        . = clean commits (no changes to file tree)

                                Before     After
        -------------------------------------------
        First modified commit | cae2a9b9 | b4edcfbf
        Last dirty commit     | d2b16a35 | a3e54748

$ git reflog expire --expire=now --all && git gc --prune=now --aggressive
Counting objects: 1855, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1825/1825), done.
Writing objects: 100% (1855/1855), done.
Total 1855 (delta 1301), reused 420 (delta 0)

$ du -sh oranif.git
560K    oranif.git

$ git push --force
Counting objects: 263, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (73/73), done.
Writing objects: 100% (263/263), 47.83 KiB | 3.19 MiB/s, done.
Total 263 (delta 202), reused 251 (delta 190)
remote: Resolving deltas: 100% (202/202), completed with 15 local objects.
To https://github.com/k2informaticsgmbh/oranif
 ! [remote rejected] refs/pull/85/head -> refs/pull/85/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/87/head -> refs/pull/87/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/91/head -> refs/pull/91/head (deny updating a hidden ref)
error: failed to push some refs to 'https://github.com/k2informaticsgmbh/oranif'

Requested reference double check to github support

c-bik commented 5 years ago

Github Response

Can you please delete these two branches:

need to wait for PRs merged

c-bik commented 5 years ago 
$ git clone --mirror https://github.com/k2informaticsgmbh/oranif
Cloning into bare repository 'oranif.git'...
remote: Enumerating objects: 37, done.
remote: Counting objects: 100% (37/37), done.
remote: Compressing objects: 100% (33/33), done.
remote: Total 1660 (delta 17), reused 10 (delta 4), pack-reused 1623
Receiving objects: 100% (1660/1660), 415.69 KiB | 1.23 MiB/s, done.
Resolving deltas: 100% (1119/1119), done.

$ du -sh oranif.git/
556K    oranif.git/

/oranif.git$ git rev-list --objects --all \
    | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
    | sed -n 's/^blob //p' \
    | grep -vF --file=<(git ls-tree -r HEAD | awk '{print $3}') \
    | sort --numeric-sort --key=2 \
    | cut -c 1-12,41- \
    | $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest

9fe41855ec4e   57KiB c_src/erloci_nif.c
48b6c3ab9418   57KiB c_src/erloci.c
dfd6a7e46279   57KiB c_src/erloci.c
4bd6feac29d2   58KiB c_src/erloci.c
b2a80750d9db   58KiB c_src/erloci.c
9139fd4d3b18   58KiB c_src/ocinif.c
314ceed5d79f   58KiB c_src/erloci.c
85e832877aef   68KiB src/erloci_nif_int.erl
edd0e2464cc8   68KiB src/erloci_int.erl
4f125b468c57   68KiB src/erloci_nif_int.erl
27e6fe39f8f8   88KiB test/oranif_eunit.erl
3104971dd871   88KiB test/oranif_eunit.erl
db76702a0c18   89KiB test/oranif_eunit.erl
66ed9e82a854   89KiB test/oranif_eunit.erl
fed067b3edf1   90KiB test/oranif_eunit.erl

History clenup complete!