RefactoringMiner is a library/API written in Java that can detect refactorings applied in the history of a Java project. Since version 3.0, RefactoringMiner can also generate Abstract Syntax Tree (AST) diff at commit and pull request level.
Currently, it supports the detection of the following refactorings:
supported by RefactoringMiner 1.0 and newer versions
supported by RefactoringMiner 2.0 and newer versions
supported by RefactoringMiner 2.1 and newer versions
supported by RefactoringMiner 2.2 and newer versions
final
, static
, abstract
, synchronized
)final
, static
, abstract
, synchronized
)final
, static
, transient
, volatile
)final
, static
, transient
, volatile
)final
)final
)final
)final
)final
, static
, abstract
)final
, static
, abstract
)class
, interface
, enum
, annotation
, record
)supported by RefactoringMiner 2.3 and newer versions
supported by RefactoringMiner 2.4 and newer versions
supported by RefactoringMiner 3.0 and newer versions
As of June 8, 2024 the precision and recall of the tool on an oracle consisting of 547 commits from 188 open-source projects is:
Refactoring Type | TP | FP | FN | Precision | Recall |
---|---|---|---|---|---|
Total | 12263 | 20 | 236 | 0.998 | 0.981 |
Extract Method | 1006 | 1 | 22 | 0.999 | 0.979 |
Rename Class | 56 | 0 | 2 | 1.000 | 0.966 |
Move Attribute | 249 | 2 | 8 | 0.992 | 0.969 |
Move And Rename Attribute | 13 | 0 | 0 | 1.000 | 1.000 |
Replace Attribute | 1 | 0 | 0 | 1.000 | 1.000 |
Rename Method | 387 | 4 | 21 | 0.990 | 0.949 |
Inline Method | 116 | 0 | 1 | 1.000 | 0.991 |
Move Method | 350 | 3 | 6 | 0.992 | 0.983 |
Move And Rename Method | 122 | 0 | 3 | 1.000 | 0.976 |
Pull Up Method | 289 | 0 | 5 | 1.000 | 0.983 |
Move Class | 1096 | 0 | 4 | 1.000 | 0.996 |
Move And Rename Class | 36 | 0 | 1 | 1.000 | 0.973 |
Move Source Folder | 3 | 0 | 0 | 1.000 | 1.000 |
Pull Up Attribute | 139 | 0 | 1 | 1.000 | 0.993 |
Push Down Attribute | 35 | 0 | 0 | 1.000 | 1.000 |
Push Down Method | 45 | 0 | 1 | 1.000 | 0.978 |
Extract Interface | 22 | 0 | 0 | 1.000 | 1.000 |
Extract Superclass | 74 | 0 | 0 | 1.000 | 1.000 |
Extract Subclass | 4 | 0 | 0 | 1.000 | 1.000 |
Extract Class | 106 | 0 | 0 | 1.000 | 1.000 |
Extract And Move Method | 101 | 0 | 69 | 1.000 | 0.594 |
Move And Inline Method | 13 | 0 | 4 | 1.000 | 0.765 |
Replace Anonymous With Class | 8 | 0 | 0 | 1.000 | 1.000 |
Rename Package | 16 | 0 | 0 | 1.000 | 1.000 |
Move Package | 10 | 0 | 0 | 1.000 | 1.000 |
Extract Variable | 282 | 0 | 0 | 1.000 | 1.000 |
Extract Attribute | 22 | 0 | 0 | 1.000 | 1.000 |
Inline Variable | 102 | 0 | 0 | 1.000 | 1.000 |
Inline Attribute | 9 | 0 | 0 | 1.000 | 1.000 |
Rename Variable | 327 | 3 | 11 | 0.991 | 0.967 |
Rename Parameter | 489 | 2 | 26 | 0.996 | 0.950 |
Rename Attribute | 146 | 0 | 9 | 1.000 | 0.942 |
Merge Variable | 6 | 0 | 0 | 1.000 | 1.000 |
Merge Parameter | 28 | 0 | 0 | 1.000 | 1.000 |
Merge Attribute | 5 | 0 | 0 | 1.000 | 1.000 |
Split Variable | 3 | 0 | 0 | 1.000 | 1.000 |
Split Parameter | 7 | 0 | 0 | 1.000 | 1.000 |
Split Attribute | 2 | 0 | 0 | 1.000 | 1.000 |
Replace Variable With Attribute | 123 | 0 | 0 | 1.000 | 1.000 |
Replace Attribute With Variable | 28 | 0 | 1 | 1.000 | 0.966 |
Parameterize Variable | 111 | 0 | 0 | 1.000 | 1.000 |
Localize Parameter | 26 | 0 | 0 | 1.000 | 1.000 |
Parameterize Attribute | 24 | 0 | 0 | 1.000 | 1.000 |
Change Return Type | 433 | 0 | 12 | 1.000 | 0.973 |
Change Variable Type | 799 | 2 | 7 | 0.998 | 0.991 |
Change Parameter Type | 651 | 1 | 11 | 0.998 | 0.983 |
Change Attribute Type | 244 | 0 | 8 | 1.000 | 0.968 |
Add Method Annotation | 331 | 0 | 1 | 1.000 | 0.997 |
Remove Method Annotation | 100 | 0 | 0 | 1.000 | 1.000 |
Modify Method Annotation | 29 | 0 | 0 | 1.000 | 1.000 |
Add Attribute Annotation | 62 | 0 | 1 | 1.000 | 0.984 |
Remove Attribute Annotation | 18 | 0 | 0 | 1.000 | 1.000 |
Modify Attribute Annotation | 7 | 0 | 0 | 1.000 | 1.000 |
Add Class Annotation | 52 | 0 | 0 | 1.000 | 1.000 |
Remove Class Annotation | 20 | 0 | 0 | 1.000 | 1.000 |
Modify Class Annotation | 35 | 0 | 0 | 1.000 | 1.000 |
Add Parameter Annotation | 34 | 0 | 0 | 1.000 | 1.000 |
Remove Parameter Annotation | 4 | 0 | 0 | 1.000 | 1.000 |
Modify Parameter Annotation | 2 | 0 | 0 | 1.000 | 1.000 |
Add Parameter | 849 | 2 | 1 | 0.998 | 0.999 |
Remove Parameter | 307 | 0 | 0 | 1.000 | 1.000 |
Reorder Parameter | 9 | 0 | 0 | 1.000 | 1.000 |
Add Variable Annotation | 1 | 0 | 0 | 1.000 | 1.000 |
Remove Variable Annotation | 4 | 0 | 0 | 1.000 | 1.000 |
Add Thrown Exception Type | 41 | 0 | 0 | 1.000 | 1.000 |
Remove Thrown Exception Type | 265 | 0 | 0 | 1.000 | 1.000 |
Change Thrown Exception Type | 9 | 0 | 0 | 1.000 | 1.000 |
Change Method Access Modifier | 332 | 0 | 0 | 1.000 | 1.000 |
Change Attribute Access Modifier | 230 | 0 | 0 | 1.000 | 1.000 |
Encapsulate Attribute | 49 | 0 | 0 | 1.000 | 1.000 |
Add Method Modifier | 89 | 0 | 0 | 1.000 | 1.000 |
Remove Method Modifier | 111 | 0 | 0 | 1.000 | 1.000 |
Add Attribute Modifier | 142 | 0 | 0 | 1.000 | 1.000 |
Remove Attribute Modifier | 143 | 0 | 0 | 1.000 | 1.000 |
Add Variable Modifier | 135 | 0 | 0 | 1.000 | 1.000 |
Add Parameter Modifier | 132 | 0 | 0 | 1.000 | 1.000 |
Remove Variable Modifier | 61 | 0 | 0 | 1.000 | 1.000 |
Remove Parameter Modifier | 39 | 0 | 0 | 1.000 | 1.000 |
Change Class Access Modifier | 77 | 0 | 0 | 1.000 | 1.000 |
Add Class Modifier | 36 | 0 | 0 | 1.000 | 1.000 |
Remove Class Modifier | 45 | 0 | 0 | 1.000 | 1.000 |
Split Package | 4 | 0 | 0 | 1.000 | 1.000 |
Merge Package | 2 | 0 | 0 | 1.000 | 1.000 |
Change Type Declaration Kind | 6 | 0 | 0 | 1.000 | 1.000 |
Collapse Hierarchy | 1 | 0 | 0 | 1.000 | 1.000 |
Replace Loop With Pipeline | 35 | 0 | 0 | 1.000 | 1.000 |
Replace Pipeline With Loop | 2 | 0 | 0 | 1.000 | 1.000 |
Replace Anonymous With Lambda | 45 | 0 | 0 | 1.000 | 1.000 |
Merge Class | 6 | 0 | 0 | 1.000 | 1.000 |
Split Class | 3 | 0 | 0 | 1.000 | 1.000 |
Split Conditional | 19 | 0 | 0 | 1.000 | 1.000 |
Invert Condition | 33 | 0 | 0 | 1.000 | 1.000 |
Merge Conditional | 14 | 0 | 0 | 1.000 | 1.000 |
Merge Catch | 2 | 0 | 0 | 1.000 | 1.000 |
Merge Method | 3 | 0 | 0 | 1.000 | 1.000 |
Split Method | 5 | 0 | 0 | 1.000 | 1.000 |
Move Code | 16 | 0 | 0 | 1.000 | 1.000 |
Assert Throws | 14 | 0 | 0 | 1.000 | 1.000 |
Try With Resources | 4 | 0 | 0 | 1.000 | 1.000 |
Replace Generic With Diamond | 77 | 0 | 0 | 1.000 | 1.000 |
Replace Conditional With Ternary | 8 | 0 | 0 | 1.000 | 1.000 |
Since release 3.0.0, RefactoringMiner requires Java 17 or newer and Gradle 7.4 or newer.
In order to build the project, run ./gradlew jar
(or gradlew jar
, in Windows) in the project's root directory.
Alternatively, you can generate a complete distribution zip including all runtime dependencies running ./gradlew distZip
.
You can also work with the project with Eclipse IDE. First, run ./gradlew eclipse
to generate Eclipse project metadata files. Then, import it into Eclipse using the Import Existing Project feature.
As of release 3.0, all RefactoringMiner tests have been migrated to JUnit 5 and do not require any more to clone repositories. Moreover, all unit tests can be executed in parallel. The more CPU cores, the faster the test suites will execute. The available test suites are:
Since version 2.0, RefactoringMiner is available in the Maven Central Repository. In order to use RefactoringMiner as a maven dependency in your project, add the following snippet to your project's build configuration file:
pom.xml
<dependency>
<groupId>com.github.tsantalis</groupId>
<artifactId>refactoring-miner</artifactId>
<version>3.0.7</version>
</dependency>
build.gradle
implementation 'com.github.tsantalis:refactoring-miner:3.0.7'
Since version 3.0, RefactoringMiner is available in DockerHub. A new image is created automatically on every Monday midnight. You can find detailed instructions on how to install and use the image at Docker README.
If you want to get refactoring information when inspecting a commit on GitHub, you can install our Refactoring Aware Commit Review Chrome extension.
The Chrome extension can detect refactorings for public projects and commits matching the following URL patterns:
https://github.com/user/project/commit/id
https://github.com/user/project/pull/id/commits/id
When you build a distributable application with ./gradlew distZip
, you can run Refactoring Miner as a command line application. Extract the file under build/distribution/RefactoringMiner-version.zip
in the desired location, and cd into the bin
folder (or include it in your path). Then, run RefactoringMiner -h
to show its usage:
> ./RefactoringMiner -h
-h Show options
-a <git-repo-folder> <branch> -json <path-to-json-file> Detect all refactorings at <branch> for <git-repo-folder>. If <branch> is not specified, commits from all branches are analyzed.
-bc <git-repo-folder> <start-commit-sha1> <end-commit-sha1> -json <path-to-json-file> Detect refactorings between <start-commit-sha1> and <end-commit-sha1> for project <git-repo-folder>
-bt <git-repo-folder> <start-tag> <end-tag> -json <path-to-json-file> Detect refactorings between <start-tag> and <end-tag> for project <git-repo-folder>
-c <git-repo-folder> <commit-sha1> -json <path-to-json-file> Detect refactorings at specified commit <commit-sha1> for project <git-repo-folder>
-gc <git-URL> <commit-sha1> <timeout> -json <path-to-json-file> Detect refactorings at specified commit <commit-sha1> for project <git-URL> within the given <timeout> in seconds. All required information is obtained directly from GitHub using the OAuth token in github-oauth.properties
-gp <git-URL> <pull-request> <timeout> -json <path-to-json-file> Detect refactorings at specified pull request <pull-request> for project <git-URL> within the given <timeout> in seconds for each commit in the pull request. All required information is obtained directly from GitHub using the OAuth token in github-oauth.properties
With a locally cloned repository, run:
> git clone https://github.com/danilofes/refactoring-toy-example.git refactoring-toy-example
> ./RefactoringMiner -c refactoring-toy-example 36287f7c3b09eff78395267a3ac0d7da067863fd
If you don't want to clone locally the repository, run:
> ./RefactoringMiner -gc https://github.com/danilofes/refactoring-toy-example.git 36287f7c3b09eff78395267a3ac0d7da067863fd 10
For all options you can add the -json <path-to-json-file>
command arguments to save the JSON output in a file. The results are appended to the file after each processed commit.
For the -gc
and -gp
options you must provide a valid OAuth token in the github-oauth.properties
file stored in the bin
folder.
You can generate an OAuth token in GitHub Settings
-> Developer settings
-> Personal access tokens
.
In both cases, you will get the output in JSON format:
{
"commits": [{
"repository": "https://github.com/danilofes/refactoring-toy-example.git",
"sha1": "36287f7c3b09eff78395267a3ac0d7da067863fd",
"url": "https://github.com/danilofes/refactoring-toy-example/commit/36287f7c3b09eff78395267a3ac0d7da067863fd",
"refactorings": [{
"type": "Pull Up Attribute",
"description": "Pull Up Attribute private age : int from class org.animals.Labrador to class org.animals.Dog",
"leftSideLocations": [{
"filePath": "src/org/animals/Labrador.java",
"startLine": 5,
"endLine": 5,
"startColumn": 14,
"endColumn": 21,
"codeElementType": "FIELD_DECLARATION",
"description": "original attribute declaration",
"codeElement": "age : int"
}],
"rightSideLocations": [{
"filePath": "src/org/animals/Dog.java",
"startLine": 5,
"endLine": 5,
"startColumn": 14,
"endColumn": 21,
"codeElementType": "FIELD_DECLARATION",
"description": "pulled up attribute declaration",
"codeElement": "age : int"
}]
},
{
"type": "Pull Up Attribute",
"description": "Pull Up Attribute private age : int from class org.animals.Poodle to class org.animals.Dog",
"leftSideLocations": [{
"filePath": "src/org/animals/Poodle.java",
"startLine": 5,
"endLine": 5,
"startColumn": 14,
"endColumn": 21,
"codeElementType": "FIELD_DECLARATION",
"description": "original attribute declaration",
"codeElement": "age : int"
}],
"rightSideLocations": [{
"filePath": "src/org/animals/Dog.java",
"startLine": 5,
"endLine": 5,
"startColumn": 14,
"endColumn": 21,
"codeElementType": "FIELD_DECLARATION",
"description": "pulled up attribute declaration",
"codeElement": "age : int"
}]
},
{
"type": "Pull Up Method",
"description": "Pull Up Method public getAge() : int from class org.animals.Labrador to public getAge() : int from class org.animals.Dog",
"leftSideLocations": [{
"filePath": "src/org/animals/Labrador.java",
"startLine": 7,
"endLine": 9,
"startColumn": 2,
"endColumn": 3,
"codeElementType": "METHOD_DECLARATION",
"description": "original method declaration",
"codeElement": "public getAge() : int"
}],
"rightSideLocations": [{
"filePath": "src/org/animals/Dog.java",
"startLine": 7,
"endLine": 9,
"startColumn": 2,
"endColumn": 3,
"codeElementType": "METHOD_DECLARATION",
"description": "pulled up method declaration",
"codeElement": "public getAge() : int"
}]
},
{
"type": "Pull Up Method",
"description": "Pull Up Method public getAge() : int from class org.animals.Poodle to public getAge() : int from class org.animals.Dog",
"leftSideLocations": [{
"filePath": "src/org/animals/Poodle.java",
"startLine": 7,
"endLine": 9,
"startColumn": 2,
"endColumn": 3,
"codeElementType": "METHOD_DECLARATION",
"description": "original method declaration",
"codeElement": "public getAge() : int"
}],
"rightSideLocations": [{
"filePath": "src/org/animals/Dog.java",
"startLine": 7,
"endLine": 9,
"startColumn": 2,
"endColumn": 3,
"codeElementType": "METHOD_DECLARATION",
"description": "pulled up method declaration",
"codeElement": "public getAge() : int"
}]
}
]
}]
}
When you build a distributable application with ./gradlew distZip
, you can run Refactoring Miner as a command line application. Extract the file under build/distribution/RefactoringMiner-version.zip
in the desired location, and cd into the bin
folder (or include it in your path). Then, run RefactoringMiner diff -h
to show its usage:
> ./RefactoringMiner diff -h
--url <commit-url> Run the diff with a GitHub commit url
--url <pr-url> Run the diff with a GitHub PullRequest url
--src <folder1> --dst <folder2> Run the diff with two local directories
--repo <repo-folder-path> --commit <commitID> Run the diff with a locally cloned GitHub repository
Each command creates a jetty server instance to visualize the AST diff in your web browser http://127.0.0.1:6789
To export the mappings/actions, add --export
to the end of the command. The files are saved by default in the RefactoringMiner bin
directory.
For example, to visualize the diff of a GitHub Pull Request, run
> ./RefactoringMiner diff --url https://github.com/JabRef/jabref/pull/11180
To visualize the diff of a GitHub commit, run
> ./RefactoringMiner diff --url https://github.com/JetBrains/intellij-community/commit/7ed3f273ab0caf0337c22f0b721d51829bb0c877
For the --url
option you must provide a valid OAuth token in the github-oauth.properties
file stored in the bin
folder.
You can generate an OAuth token in GitHub Settings
-> Developer settings
-> Personal access tokens
.
If you are using RefactoringMiner in your research, please cite the following papers:
Nikolaos Tsantalis, Matin Mansouri, Laleh Eshkevari, Davood Mazinanian, and Danny Dig, "Accurate and Efficient Refactoring Detection in Commit History," 40th International Conference on Software Engineering (ICSE 2018), Gothenburg, Sweden, May 27 - June 3, 2018.
@inproceedings{Tsantalis:ICSE:2018:RefactoringMiner,
author = {Tsantalis, Nikolaos and Mansouri, Matin and Eshkevari, Laleh M. and Mazinanian, Davood and Dig, Danny},
title = {Accurate and Efficient Refactoring Detection in Commit History},
booktitle = {Proceedings of the 40th International Conference on Software Engineering},
series = {ICSE '18},
year = {2018},
isbn = {978-1-4503-5638-1},
location = {Gothenburg, Sweden},
pages = {483--494},
numpages = {12},
url = {http://doi.acm.org/10.1145/3180155.3180206},
doi = {10.1145/3180155.3180206},
acmid = {3180206},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {Git, Oracle, abstract syntax tree, accuracy, commit, refactoring},
}
Nikolaos Tsantalis, Ameya Ketkar, and Danny Dig, "RefactoringMiner 2.0," IEEE Transactions on Software Engineering, vol. 48, no. 3, pp. 930-950, March 2022.
@article{Tsantalis:TSE:2020:RefactoringMiner2.0,
author={Tsantalis, Nikolaos and Ketkar, Ameya and Dig, Danny},
title={RefactoringMiner 2.0},
journal={IEEE Transactions on Software Engineering},
year={2022},
volume={48},
number={3},
pages={930-950},
doi={10.1109/TSE.2020.3007722}
}
Pouria Alikhanifard and Nikolaos Tsantalis, "A Novel Refactoring and Semantic Aware Abstract Syntax Tree Differencing Tool and a Benchmark for Evaluating the Accuracy of Diff Tools," ACM Transactions on Software Engineering and Methodology, 2024. (submitted Major revision)
@misc{Alikhanifard:TOSEM:2024:RefactoringMiner3.0,
title={A Novel Refactoring and Semantic Aware Abstract Syntax Tree Differencing Tool and a Benchmark for Evaluating the Accuracy of Diff Tools},
author={Pouria Alikhanifard and Nikolaos Tsantalis},
year={2024},
eprint={2403.05939},
archivePrefix={arXiv},
primaryClass={cs.SE}
}
Keynote at the Fifth International Workshop on Refactoring (IWoR 2021)
RefactoringMiner has been used in the following studies:
RefactoringMiner can automatically detect refactorings in the entire history of git repositories, between specified commits or tags, or at specified commits.
In the code snippet below we demonstrate how to print all refactorings performed in the toy project https://github.com/danilofes/refactoring-toy-example.git.
GitService gitService = new GitServiceImpl();
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
Repository repo = gitService.cloneIfNotExists(
"tmp/refactoring-toy-example",
"https://github.com/danilofes/refactoring-toy-example.git");
miner.detectAll(repo, "master", new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});
You can also analyze between commits using detectBetweenCommits
or between tags using detectBetweenTags
. RefactoringMiner will iterate through all non-merge commits from start commit/tag to end commit/tag.
// start commit: 819b202bfb09d4142dece04d4039f1708735019b
// end commit: d4bce13a443cf12da40a77c16c1e591f4f985b47
miner.detectBetweenCommits(repo,
"819b202bfb09d4142dece04d4039f1708735019b", "d4bce13a443cf12da40a77c16c1e591f4f985b47",
new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});
// start tag: 1.0
// end tag: 1.1
miner.detectBetweenTags(repo, "1.0", "1.1", new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});
It is possible to analyze a specifc commit using detectAtCommit
instead of detectAll
. The commit
is identified by its SHA key, such as in the example below:
miner.detectAtCommit(repo, "05c1e773878bbacae64112f70964f4f2f7944398", new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});
It is possible to detect refactorings between the Java files in two directories containing the code before and after some changes. This feature supports the detection of renamed and moved classes, and automatically excludes from the analysis any files with identical contents:
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
// You must provide absolute paths to the directories. Relative paths will cause exceptions.
File dir1 = new File("/home/user/tmp/v1");
File dir2 = new File("/home/user/tmp/v2");
miner.detectAtDirectories(dir1, dir2, new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
// You must provide absolute paths to the directories. Relative paths will cause exceptions.
Path dir1 = Paths.get("/home/user/tmp/v1");
Path dir1 = Paths.get("/home/user/tmp/v2");
miner.detectAtDirectories(dir1, dir2, new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});
You can provide two maps (before and after the changes) where the keys are file paths, and the values are the corresponding file contents.
Each key should correspond to a file path starting from the root of the repository. For example, src/org/refactoringminer/api/GitHistoryRefactoringMiner.java
.
After populating the maps, you can use the following code snippet:
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
// Each key should correspond to a file path starting from the root of the repository
Map<String, String> fileContentsBefore;
Map<String, String> fileContentsAfter;
// populate the maps
miner.detectAtFileContents(fileContentsBefore, fileContentsAfter, new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});
To use this API, please provide a valid OAuth token in the github-oauth.properties
file.
You can generate an OAuth token in GitHub Settings
-> Developer settings
-> Personal access tokens
.
If you don't want to clone locally the repository, you can use the following code snippet:
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
miner.detectAtCommit("https://github.com/danilofes/refactoring-toy-example.git",
"36287f7c3b09eff78395267a3ac0d7da067863fd", new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
}, 10);
To use this API, please provide a valid OAuth token in the github-oauth.properties
file.
You can generate an OAuth token in GitHub Settings
-> Developer settings
-> Personal access tokens
.
If you want to analyze all commits of a pull request, you can use the following code snippet:
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
String repo = "https://github.com/apache/drill.git";
miner.detectAtPullRequest(repo, 1807, new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
}, 100);
RefactoringMiner is actually the only tool that generates AST diff at commit level, supports multi-mappings (one-to-many, many-to-one, many-to-many mappings), matches AST nodes of different AST types, and supports semantic diff in a fully refactoring-aware fashion. You can explore its advanced AST diff capabilities in our AST Diff Gallery.
All AST Diff APIs return a ProjectASTDiff
object. By calling getDiffSet()
on it, you can obtain a
Set<ASTDiff>
, where each ASTDiff object corresponds to a pair of Java Compilation Units.
ASTDiff
extends com.github.gumtreediff.actions.Diff
and thus it is compatible with the GumTree core APIs.
More detailed documentation can be found in GitHistoryRefactoringMiner JavaDoc.
// With a locally cloned git repository
GitService gitService = new GitServiceImpl();
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
Repository repo = gitService.cloneIfNotExists(
"tmp/refactoring-toy-example",
"https://github.com/danilofes/refactoring-toy-example.git");
ProjectASTDiff projectASTDiff = miner.diffAtCommit(repo,
"36287f7c3b09eff78395267a3ac0d7da067863fd");
Set<ASTDiff> diffs = projectASTDiff.getDiffSet();
// To visualize the diff add the following line
new WebDiff(projectASTDiff).run();
To use the following API, please provide a valid OAuth token in the github-oauth.properties
file.
You can generate an OAuth token in GitHub Settings
-> Developer settings
-> Personal access tokens
.
// With all information fetched directly from GitHub
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
String repo = "https://github.com/danilofes/refactoring-toy-example.git";
ProjectASTDiff projectASTDiff = miner.diffAtCommit(repo,
"36287f7c3b09eff78395267a3ac0d7da067863fd", 10);
Set<ASTDiff> diffs = projectASTDiff.getDiffSet();
// To visualize the diff add the following line
new WebDiff(projectASTDiff).run();
To use the following API, please provide a valid OAuth token in the github-oauth.properties
file.
You can generate an OAuth token in GitHub Settings
-> Developer settings
-> Personal access tokens
.
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
String repo = "https://github.com/JabRef/jabref.git";
int PR = 10847;
ProjectASTDiff projectASTDiff = miner.diffAtPullRequest(repo, PR, 100);
Set<ASTDiff> diffs = projectASTDiff.getDiffSet();
// To visualize the diff add the following line
new WebDiff(projectASTDiff).run();
// With two directories containing Java source code (File API)
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
// You must provide absolute paths to the directories. Relative paths will cause exceptions.
File dir1 = new File("/home/user/tmp/v1");
File dir2 = new File("/home/user/tmp/v2");
ProjectASTDiff projectASTDiff = miner.diffAtDirectories(dir1, dir2);
Set<ASTDiff> diffs = projectASTDiff.getDiffSet();
// To visualize the diff add the following line
new WebDiff(projectASTDiff).run();
// With two directories containing Java source code (Path API)
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
// You must provide absolute paths to the directories. Relative paths will cause exceptions.
Path dir1 = Paths.get("/home/user/tmp/v1");
Path dir1 = Paths.get("/home/user/tmp/v2");
ProjectASTDiff projectASTDiff = miner.diffAtDirectories(dir1, dir2);
Set<ASTDiff> diffs = projectASTDiff.getDiffSet();
// To visualize the diff add the following line
new WebDiff(projectASTDiff).run();
All classes implementing the Refactoring
interface include refactoring-specific location information.
For example, ExtractOperationRefactoring
offers the following methods:
getSourceOperationCodeRangeBeforeExtraction()
: Returns the code range of the source method in the parent commitgetSourceOperationCodeRangeAfterExtraction()
: Returns the code range of the source method in the child commitgetExtractedOperationCodeRange()
: Returns the code range of the extracted method in the child commitgetExtractedCodeRangeFromSourceOperation()
: Returns the code range of the extracted code fragment from the source method in the parent commitgetExtractedCodeRangeToExtractedOperation()
: Returns the code range of the extracted code fragment to the extracted method in the child commitgetExtractedOperationInvocationCodeRange()
: Returns the code range of the invocation to the extracted method inside the source method in the child commitEach method returns a CodeRange
object including the following properties:
String filePath
int startLine
int endLine
int startColumn
int endColumn
Alternatively, you can use the methods List<CodeRange> leftSide()
and List<CodeRange> rightSide()
to get a list of CodeRange
objects for the left side (i.e., parent commit) and right side (i.e., child commit) of the refactoring, respectively.
All method-related refactoring (Extract/Inline/Move/Rename/ExtractAndMove Operation) objects come with a UMLOperationBodyMapper
object, which can be obtained by calling method getBodyMapper()
on the refactoring object.
Let's consider the Extract Method refactoring in commit JetBrains/intellij-community@7ed3f27
ExtractOperationRefactoring refactoring = ...;
UMLOperationBodyMapper mapper = refactoring.getBodyMapper();
List<StatementObject> newLeaves = mapper.getNonMappedLeavesT2(); //newly added leaf statements
List<CompositeStatementObject> newComposites = mapper.getNonMappedInnerNodesT2(); //newly added composite statements
List<StatementObject> deletedLeaves = mapper.getNonMappedLeavesT1(); //deleted leaf statements
List<CompositeStatementObject> deletedComposites = mapper.getNonMappedInnerNodesT1(); //deleted composite statements
For the Extract Method Refactoring example shown above mapper.getNonMappedLeavesT2()
returns the following statements:
final String url = pageNumber == 0 ? "courses" : "courses?page=" + String.valueOf(pageNumber);
final CoursesContainer coursesContainer = getFromStepic(url,CoursesContainer.class);
return coursesContainer.meta.containsKey("has_next") && coursesContainer.meta.get("has_next") == Boolean.TRUE;
ExtractOperationRefactoring refactoring = ...;
UMLOperationBodyMapper mapper = refactoring.getBodyMapper();
for(AbstractCodeMapping mapping : mapper.getMappings()) {
AbstractCodeFragment fragment1 = mapping.getFragment1();
AbstractCodeFragment fragment2 = mapping.getFragment2();
Set<Replacement> replacements = mapping.getReplacements();
for(Replacement replacement : replacements) {
String valueBefore = replacement.getBefore();
String valueAfter = replacement.getAfter();
ReplacementType type = replacement.getType();
}
}
For the Extract Method Refactoring example shown above mapping.getReplacements()
returns the following AST node replacement for the pair of matched statements:
final List<CourseInfo> courseInfos = getFromStepic("courses",CoursesContainer.class).courses;
final List<CourseInfo> courseInfos = coursesContainer.courses;
Replacement: getFromStepic("courses",CoursesContainer.class)
-> coursesContainer
ReplacementType: VARIABLE_REPLACED_WITH_METHOD_INVOCATION
ExtractOperationRefactoring refactoring = ...;
UMLOperationBodyMapper mapper = refactoring.getBodyMapper();
Set<Refactoring> overlappingRefactorings = mapper.getRefactorings();
For the Extract Method Refactoring example shown above mapper.getRefactorings()
returns the following refactoring:
Extract Variable coursesContainer : CoursesContainer
in method
private addCoursesFromStepic(result List<CourseInfo>, pageNumber int) : boolean
from class com.jetbrains.edu.stepic.EduStepicConnector
because variable coursesContainer = getFromStepic(url,CoursesContainer.class)
has been extracted from the following statement of the original method by replacing string literal "courses"
with variable url
:
final List<CourseInfo> courseInfos = getFromStepic("courses",CoursesContainer.class).courses;