PR3R00T
commented
1 month ago @0xtavian Hey, I think I found a workaround: Taking the error I found it was at: (https://github.com/sensepost/gowitness/blob/6b10eaeba7a6dcaad4b487fc52f114a952e01295/storage/db.go#L87C1-L87C62), Working backwards while learning Go programming I recompiled with some more debugging commands and found that from line 68 calling the parseDBLocation function was the issue, (https://github.com/sensepost/gowitness/blob/6b10eaeba7a6dcaad4b487fc52f114a952e01295/storage/db.go#L30) checking what the dbLocation and the location.Scheme parameters show the issue quite well, the dblocation parameter will parse in the filename of the db, the if statement of "sqlite" if obtained from the go URL module and its Parse function,
func parseDBLocation(dbLocation string) (*url.URL, string, error) {
// Parse the DB URI.
location, err := url.Parse(dbLocation)
if err != nil {
return nil, "", err
}
// Ensure the sqlite DB file path is correctly parsed via url.Parse
if location.Scheme == "sqlite" {
switch {
case location.Host == "" && location.Path != "":
return location, location.Path, nil
case location.Host != "" && location.Path == "":
return location, location.Host, nil
case location.Host == "" && location.Path == "":
return location, "gowitness.sqlite3", nil
default:
return location, strings.TrimPrefix(dbLocation, "sqlite://"), nil
}
}
return location, dbLocation, niladding in a filename such as blah.sqlite3 the URL module does not give me a scheme back
./gowitness-2.5.1-linux-amd64 merge -i 01.sqlite3 -i 02.sqlite3 -o temp.sqlite3
26 May 2024 11:10:25 INF number of dbs to process database-count=2
DBLocation: temp.sqlite3
Scheme:
Host:
26 May 2024 11:10:25 FTL could not open destination db error="unsupported database URI provided" destination=temp.sqlite3however if you change the filenames to be sqlite///blah.sqlite3 , the scheme is extracted from url.parse to "sqlite" which satisfies the if statement then then the trim (strings.TrimPrefix(dbLocation, "sqlite://")) command kicks in and removes the prefix.
So really your commands should be
./gowitness-2.5.1-linux-amd64 merge -i sqlite:///path/01.sqlite3 -i sqlite:///path/02.sqlite3 -o sqlite:///path/temp.sqlite3
26 May 2024 11:13:42 INF number of dbs to process database-count=2
DBLocation: sqlite://temp.sqlite3
Scheme: sqlite
Host: temp.sqlite3
26 May 2024 11:13:42 INF writing results to a new database db-path=sqlite://temp.sqlite3
26 May 2024 11:13:42 INF processing source database file=sqlite:///path/01.sqlite3
DBLocation: sqlite:///path/01.sqlite3
Scheme: sqlite
Host:
26 May 2024 11:13:43 INF done processing db processed-rows=46 source-db=sqlite:///path/01.sqlite3
26 May 2024 11:13:43 INF processing source database file=sqlite:///path/02.sqlite3
DBLocation: sqlite:///path/02.sqlite3
Scheme: sqlite
Host:
26 May 2024 11:13:43 INF done processing db processed-rows=37 source-db=sqlite:///path/02.sqlite3My guess is that a better logic of file identification is required and url.parse isn't fit for dealing with sqlite file checking
Describe the bug
Merging databases is broken in version 2.5.0 and version 2.5.1. The last working version was 2.4.2.
To Reproduce Steps to reproduce the behavior:
I also tried with version 2.5.0 with the same result
Expected behavior Merging should work with versions higher than 2.4.2
Screenshots
Version Information:
Additional context Add any other context about the problem here.