s-rah / onionscan

OnionScan is a free and open source tool for investigating the Dark Web.
https://twitter.com/OnionScan
Other
2.89k stars 599 forks source link

ERROR: Unknown hidden service type #162

Open HubGrit opened 5 years ago

HubGrit commented 5 years ago

If I do sudo $GOPATH/bin/onionscan facebookcorewwwi.onion the program runs ok, as it does with other existing onion sites. But when I run my own onion site that i can view ok from the Tor browser(from a different device and network) and I run onionscan on it i get ERROR: Unknown hidden service type. Would this be because I haven't published my onion site anywhere? Any advice would be appreciated.

LordEvron commented 5 years ago

This happen because you are using a V3 .onion address that is 56 char long instead of 16... To fix that navigate into the project source file and edit the file called "onionscan/utils/validation.go"

Change this line --> matched, := regexp.MatchString((^|\.)[a-z2-7]{16}\.onion$, identifier) TO: matched, := regexp.MatchString((^|\.)[a-z2-7]{16,56}\.onion$, identifier)

basically just add ",56" near the 16 so it will match all the length from 16 to 56...

rebuild the project eg $ go install github.com/s-rah/onionscan and try again.. now it should work..

Notice that this is a quick fix only, since it match ANY the address FROM 16 To 56 chars length. A better fix should match ONLY 16 or 56 length..

OxMarco commented 5 years ago

The suggested fix doesn't work. I even tried replacing the entire validation function so that it would always return true

LordEvron commented 5 years ago

That is strange.. Are you sure you are recompiling the code? Try to add a print statement inside the validation function and see if the new line get printed when you execute the application..

OxMarco commented 5 years ago

I tried to add a message and compile with go install onionscan but it does not work. Maybe the Unknown hidden service type error pops up somewhere else.

LordEvron commented 5 years ago

What does not work? Do you see your new line printed? I suspect that you are not rebuilding the project (look at the build logs). Remove .EXE file and rebuild with $go build

If you still get the message error message but you are not seeing validation print statement, try to add a new print statement in the beginning of the code, and see if that gets printed.

musicin3d commented 4 years ago

For me, making the suggested edit solves the issue. I have tested by running the source directly and by (re)compiling. It seems like this should be supported. Has someone already make a PR?

... Last commit was 3 years ago. Is this still being maintained?

LordEvron commented 4 years ago

There are already several PR trying to fix this issue. In some of them i even suggest a better way to handle it like in this: https://github.com/s-rah/onionscan/pull/161

But I do not think this project is maintained anymore.

FoxHound0x00 commented 3 years ago

This happen because you are using a V3 .onion address that is 56 char long instead of 16... To fix that navigate into the project source file and edit the file called "onionscan/utils/validation.go"

Change this line --> matched, := regexp.MatchString((^|\.)[a-z2-7]{16}\.onion$, identifier) TO: matched, := regexp.MatchString((^|\.)[a-z2-7]{16,56}\.onion$, identifier)

basically just add ",56" near the 16 so it will match all the length from 16 to 56...

rebuild the project eg $ go install github.com/s-rah/onionscan and try again.. now it should work..

Notice that this is a quick fix only, since it match ANY the address FROM 16 To 56 chars length. A better fix should match ONLY 16 or 56 length..

Thank you this worked for me

mayben0x commented 3 years ago

This solution doesnt work

ERROR: utils/validation.go:13:38: syntax error: unexpected |, expecting expression utils/validation.go:13:39: invalid character U+005C '\' utils/validation.go:13:64: invalid character U+0024 '$' utils/validation.go:16:2: syntax error: non-declaration statement outside function body

Change this line --> matched, := regexp.MatchString((^|.)[a-z2-7]{16}.onion$, identifier) TO: matched, := regexp.MatchString((^|.)[a-z2-7]{16,56}.onion$, identifier)

FoxSca commented 3 years ago

ERROR: Unknown hidden service type:

some suggestion

StasonJatham commented 3 years ago

Tried the solution and my own code. Neither seems to work. I also compiled it with go build and tried it as well not just go run. Didn't make a difference. v2 Adresses work. Strangely enough, even when changing both returns to true not matter what, it is STILL unknown service.... weird

func IsOnion(identifier string) bool {
    // TODO: At some point we will want to support i2p

    if len(identifier) >= 22 && strings.HasSuffix(identifier, ".onion") {
        matched, _ := regexp.MatchString(`(^|\.)[a-z2-7]{16,56}\.onion$`, identifier)
        //matched, _ := regexp.MatchString(`(^|)(?m)(?:https|http)://[a-z0-9]{56}\.onion.*`, identifier)
        log.Println(matched)
        log.Println(identifier)
        return matched
    }
    return false
}
╰─$ go run . --verbose --torProxyAddress=127.0.0.1:9051 zqktlwiuavvvqqt4ybvgvi7tyo4hjl5xgfuvpdf6otjiycgwqbym2qad.onion          130 ↵
2021/11/11 18:45:20 Starting Scan of zqktlwiuavvvqqt4ybvgvi7tyo4hjl5xgfuvpdf6otjiycgwqbym2qad.onion
2021/11/11 18:45:20 This might take a few minutes..

2021/11/11 18:45:20 ERROR: Unknown hidden service type: zqktlwiuavvvqqt4ybvgvi7tyo4hjl5xgfuvpdf6otjiycgwqbym2qad.onion
StasonJatham commented 3 years ago

So i figured it out. I dowloaded the code then built it. the problem was that go was using onionskin as a library so whatever I was changing int he code didn't do anything because it kept grabbing the "old" code.

I solved it by doing

go mod init github.com/StasonJatham/onionscan
go mod tidy 

Then in pipeline.go I changed the code as above (regex to it accepts new v3 addresses) and Importe "my" library

"github.com/StasonJatham/onionscan/utils"

Now it uses my code instead of the "old" version. Since this is not maintained anymore I will probably just fork it and continue my work there!

Cheers

linparkkin commented 3 years ago

So i figured it out. I dowloaded the code then built it. the problem was that go was using onionskin as a library so whatever I was changing int he code didn't do anything because it kept grabbing the "old" code.

I solved it by doing

go mod init github.com/StasonJatham/onionscan
go mod tidy 

Then in pipeline.go I changed the code as above (regex to it accepts new v3 addresses) and Importe "my" library

"github.com/StasonJatham/onionscan/utils"

Now it uses my code instead of the "old" version. Since this is not maintained anymore I will probably just fork it and continue my work there!

Cheers

Other than what he said, I had to replace the old import dependencies in main.go file, in few word I replaced:

import (
        ..
    "github.com/s-rah/onionscan/onionscan"
        ..
    "github.com/s-rah/onionscan/utils"
    ..
)

with the edited ones, for istance:

import (
        ..
    "<yourmodule>/onionscan/onionscan"
        ..
    "<yourmodule>/onionscan/utils"
    ..
)
mor commented 2 years ago

I solved it by doing

go mod init github.com/StasonJatham/onionscan
go mod tidy 

Then in pipeline.go I changed the code as above (regex to it accepts new v3 addresses) and Importe "my" library

"github.com/StasonJatham/onionscan/utils"

Now it uses my code instead of the "old" version. Since this is not maintained anymore I will probably just fork it and continue my work there!

Cheers

Can't get this to work. Somewhere in the orig code it only wants to build from the /s-rah/ repository and completely ignores changes I push to my forked repository. Everything is there in my fork, local/remote syncd, but the "go build" just always recompiles /s-rah/ code... Where is the secret switch to enable editing and recompiling? Sorry, I'm a complete "go" newb, and a low-level githubber too. Can anyone help?

CypherpunkSamurai commented 2 years ago

Anyone looking for an onionscan version that works with v3 addresses (or trying to add support) feel free to checkout my repo. I've also added ci for automated releases.

Basically, Use grep and sed to replace all s-rah/onionscan module imports to your Github repo. Run this in a bash shell (git bash for windows works too)

 grep -Rrl 'cypherpunksamurai\/onionscan' ./ | xargs sed -Ei 's/cypherpunksamurai\/onionscan/CypherpunkSamurai\/onionscan/g'
harr1424 commented 4 months ago

So i figured it out. I dowloaded the code then built it. the problem was that go was using onionskin as a library so whatever I was changing int he code didn't do anything because it kept grabbing the "old" code.

I solved it by doing

go mod init github.com/StasonJatham/onionscan
go mod tidy 

Then in pipeline.go I changed the code as above (regex to it accepts new v3 addresses) and Importe "my" library

"github.com/StasonJatham/onionscan/utils"

Now it uses my code instead of the "old" version. Since this is not maintained anymore I will probably just fork it and continue my work there!

Cheers

This was very helpful - but I needed to change all references to match my local module github.com/harr1424/onionscan using find and replace.