Fatal transport error: Connection refused (Connection refused)

[ali3assi]

I do the following:

1- I download the .jar from: https://downloads.dbpedia-spotlight.org/spotlight/dbpedia-spotlight-1.0.0.jar 2- I donlwoad the model from https://downloads.dbpedia-spotlight.org/2016-10/en/model/en.tar.gz 3- run the nohup java -jar dbpedia-spotlight-1.0.jar en http://localhost:2222/rest &

In the nohuup.out i have the following log:

538 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryQuantizedCountStore...
689 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (149 ms)
689 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryTokenTypeStore...
1429 [main] INFO org.dbpedia.spotlight.db.memory.MemoryTokenTypeStore - Creating reverse-lookup for Tokens.
2062 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (1372 ms)
2063 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemorySurfaceFormStore...
42082 [main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Summing total SF counts.
44690 [main] INFO org.dbpedia.spotlight.db.memory.MemorySurfaceFormStore - Creating reverse-lookup for surface forms, adding normalized surface forms.
45666 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (43603 ms)
45667 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryResourceStore...
48382 [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Creating reverse-lookup for DBpedia resources.
48996 [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Counting total support...
49172 [main] INFO org.dbpedia.spotlight.db.memory.MemoryResourceStore - Done.
49173 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (3505 ms)
49174 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryCandidateMapStore...
115755 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (66580 ms)
115756 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading MemoryContextStore...
132272 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (16515 ms)
264874 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Loading FSADictionary...
266088 [main] INFO org.dbpedia.spotlight.db.memory.MemoryStore$ - Done (1214 ms)
266887 [main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 1 disambiguators.
266888 [main] INFO org.dbpedia.spotlight.web.rest.Server - Initiated 2 spotters.
May 25, 2018 5:45:30 PM com.sun.grizzly.Controller logVersion
INFO: GRIZZLY0001: Starting Grizzly Framework 1.9.48 - 25/05/18 5:45 PM
Server started in /project/6008168/tamouze listening on http://localhost:2222/rest

When i execute my code i get the following exception:

ERROR 2018-05-25 23:40:46,583 org.spotlight.Main.main() [DbSpotlightClient] - Fatal transport error: Connection refused (Connection refused)
ERROR 2018-05-25 23:40:46,583 org.spotlight.Main.main() [DbSpotlightClient] - POST
org.dbpedia.spotlight.exceptions.AnnotationException: Transport error executing HTTP request.
        at org.spotlight.DbAnnotationClient.request(DbAnnotationClient.java:58)
        at org.spotlight.DbSpotlightClient.extract(DbSpotlightClient.java:48)
        at org.spotlight.Main.main(Main.java:14)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:121)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:610)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:445)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at org.spotlight.DbAnnotationClient.request(DbAnnotationClient.java:42)

my code is :

public class DbSpotlightClient extends DbAnnotationClient {

//  private  String  API_URL    = "http://api.dbpedia-spotlight.org/en/annotate"; 
 private  String  API_URL    = "http://localhost:2222/rest";
    List<ResourceItem>resources;
    public List<ResourceItem> extract(String text)throws AnnotationException{
        String spotlightResponse=null;
        resources = null;
        try{
                //client = HttpClientBuilder.create().build();
                client = new DefaultHttpClient();
                HttpPost post = new HttpPost(API_URL);
                List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
                urlParameters.add(new BasicNameValuePair("text", text));
                urlParameters.add(new BasicNameValuePair("confidence", "0.25"));
                post.setEntity(new UrlEncodedFormEntity(urlParameters));
                post.setHeader("Accept", "application/json");
                spotlightResponse = request(post);
            Announce.message("spotlightResponse= ", spotlightResponse);
        } 
        catch(UnsupportedEncodingException e){
            throw new AnnotationException("Could not encode text.", e);
        } 
        assert spotlightResponse != null;

        try{
            AnnotationUnit annotationUnit = new Gson().fromJson(spotlightResponse.toString(), AnnotationUnit.class);

            resources = annotationUnit.getResources();
        }catch(Exception e){
            throw new AnnotationException("Received invalid response from DBpedia Spotlight API.");
        }
        return resources;
    }

    public static void main(String[] args) throws Exception {
        Announce.doing("Starting DBpediaSpotlightClient main ");
        DbSpotlightClient c = new DbSpotlightClient ();
        File input = new File("./abstract/test1.txt");
        File output = new File("./datasets/out/marsAnnotated.txt");
        c.evaluate(input, output);
        Announce.done();

    }
}

Noting that if i execute the following:

curl http://localhost:2222/rest/annotate \
  -H "Accept: text/xml" \
  --data-urlencode "text=Brazilian state-run giant oil company Petrobras signed a three-year technology and research cooperation agreement with oil service provider Halliburton." \
  --data "confidence=0" \
  --data "support=0"

We get the following:

<?xml version="1.0" encoding="utf-8"?>
<Annotation text="Brazilian state-run giant oil company Petrobras signed a three-year technology and research cooperation agreement with oil service provider Halliburton." confidence="0.0" support="0" types="" sparql="" policy="whitelist">
<Resources>
<Resource URI="http://dbpedia.org/resource/Brazil" support="96461" types="Wikidata:Q6256,Schema:Place,Schema:Country,DBpedia:PopulatedPlace,DBpedia:Place,DBpedia:Location,DBpedia:Country" surfaceForm="Brazilian" offset="0" similarityScore="0.965930684523081" percentageOfSecondRank="0.034833824868363776"/>
<Resource URI="http://dbpedia.org/resource/Giant_star" support="1230" types="" surfaceForm="giant" offset="20" similarityScore="0.8199720139395985" percentageOfSecondRank="0.19329921512879014"/>
<Resource URI="http://dbpedia.org/resource/Petroleum" support="14717" types="" surfaceForm="oil company" offset="26" similarityScore="0.5771024950036353" percentageOfSecondRank="0.7326844635626644"/>
<Resource URI="http://dbpedia.org/resource/Petrobras" support="771" types="Wikidata:Q43229,Wikidata:Q24229398,DUL:SocialPerson,DUL:Agent,Schema:Organization,DBpedia:Organisation,DBpedia:Company,DBpedia:Agent" surfaceForm="Petrobras" offset="38" similarityScore="1.0" percentageOfSecondRank="0.0"/>
<Resource URI="http://dbpedia.org/resource/Sign_language" support="1927" types="" surfaceForm="signed" offset="48" similarityScore="0.7276535296615254" percentageOfSecondRank="0.17575346113982937"/>
<Resource URI="http://dbpedia.org/resource/Technology" support="12462" types="" surfaceForm="technology" offset="68" similarityScore="0.9914358988874088" percentageOfSecondRank="0.00851189308814701"/>
<Resource URI="http://dbpedia.org/resource/Cooperation" support="714" types="" surfaceForm="cooperation" offset="92" similarityScore="0.718889126469838" percentageOfSecondRank="0.3076037508336602"/>
<Resource URI="http://dbpedia.org/resource/Petroleum" support="14717" types="" surfaceForm="oil" offset="119" similarityScore="0.9495596915391201" percentageOfSecondRank="0.05284578455873624"/>
<Resource URI="http://dbpedia.org/resource/Service_provider" support="304" types="" surfaceForm="service provider" offset="123" similarityScore="0.9387022772639211" percentageOfSecondRank="0.05629789285158675"/>
<Resource URI="http://dbpedia.org/resource/Halliburton" support="707" types="Wikidata:Q43229,Wikidata:Q24229398,DUL:SocialPerson,DUL:Agent,Schema:Organization,DBpedia:Organisation,DBpedia:Company,DBpedia:Agent" surfaceForm="Halliburton" offset="140" similarityScore="0.9999999998892122" percentageOfSecondRank="0.0"/>
</Resources>
</Annotation>

Any help please to identify the problem?

[sandroacoelho]

Dear @TamouzeAssi ,

You can use whatever client that supports REST. I have created to you a small example using httpcomponents https://github.com/dbpedia-spotlight/examples

Best,

[ali3assi]

Dear @sandroacoelho ,

Thank you for help. I think the the resourceItem miss two important functions: getUri and getTypes. I was not able to run the code in eclipse without these two function. By the way, can i know please how can for example fixe the confidence parameter and support. In addition i want to get the result including the surfaceForm, etc...

Thank you again and waintin for helpful ideas

[sandroacoelho]

Hi @TamouzeAssi ,

I used lombok (http://www.baeldung.com/intro-to-project-lombok) that provide getters and setters for all private variables in ResourceItem.

For confidence parameter, just add here, something like params.add(new BasicNameValuePair("confidence", "0.5"));

For more info, check https://hc.apache.org/httpcomponents-client-ga/

Best

[ali3assi]

Hi Sir @sandroacoelho,

I have a file to annotate. i tried to parse this file, take a line at time and pass it to

annotationUnit = spotlight.get(line);

For some lines, it is ok, we get perfectly the results, but for some other lines we get the following error (these lines are good,):

com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:226)
        at com.google.gson.Gson.fromJson(Gson.java:927)
        at com.google.gson.Gson.fromJson(Gson.java:892)
        at com.google.gson.Gson.fromJson(Gson.java:841)
        at com.google.gson.Gson.fromJson(Gson.java:813)
        at org.dbpedia.spotlight.rest.Spotlight.get(Spotlight.java:53)
        at org.dbpedia.spotlight.rest.Spotlight.get(Spotlight.java:121)
        at org.dbpedia.spotlight.Example.replace(Example.java:76)
        at org.dbpedia.spotlight.Example.main(Example.java:198)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $
        at com.google.gson.stream.JsonReader.beginObject(JsonReader.java:385)
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:215)
        ... 14 more

Does exists another way to do the job safely (i.e., like for example retry annotation tentative, or may be limit the requests per second )?

Thank you

[sandroacoelho]

Are you running it using our endpoint or localhost? If you are using our endpoint, try to run it locally

[ali3assi]

@sandroacoelho ,

I just run the code you sent, so it uses your endpoint. Can you please sepcify the step to take and how can i execute it locally? what i need?

[sandroacoelho]

Just run our Docker and modify this line to http://localhost:2222/rest/annotate

Best,

[ali3assi]

@sandroacoelho Sorry, you mean changing the url in the code to http://localhost:2222/rest/annotate and then execute docker by typing (only) the following: docker run -i -p 2222:80 dbpedia/spotlight-english spotlight.sh

[sandroacoelho]

Yes. You got it ;)

[ali3assi]

@sandroacoelho Im so sorry for disturbing. Im working on a server which does not accept DOCKER for security. It accept only singularity.

The problem i dont have experience with both. So by chance do you have any documentaion on how we can run dbpedia-spotlight using singularity, please

[sandroacoelho]

Maybe the section Run your own server at https://github.com/dbpedia-spotlight/dbpedia-spotlight-model can help you

[ali3assi]

@sandroacoelho you are the best~ thank you you save my time :)