search

freme-project / freme-ner

Apache License 2.0
6 stars 1 forks source link

domains are not loaded #106

Closed ArneBinder closed 8 years ago

ArneBinder commented 8 years ago

This request

curl -X POST --header "Content-Type: text/n3" --header "Accept: text/n3" "http://api-dev.freme-project.eu/current/e-entity/freme-ner/documents?input=I%20like%20Berlin%2C%20the%20capitol%20of%20Germany!&informat=text&outformat=turtle&language=en&dataset=dbpedia&mode=all&domain=TaaS-2007"

produces

{
  "exception": "java.util.NoSuchElementException",
  "path": "/e-entity/freme-ner/documents",
  "message": "key not found: TaaS-2007",
  "error": "Internal Server Error",
  "status": 500,
  "timestamp": 1464081939173
}

Debuggin the code reveals that the domains map is empty, the domains.csv seems not to be loaded.

jnehring commented 8 years ago

@xFran I wonder why this was not mentioned earlier. Do you still use the domain feature?

x-fran commented 8 years ago

@jnehring Yes, we still use domains.

jnehring commented 8 years ago

@jnehring Yes, we still use domains.

I think the error was not discovered earlier because the domain features just reduces the amount of entities you get. So when we have this issue fixed you will get less noise in the entities.

@ArneBinder will further examine the bug.

ArneBinder commented 8 years ago

The config is loaded correctly, see this: bug But this is never executed succesfully. val inputStream = new FileInputStream(file) in the scala function Source.fromFile(file: JFile, bufferSize: Int) produces a FileNotFoundException.

sandroacoelho commented 8 years ago

Hi @ArneBinder, @jnehring

The bug is here, but I am a little bit lost. Why the Scala code is commented?

jnehring commented 8 years ago

@ArneBinder you commented out the code. Do you think it is ok when we use the code?

ArneBinder commented 8 years ago

The code was unused, so I commented it to avoid any confusion/false debugging. Since the architecture update the scala endpoints are not in use anymore, the enrichment functionality is used directly via the e-link java api which accesses the scala functions in FremeNer.scala. @sandroacoelho do you have any other ideas why the domains.csv file is not loaded and this code fails? As you can see here and below the domain and types parameter is integrated. Debugging revealed that just the list of domains which should be loaded from the domains.csv stays empty.

sandroacoelho commented 8 years ago

Hi all,

Our domains file had no commas between key and label. I have fixed it here and created a test case for this issue. Just one more question: when we fill with an invalid domain, should we throw an exception or ignore it and continue?

Could someone redeploy and teste it again?

Thanks,

jnehring commented 8 years ago

After the release last friday the automatic redeploy stopped to work. Now I have updated this and the system automatically redeploys any changes. So when you commit a change to the FREME NER repository, it should be online about 5 minutes later. Check for more information.

So @ArneBinder you can test this now.

ArneBinder commented 8 years ago

The domaines are still not loaded. I get:

{
  "exception": "java.util.NoSuchElementException",
  "path": "/e-entity/freme-ner/documents",
  "message": "key not found: TaaS-2005",
  "error": "Internal Server Error",
  "status": 500,
  "timestamp": 1468843797105
}

Use the api documentation to reproduce this.

sandroacoelho commented 8 years ago

Hi @jnehring: great job. Thanks for it

@ArneBinder / @jnehring : could you please check if the domains file is up-to-date at the server?

Best,

ArneBinder commented 8 years ago

Hi @sandroacoelho, if it is on github it is also on the jenkins server.

m1ci commented 8 years ago

I think you should write also the domain name like TaaS-2005 Sport and now just the ID.

On Monday, 18 July 2016, ArneBinder notifications@github.com wrote:

The domaines are still not loaded. I get:

{ "exception": "java.util.NoSuchElementException", "path": "/e-entity/freme-ner/documents", "message": "key not found: TaaS-2005", "error": "Internal Server Error", "status": 500, "timestamp": 1468843797105 }

Use the api documentation http://api-dev.freme-project.eu/doc/api-doc/full.html#!/e-Entity/executeFremeNer to reproduce this.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/freme-project/freme-ner/issues/106#issuecomment-233311891, or mute the thread https://github.com/notifications/unsubscribe-auth/AA7rR6plXvICSKxyStHSskqZ17hWyZSIks5qW23dgaJpZM4IlVUk .

Milan Dojchinovski http://dojchinovski.mk

sandroacoelho commented 8 years ago

Hi @m1ci : I've changed it. @ArneBinder : Could you please try again?

Best,

ArneBinder commented 8 years ago

Hi @sandroacoelho,

please use the interactive api documentation to try the request.

But I think the problem is still that the file is not found. Where exactly does the scala code look for it? Currently this domains.csv (from freme-packages/freme-ner-dev/...) will be moved into the config folder which is added to the java classpath. Furthermore, this line (also freme-packages/freme-ner-dev/...) defines the location to look for the domains file. In some way this does not fit together: As mentioned before, this line fails to open the file.

Best, Arne

PS: I just switched to freme.ner.domainsFile=domains.csv which does not work either.

sandroacoelho commented 8 years ago

@ArneBinder when we don't provide a folder as you have put on the config file freme.ner.domainsFile=domains.csv, Scala/Java tries to locate the file in the current directory.

In this commit, I put a graceful error handling do deal with invalid keys. It's strange that we are receiving java.util.NoSuchElementException. Also, I have tested using the interactive API documentation and Freme Broker about 3 hours ago and has worked.

Best,

ArneBinder commented 8 years ago

@sandroacoelho ok, perhaps I did something wrong... what was the exact curl call you have used? I used this:

curl -X POST --header 'Content-Type: text/plain' -d 'Welcome to Berlin, the capital of Germany!' 'http://api-dev.freme-project.eu/current/e-entity/freme-ner/documents?language=en&dataset=dbpedia&mode=all&domain=TaaS-2005'

Did I miss something?

I put a graceful error handling do deal with invalid keys

I agree, this is really strange.

sandroacoelho commented 8 years ago

Hi @ArneBinder :

Could you please try it?

curl -X POST -H "Content-Type: text/plain" -H "Cache-Control: no-cache" -d 'Welcome to the city of Prague' "http://api.freme-project.eu/current/e-entity/freme-ner/documents/?language=en&dataset=dbpedia&domain=TaaS-1510"
ArneBinder commented 8 years ago

@sandroacoelho thanks a lot, it works for me, too (I switched back to freme.ner.domainsFile=config/domains.csv for dev like in the live version)!

Regarding:

I put a graceful error handling do deal with invalid keys

I agree, this is really strange.

I switched to freme-ner:0.8-SNAPSHOT in the freme-parent (which sets the versions of all freme artefacts). Now your changes should be integrated. But are you sure that we should not notify the user if an entered domain id does not exist?

sandroacoelho commented 8 years ago

Hi @ArneBinder

I created a new issue (#124) for 0.8-SNAPSHOT.

But are you sure that we should not notify the user if an entered domain id does not exist?

About invalid domians, I will return the original code.

Best,