search

freme-project / e-Entity

Apache License 2.0
1 stars 1 forks source link

[DBpedia-Spotlight] support for more languages #39

Closed m1ci closed 8 years ago

m1ci commented 8 years ago

Currently, DBpedia spotlight integrated in FREME supports only English. Extend DBpedia Spotlight also to other languages.

jnehring commented 8 years ago

Is this really in the scope of FREME 0.4? We discussed

freme-project/technical-discussion#58 freme-project/technical-discussion#57

Also I thought that we are going to introduce new languages into FREME NER instead of into DBPedia Spotlight. Also I did not find this issue in the specification.

koidl commented 8 years ago

This is very important - we are currently only using english on FREME NER but I was always under the impression that it already supports more languages we can test very soon. Is that not the case? I don't think we will be able to switch over to Spotlight if we have a different language requirement?

m1ci commented 8 years ago

This is very important - we are currently only using english on FREME NER but I was always under the impression that it already supports more languages we can test very soon. Is that not the case?

FREME NER supports 6 languages: EN, DE, NL, IT, FR, ES. Due to migration workload we have loaded (for now) only English, as soon as Nilesh recovers, we will load also the other models.

DBpedia Spotlight also supports several languages, however, at the moment only English is integrated in e-Entity. This issue is about integrating also other languages in DBpedia Spotlight.

Please all note, that we don't host and maintain DBpedia Spotlight, in FREME we are using public instance of DBpedia Spotlight. So if that instance is down, we cant do anything with it but contact its maintainers and ask them to fix it.

@jnehring please give us some more days for estimation.

koidl commented 8 years ago

@m1ci sound good - hope Nilesh gets well very soon

@jnehring whats the status on backup solutions. I dont understand the setup enough and maybe this need a different thread in github? Are we 100% depending on dbpedia spotlights public instance with FREME NER? What happens if that goes down? I thought we have our own servers now.... sorry for the many questions... just a bit in the dark about this which is fine for the moment

m1ci commented 8 years ago

whats the status on backup solutions. I dont understand the setup enough and maybe this need a different thread in github? Are we 100% depending on dbpedia spotlights public instance with FREME NER? What happens if that goes down? I thought we have our own servers now.... sorry for the many questions... just a bit in the dark about this which is fine for the moment

hey, FREME NER has nothing to do with DBpedia Spotlight. They are independent from each other. FREME NER is running on our own servers.

jnehring commented 8 years ago

@jnehring whats the status on backup solutions.

@koidl The bad news: we do not have a backup solution right now. We have only one instance of FREME live running and if that goes down you could use FREME dev but this is very unreliable. DBPedia Spotlight is an external service not maintained by us. I think that dbpedia spotlight is not a good backup solution for FREME NER because it is a) not suitable for processing many documents and b) has a different feature set from FREME NER. I do not think that dbpedia spotlight offers the topic detection we are currently working on for FREME NER.

The good news: FREME live is very stable, actually it never crashed and only got restarted when I installed updates.

For 100% security you can install your own copy of FREME on your servers. Right now we are working on simplifying that process, wait for FREME 0.4 and you can easily setup your own FREME NER system.

koidl commented 8 years ago

@jnehring thanks for the answer, wondering though if we should move the backup issue to a new thread. I am getting the impression you want to drop the backup issue? If wripl hosts FREME NER itself we wont need the API anymore. The whole reason (at least from my perspective) to have an API is to avoid us having to set up hosting, figure out the IP routing, make sure we backup etc. pp. Maybe I am miss understanding though. I also don't think its simple to set up. Keep in mind its not only the hosting of the instance its also a fallback logic we would have to develop in case the endpoint is offline. We are fine at the moment due to not having significant mass of paying customer, this however is the opportunity as it buys us time to sort this out. Maybe we can keep the topic open for the moment even if we dont have a clear solution yet?

fsasaki commented 8 years ago

Hi Kevin and all,

we definitely need at the FREME server two instances of FREME NER: one for experimenting, used in FREME dev. And one used in FREME the deployed version. As of writing we have just one and there is a mismatch e.g. between what the languages for FREME NER documented in the 0.4 version page and the reality - currently only English. This would not have happened with two instance of FREME NER. So this should be a hard requirement for FREME 0.4.

2015-09-15 15:25 GMT+02:00 Kevin Koidl notifications@github.com:

@jnehring https://github.com/jnehring thanks for the answer, wondering though if we should move the backup issue to a new thread. I am getting the impression you want to drop the backup issue? If wripl hosts FREME NER itself we wont need the API anymore. The whole reason (at least from my perspective) to have an API is to avoid us having to set up hosting, figure out the IP routing, make sure we backup etc. pp. Maybe I am miss understanding though. I also don't think its simple to set up. Keep in mind its not only the hosting of the instance its also a fallback logic we would have to develop in case the endpoint is offline. We are fine at the moment due to not having significant mass of paying customer, this however is the opportunity as it buys us time to sort this out. Maybe we can keep the topic open for the moment even if we dont have a clear solution yet?

— Reply to this email directly or view it on GitHub https://github.com/freme-project/e-Entity/issues/39#issuecomment-140392441 .

jnehring commented 8 years ago

I did not want to avoid the question but it is not easy to answer. Stability of software is in general a hard topic and (i am getting philosophic here) a 100% reliable computer system does not exist. But through an extensive and thorough quality management we push the reliablity near 100%. I wrote some more thoughts on the stability of FREME in a new issue, lets move the backup / stability discussion there.

jnehring commented 8 years ago

This is no longer up to date