meetDeveloper / freeDictionaryAPI

There was no free Dictionary API on the web when I wanted one for my friend, so I created one.
https://dictionaryapi.dev/
GNU General Public License v3.0
2.66k stars 266 forks source link

[IMPORTANT] Moved to Wiktionary - Non english languages removed #102

Open meetDeveloper opened 3 years ago

meetDeveloper commented 3 years ago

tl;dr.

Hello,

I am planning to start using Wiktionary as source for Dictionary API. This will be an replacement to the current source, and due to this languages other than English will have to be dropped. Though for French and Spanish languages I will try to bring support soon.

The original texts of Wiktionary entries are dual-licensed to the public under both the Creative Commons Attribution-ShareAlike 3.0 Unported License (CC-BY-SA) and the GNU Free Documentation License (GFDL), more information about this here. Wiktionary sources information from these public domain sources.

Using Wiktionary as a source will help developers using my API a lot as they won't have to worry about using my API for commercial usage etc.

Currently Wiktionary for English language is pretty solid and we can rely on that, I have compared many entries and see no problem in using that as our source for English language.

The only downside I see is that the API will be exclusively for English language as for other languages as Wiktionary lags behind, but you can expect other languages to be added as soon as possible. Given that majority of users of this API are English speaking users, this change will affect relatively fewer users.

Users that need to have dictionary for other languages for their own personal use can use google search, google provides dictionary for many languages, I will add a detailed write up regarding how to search meaning for different languages on google later. In case you need an API to power your app, I suggest you to look at the Oxford Dictionary API, AFAIK they have Dictionary API that supports many languages.

I will maintain the structure of the API same and changes if any will only be additive in nature.

Thanking You meetDeveloper

rrbadam commented 3 years ago

Fine with it

queleok commented 3 years ago

Would it be an additional source, or a replacement for the current one?

meetDeveloper commented 3 years ago

@queleok This will be an replacement.

meetDeveloper commented 3 years ago

Wikitionary uses following as sources: https://en.wiktionary.org/wiki/Wiktionary:Public_domain_sources

queleok commented 3 years ago

I see, thank you, @meetDeveloper . I've tried analyzing the dictionary of 100k most frequently used English words according to the Wiktionary, there were some odd items like clearly French or German words. Of course, it's not a showstopper for the idea, so I guess it should be fine :)

phibr0 commented 3 years ago

When will this change happen?

ichmoimeyo commented 3 years ago

HI

Have you checked out ...

https://yandex.com/dev/dictionary/ https://yandex.com/legal/dictionary_api/ FREE

http://www.collinslanguage.com/collins-api/ FREE -> 5000 p/m | £75p/m -> 30000 p/m

meetDeveloper commented 3 years ago

@ichmoimeyo They support English which I can get from Wiktionary.

ichmoimeyo commented 3 years ago

@meetDeveloper https://yandex.com/dev/dictionary/doc/dg/reference/getLangs.html [with russian emphasis] http://www.collinslanguage.com/collins-api/ [Bilingual or monolingual/bilingual]

ms-dc commented 3 years ago

any eta to non english language coming back? portuguese in particular

piever commented 3 years ago

The only downside I see is that the API will be exclusively for English language as for other languages as Wiktionary lags behind, but you can expect other languages to be added as soon as possible. Given that majority of users of this API are English speaking users, this change will affect relatively fewer users.

I confess I would love it if more languages were available. At a cursory look, it seems as if Wiktionary is reasonably good in other languages as well (I've just tested some random words on eg Italian, Portuguese and French and the definitions seem fine). Is the problem that the wiktionary API is inconsistent across languages?

Otherwise, I imagine it would cost relatively little to just support other languages as well (even if it turns that Wiktionary lags behind a bit on those, I guess it's still better than not supporting them at all).

103sbavert commented 3 years ago

I can't complain as I pay 0 bucks to use this api but this will break my app. Brazilian Portuguese is already broken :'(

OultimoCoder commented 3 years ago

Yeah I would really prefer the change didn't happen. I am using this API for a language flashcard app and this is super useful for dictionary definitions in a large variety of languages.

1over137 commented 3 years ago

I really think there is no reason to do this. English Wiktionary has its own simple and easy to use API, but other languages are less structured. This is going to break the monolingual dictionary function in my app.

ac-lap commented 3 years ago

Do we have an ETA by when this migration would be complete?

arshsaxena commented 2 years ago

Is this migration completed?

meetDeveloper commented 2 years ago

@arshsaxena @ac-lap @sbeve72 @OultimoCoder @piever This change will happen today. I hope to bring other languages soon but parsing Wiktionary for other languages is a bit difficult and will take some time.

meetDeveloper commented 2 years ago

This migration is completed. Now the API is only for English and using Wiktionary as source also I am sending in response sourceUrls from which the definitions have been parsed and the licenses.

1over137 commented 2 years ago

I think this will make this API completely useless, as English Wiktionary already has its own API.

phibr0 commented 2 years ago

@arshsaxena @ac-lap @sbeve72 @OultimoCoder @piever This change will happen today. I hope to bring other languages soon but parsing Wiktionary for other languages is a bit difficult and will take some time.

If the API were open source I would really like to help here

1over137 commented 2 years ago

wiktionary for other languages is a lot less structured though i remember seeing other projects that parses russian or french ones.

piever commented 2 years ago

Thanks for the notification!

I imagine (correct me if I'm wrong) that at this point, this project is primarily a "conversion layer" between the Wiktionary API and the free dictionary API. Personally, I think that it is valuable, in that it was pretty straightforward for me to work with this API, but it looks like working directly with the Wiktionary API would have been harder.

Sorry about the slight off-topic, but I was wondering... Given that (from what I see in the README) it is costly to keep up servers that provide this API, would it make sense to package this as a plain module that one could load (say from CDN) and use it directly (to convert the result of calls to the Wiktionary API)?

ngocsangyem commented 2 years ago

Does the source code need to be updated @meetDeveloper ?

meetDeveloper commented 2 years ago

@ngocsangyem yes it needs to be updated.

ngocsangyem commented 2 years ago

It is very nice If you update the new source code soon. I need to make some customs for the API to host my app. @meetDeveloper

meetDeveloper commented 2 years ago

@piever Yes you are right, Wiktionary exposes dumps that anyone can parse and use it in their own work. I was also thinking to package this API in a plain module to avoid server costs, this is something I am exploring and will likely start work in few weeks.

Etiene commented 2 years ago

Yeah I would really prefer the change didn't happen. I am using this API for a language flashcard app and this is super useful for dictionary definitions in a large variety of languages.

I agree!

spacelover1 commented 2 years ago

I want to use Oxford dictionary API and have trouble passing API key and ID. I get response using postman but in code it doesn't pass them. Anyone used this API?

abhagsain commented 2 years ago

This migration is completed. Now the API is only for English and using Wiktionary as source also I am sending in response sourceUrls from which the definitions have been parsed and the licenses.

It'd be great if you merge your latest changes to master @meetDeveloper

alexguillamon commented 10 months ago

This is a useful project but others would like to run the server locally and or further make changes can you please push the latest changes to master? @meetDeveloper

AndrewDiMola commented 8 months ago

Has anyone created an adapted list of words that can be retrieved after the migration change? I used to use this file list as a source of truth, but because of the migration, a lot of words no longer retrieve results.

https://github.com/meetDeveloper/freeDictionaryAPI/blob/master/meta/wordList/english.txt

KlavsKlavsen commented 5 months ago

can anyone point to how to add a new language from wictionary to this? its really a great idea to use wictionary as source - hoping I can figure out how to add Hindi - so I can use this (and possibly improve - by using wictionary)