meetDeveloper / freeDictionaryAPI

There was no free Dictionary API on the web when I wanted one for my friend, so I created one.
https://dictionaryapi.dev/
GNU General Public License v3.0
2.63k stars 266 forks source link

Not Found and Bad Gateway response #6

Closed tomslloyd closed 4 years ago

tomslloyd commented 6 years ago

It will not allow me to request the output with my local server i keep getting the following results,

404 Not Found: Requested route ('googledictionaryapi.eu-gb.mybluemix.net') does not exist. Bad Gateway response too.

please fix this,

what are the request limits for your server.

meetDeveloper commented 6 years ago

I tried today itself, it is not giving me that error. Can you check again?

tomslloyd commented 6 years ago

It only happens when i call the URL from PHP to retrieve data, then when it errors i cannot access the address then for some time, i think i might be doing too many requests at once what is the limit per sec/min.

i got this error now at 34/07/2018 08:21 PM GMT 404 Not Found: Requested route ('googledictionaryapi.eu-gb.mybluemix.net') does not exist. and then this 502 Bad Gateway: Registered endpoint failed to handle the request.

can you provide me with help to using your app on my own node.js/composer. i am a noob with them, i have them installed on windows and XAMPP but not too sure how to use them.

meetDeveloper commented 6 years ago

Can you tell from start what do you want to achieve??

meetDeveloper commented 6 years ago

Also You can use https://dictionaryapi.herokuapp.com if the other one is down. I guess high number of requests are not supported, I am having free pack of the hosted service. If you could tell me what do you want to achieve maybe we can find some other way to achieve that.

tomslloyd commented 6 years ago

https://dictionaryapi.herokuapp.com loads the search box, but on search it shows application Error.

i have a list of the oxford dictionary words in a DB sum 300K words, i am trying to use your app to pull the data into that DB so i can then release a full english dictionary API with a load of features,

at the moment it is on a dev server so the url sent is not a reachable url just on local network, i am using php's file_get_contents and have tried curl to access the page. and add the data to DB.

it worked for the first 78 but now keeps me from accessing it from script, only directly from it. soon as the script tries to load it it errors.

i have limited the script to load 1 then wait 60 secs till the next but nothing works it wont let me use the script to search anymore.

meetDeveloper commented 6 years ago

300k is a lot. I can make a server for you such that you use your own ip to access Google dictionary as an api but still 300k would block you for a while. You can try these free options maybe it will help you https://medium.com/@martin.breuss/finding-a-useful-dictionary-api-52084a01503d , also do read the comments of article too. Also may I know what sort of features you are thinking to add . And how did you find my api?

tomslloyd commented 6 years ago

i know it's a pain, that why i cut it down to run a few at a time like 20-50 searches then wait 45-60 secs to make another request. i did look at that exact page a while ago a after looking through them it doesnt provide it all its one or the other and has a limit to requests a month so would take longer to finish, yours actually provides what i want exactly.

i intend to make it like google's old api but with more features like;

and maybe some more based on what i can scrap together to fill the data and provide

i found you searching google dictionary api and a comment you made on stackoverflow. found what you made very useful for my plan but may have blocked myself from it in the process of making it.

meetDeveloper commented 6 years ago

'but may have blocked myself from it in the process of making it.' sorry I could not get meaning of this line .

On Wed, Jul 25, 2018, 10:11 AM Tom L notifications@github.com wrote:

i know it's a pain, that why i cut it down to run a few at a time like 20-50 searches then wait 45-60 secs to make another request. i did look at that exact page a while ago a after looking through them it doesnt provide it all its one or the other and has a limit to requests a month so would take longer to finish, yours actually provides what i want exactly.

i intend to make it like google's old api but with more features like;

  • word of the day
  • countdown feature - where you add a group of letters and it will find the words that can be made from it
  • search function (like yours/googles old api)
  • search by criteria functionality

and maybe some more based on what i can scrap together to fill the data and provide

i found you searching google dictionary api and a comment you made on stackoverflow. found what you made very useful for my plan but may have blocked myself from it in the process of making it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/meetDeveloper/googleDictionaryAPI/issues/6#issuecomment-407631259, or mute the thread https://github.com/notifications/unsubscribe-auth/Ae8z7DDASRuL9tdKZ_ivxFROuwunnLKYks5uJ_bvgaJpZM4VaW03 .

tomslloyd commented 6 years ago

sorry, i mean, i may have been blocked or had the url i use in my dev server blocked from accessing it, while i was making the API.

meetDeveloper commented 6 years ago

Why do you have want to query all the definition in one go?? Why can't the person who will be using your api query themselves and then your api in backend call my api. That way it won't get blocked.

On Wed, Jul 25, 2018, 11:24 AM Tom L notifications@github.com wrote:

sorry, i mean, i may have been blocked or had the url i use in my dev server blocked from accessing it, while i was making the API.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/meetDeveloper/googleDictionaryAPI/issues/6#issuecomment-407641732, or mute the thread https://github.com/notifications/unsubscribe-auth/Ae8z7HkdqWMqNHFdnMW4427wPoo7v3NAks5uKAgVgaJpZM4VaW03 .

tomslloyd commented 6 years ago

I was thinking the same thing actually and then store the values then so it doesn't need to be recalled. thank you for your help.

tomslloyd commented 6 years ago

could you add to it to include the origin part of the section too.

meetDeveloper commented 6 years ago

Yes I will add it too, Can you tell me all things that you want. Also that origin part is an image. Shall I include that as link like I have included link for audio.

tomslloyd commented 6 years ago

thank you, together i think we can make my API Awesome,

i was looking towards the text below thee image, but including the image too cant hurt could you add the part below noun/verb bit that shows the additional defines like the plural noun too. and is there a way to get translation to another language by request.

other than that, i dont know if you can scrap anything else off it ha.

tomslloyd commented 6 years ago

also add the direct type like noun/verb/conjunction thats above the additional defines

meetDeveloper commented 6 years ago

I can add those things, I have to restructure the API so that it would be easy to visualize and that would take time, I can add origin part quickly. What do you mean by addition define, can you share picture with red arrows marking what other things you want?

tomslloyd commented 6 years ago

Everytime i try to request the site from my dev site it still gives me either of the errors do you have the ability to remove blocked sites from the host. it wont ket me access it.

and sure this is what i mean. github-example

in 3 you see the noun and plural noun parts. some words have them some dont that what i mean

tomslloyd commented 6 years ago

i have made it work on localhost finally so i got the ability to do it without getting blocked while testing, i have created a pull request for my updates to app.js which include these options and fixed a bug for you too. pulling the images might be bad as they are raw data so its a bit much to output i think, but text works just as fine.

meetDeveloper commented 6 years ago

I included your code.

meetDeveloper commented 6 years ago

But it has some problems, search for any word now.

tomslloyd commented 6 years ago

i did i seen the error in the Origin part, where it says Origin before the text, i realized that a while a go, i have fixed it in my fork from you, i can create another pull request, or you can pull it over.

look at the word still in google it has two definitions, i tried to loop the search but cannot get the text for each definition section, just the first section then the second is just the firsts data. need to find a way to push all data to a definitions array to allow for multiple ones, so move all but the word return to an array of that definition.

origin fix

 var origin = definitions.find(".vmod").first().next("div.xpdxpnd").find("div").last().text();
dictionary.origin = origin;

Edit i also got mine to get antonyms from the search if present. check my fork. using this

var antonymsText = $(this).find("div.xpdxpnd td.lr_dct_nyms_ttl + td > span:not([data-log-string='antonyms-more-click'])").text();
var antonyms = antonymsText.split(/,|;/).filter(antonyms => antonyms!= ' ' && antonyms).map(function(item) {
                                        return item.trim();
                                   });
...
if(antonyms.length > 0)
                        newDefinition.antonyms = antonyms;
meetDeveloper commented 6 years ago

Can you create another pull request??

meetDeveloper commented 5 years ago

@tomslloyd I have changed the API a little kindly see the latest post.

tomslloyd commented 5 years ago

@meetDeveloper i noticed that you switched to Oxford Dictionary Searcher, should probable change the name of this repo otherwise might confuse users.

on another hand after review of the Oxford Dictionary API i started looking into there API and have made progress on making an Oxford API PHP Lib to handle all there API calls and params to get results, i may end up attaching this to my project as this way i can make any number of internal calls to the API and not hit any request limit or restrictions.

I will however help in anyway to get your node version able to do the same thing as it did with google but better.

you should not take any ones API Key or ID, instead just offer this as a search input as well if they have one or just use your own for this app with the option of using provided Keys and ID's.

meetDeveloper commented 5 years ago

@tomslloyd How can you make any number of internal calls to the API and not get hit any request limit?

tomslloyd commented 5 years ago

When I say API I mean my own API not the Oxford API .. and mainly due to me actually owning the whole server .. I can alter the request limits.

My plan is to store the info retrieved from Oxford API allowing me to reduce the number of calls to the oxford API. .

meetDeveloper commented 5 years ago

@tomslloyd So why don't you use mine, I can also set up the database to store the words so to not query the API again and again.

tomslloyd commented 5 years ago

I did think about it, but I have a lot more being stored than just the words and definitions. So it makes more sense to do this on my end. Functionality wise I need it to do a lot more than just get definitions so it's better for me to use my own API access to control the data flow and output for what I need. it did seem a bit redundant to call your API that calls another API just to get the same data set that I can get, when I could just use the direct source .

This will give you more requests for other users, as mine will be resource heavy and wouldn't want to place the load on yours or anyones server, but my own, as I know they can take it, i have a few big servers that I can use for this project .

meetDeveloper commented 5 years ago

@tomslloyd Oh, Actually I do not call another API, instead just scrap the results of the Oxford page with identical completely identical definitions like google. Because google also sources data from the Oxford Dictionary. I stopped using google as source, because they contacted me and asked me to change source, that is why I shifted to Oxford. I did not used the Oxford API, as it had limits, so it won't fulfill the purpose, as I did not used any API, this does not make any redundant call to another API.

Also can you tell me what all things do you need, I will include those things, it will probably help other people also.

May I ask for what purpose do you need Dictionary Data?

tomslloyd commented 5 years ago

it time Oxford will ask you to get an API key as it is a break of there terms of use. when it comes to that i have a PHP class for handling there API ill put it up and you can use that.

But for now i need everything like sentences, sound file link, all text about each word, synonyms, antonyms, origin, alternate know as words. everything the old one use to do that you pulled when i wrote a patch to include a bit more details.

you already have the more than one definition for a word scrape working so thats good, doesnt work for the word still

tomslloyd commented 5 years ago

you should look into the full amount of data you can get from their API they only show a small amount compared to if you used there API you could offer them way more.

even with their limits you could get so much more.

meetDeveloper commented 5 years ago

@tomslloyd It works for still also, see again.

tomslloyd commented 5 years ago

this is the data return from still from the oxford API , its a lot more than what yours shows, if you get an api key use a database to store this json then reserve if called. it would be better for you and still has language support you just set up language tables as well.

{
metadata: {
provider: "Oxford University Press"
},
results: [
{
id: "still",
language: "en",
lexicalEntries: [
{
entries: [
{
etymologies: [
"Old English stille (adjective and adverb), stillan (verb), of West Germanic origin, from a base meaning ‘be fixed, stand’"
],
grammaticalFeatures: [
{
text: "Positive",
type: "Degree"
}
],
homographNumber: "100",
senses: [
{
definitions: [
"not moving or making a sound"
],
examples: [
{
text: "the still body of the young man"
},
{
text: "the sheriff commanded him to stand still and drop the gun"
},
{
text: "she sat very still, her eyes closed"
},
{
text: "he lay still, unable to move"
}
],
id: "m_en_gbus0995370.005",
short_definitions: [
"not moving or making sound"
],
subsenses: [
{
definitions: [
"(of air, water, or the weather) undisturbed by wind, sound, or current; calm and tranquil"
],
examples: [
{
text: "her voice carried on the still air"
},
{
text: "a still autumn day"
}
],
id: "m_en_gbus0995370.009",
short_definitions: [
"undisturbed by wind or current"
],
thesaurusLinks: [
{
entry_id: "still",
sense_id: "t_en_gb0014122.002"
}
]
},
{
crossReferences: [
{
id: "sparkling",
text: "sparkling",
type: "see also"
}
],
definitions: [
"(of a drink) not effervescent."
],
id: "m_en_gbus0995370.010",
regions: [
"British"
],
short_definitions: [
"not effervescent"
]
}
],
thesaurusLinks: [
{
entry_id: "still",
sense_id: "t_en_gb0014122.001"
}
]
}
]
}
],
language: "en",
lexicalCategory: "Adjective",
pronunciations: [
{
audioFile: "http://audio.oxforddictionaries.com/en/mp3/still_gb_1.mp3",
dialects: [
"British English"
],
phoneticNotation: "IPA",
phoneticSpelling: "stɪl"
}
],
text: "still"
},
{
entries: [
{
grammaticalFeatures: [
{
text: "Singular",
type: "Number"
}
],
homographNumber: "101",
senses: [
{
definitions: [
"deep silence and calm; stillness"
],
examples: [
{
text: "the still of the night"
}
],
id: "m_en_gbus0995370.014",
notes: [
{
text: "mass noun",
type: "grammaticalNote"
}
],
short_definitions: [
"deep silence and calm"
],
thesaurusLinks: [
{
entry_id: "still",
sense_id: "t_en_gb0014122.003"
}
]
},
{
definitions: [
"an ordinary static photograph as opposed to a motion picture, especially a single shot from a cinema film"
],
domains: [
"Film",
"Photography"
],
examples: [
{
text: "stills photography"
},
{
text: "film stills"
}
],
id: "m_en_gbus0995370.017",
short_definitions: [
"ordinary static photograph as opposed to motion picture"
]
}
]
},
{
etymologies: [
"mid 16th century: from the rare verb still ‘extract by distillation’, shortening of distil"
],
grammaticalFeatures: [
{
text: "Singular",
type: "Number"
}
],
homographNumber: "500",
senses: [
{
definitions: [
"an apparatus for distilling alcoholic drinks such as whisky."
],
domains: [
"Drink"
],
id: "m_en_gbus0995380.005",
short_definitions: [
"apparatus for distilling alcoholic drinks such as whisky"
]
}
]
}
],
language: "en",
lexicalCategory: "Noun",
pronunciations: [
{
audioFile: "http://audio.oxforddictionaries.com/en/mp3/still_gb_1.mp3",
dialects: [
"British English"
],
phoneticNotation: "IPA",
phoneticSpelling: "stɪl"
}
],
text: "still"
},
{
entries: [
{
grammaticalFeatures: [
{
text: "Positive",
type: "Degree"
}
],
homographNumber: "102",
senses: [
{
definitions: [
"up to and including the present or the time mentioned; even now (or then) as formerly"
],
examples: [
{
text: "he still lives with his mother"
},
{
text: "it was still raining"
}
],
id: "m_en_gbus0995370.022",
short_definitions: [
"up to and including present or time mentioned"
],
subsenses: [
{
definitions: [
"referring to something that will or may happen in the future"
],
examples: [
{
text: "we could still win"
}
],
id: "m_en_gbus0995370.024",
short_definitions: [
"referring to something that will or may happen in future"
]
}
],
thesaurusLinks: [
{
entry_id: "still",
sense_id: "t_en_gb0014122.004"
}
]
},
{
definitions: [
"nevertheless; all the same"
],
examples: [
{
text: "I'm afraid he's crazy. Still, he's harmless"
}
],
id: "m_en_gbus0995370.026",
short_definitions: [
"nevertheless"
],
thesaurusLinks: [
{
entry_id: "still",
sense_id: "t_en_gb0014122.005"
}
]
},
{
definitions: [
"even (used with comparatives for emphasis)"
],
examples: [
{
text: "Hank, already sweltering, began to sweat still more profusely"
},
{
text: "write, or better still, type, captions for the pictures"
}
],
id: "m_en_gbus0995370.028",
short_definitions: [
"even"
]
}
]
}
],
language: "en",
lexicalCategory: "Adverb",
pronunciations: [
{
audioFile: "http://audio.oxforddictionaries.com/en/mp3/still_gb_1.mp3",
dialects: [
"British English"
],
phoneticNotation: "IPA",
phoneticSpelling: "stɪl"
}
],
text: "still"
},
{
entries: [
{
grammaticalFeatures: [
{
text: "Present",
type: "Tense"
}
],
homographNumber: "103",
senses: [
{
definitions: [
"make or become still; quieten"
],
examples: [
{
text: "she raised her hand, stilling Erica's protests"
},
{
text: "the din in the hall stilled"
}
],
id: "m_en_gbus0995370.032",
short_definitions: [
"make or become still"
],
thesaurusLinks: [
{
entry_id: "still",
sense_id: "t_en_gb0014122.006"
},
{
entry_id: "still",
sense_id: "t_en_gb0014122.007"
}
]
}
]
}
],
language: "en",
lexicalCategory: "Verb",
pronunciations: [
{
audioFile: "http://audio.oxforddictionaries.com/en/mp3/still_gb_1.mp3",
dialects: [
"British English"
],
phoneticNotation: "IPA",
phoneticSpelling: "stɪl"
}
],
text: "still"
}
],
type: "headword",
word: "still"
}
]
}
meetDeveloper commented 5 years ago

I meant I get two definitions for still which is what Oxford Shows.

meetDeveloper commented 5 years ago

@tomslloyd How is everything going?

meetDeveloper commented 5 years ago

@tomslloyd I have added origin also.