TheLethalCode / Artemis-arrow

14 stars 31 forks source link

Added myanimelist.py to get anime list #26

Closed CircleJerkHug closed 5 years ago

CircleJerkHug commented 5 years ago

Added a code to fetch and scrape the data from myanimelist.net using requests and BeautifulSoup4 module. Extracted the data i.e Name of the Anime and rating, however unable to fetch the data which appears on hovering the text, will be figured it out later in progress. The fetched data is saved in anime_scrape.json file which is to be used later. Also deleted the previous code that were worthless.

TheLethalCode commented 5 years ago

Ok good, you are going in the right direction. But there are certain important things you have to have in mind when you are coding in python. 1) Do not have everything in global space. Compartmentalize your program into functions. Always keep this in mind. 2) Use if name == 'main' in your program. https://stackoverflow.com/questions/419163/what-does-if-name-main-do If you do not understand how to use it, check out the other programs. 3) Try not to exceed any line with more than 80 characters. Keep the above things in mind. Now, coming to the important thing, you are scraping the anime by most popular, and you end up scraping only the first page. That is not what we want. We want to scrape this link http://myanimelist.net/anime/ where index number varies from 1 to some 30000. Each anime has a separate index number. From the web page of that anime, scrape all things you think is important. Then dump them in a json file. Feel free to experiment with requests library.