Closed matteofabbri closed 9 years ago
So, what's the reason for this issue? Showing Scrappy? ;)
Feel free to rewrite the script to use it. :)
Just the suggestion that catch itemscopes and properties is far even more simple than looking for simple items :D
Yeah, well. This project started way before something like Scrappy was even released. And: IMDb is living. They most likely didn't have schema.org implemented back then. But yeah, you're right. ;)
Nice job but if I try to scrap a movie (ex:Ironman) with Scrappy (https://www.mashape.com/netfluid-framework/scrappy-web-scraper-for-html5-microdata-semantic-elements) i discover that lot more information are presents
[ { "itemType":"http://schema.org/Movie", "awards":[ "Nominated for 2 Oscars.", "Another 18 wins & 51 nominations." ], "description":[ "Tony Stark. Genius, billionaire, playboy, philanthropist. Son of legendary inventor and weapons contractor Howard Stark. When Tony Stark is assigned to give a weapons presentation to an Iraqi unit led by Lt. Col. James Rhodes, he's given a ride on enemy lines. That ride ends badly when Stark's Humvee that he's riding in is attacked by enemy combatants. He survives - barely - with a chest full of shrapnel and a car battery attached to his heart. In order to survive he comes up with a way to miniaturize the battery and figures out that the battery can power something else. Thus Iron Man is born. He uses the primitive device to escape from the cave in Iraq. Once back home, he then begins work on perfecting the Iron Man suit. But the man who was put in charge of Stark Industries has plans of his own to take over Tony's technology for other matters. Written by halo1k", "After being held captive in an Afghan cave, an industrialist creates a unique weaponized suit of armor to fight against evil. This leads him to conflict within his own company." ], "keywords":[ "Plot Keywords: armor | cave | iron | genius | missile | See All (198) »", "armor", "cave", "iron", "genius", "missile" ], "genre":[ "Genres: Action | Adventure | Sci-Fi", "Action", "Adventure", "Sci-Fi" ], "url":[ "technical?ref_=tt_dt_spec", "/offsite/?page-action=offsite-facebook&token=BCYnyIe9ryC6rTJWzl5VnvWcSJucUcsiAH2QmS10UhlMNt4dnbwSpso2OwpynBXaaovXSVIwnGOf%0D%0AlHxXtkAzXII6cTCy2GFoJ4ZF1X_yt-D_4ZvBAQv0jS11eN1UsuYW0zUePWLilYMls5PDB28IHuXE%0D%0AFuafhnmRxSrdI5Y9FhCM-iGmrgUpASHXwLDIKuzE6yVxvqeemt4ah2KaWqfJL6tL9E8PzsQxDuR%0D%0A1MZmzq3oEhI%0D%0A&ref=tt_pdt_ofs_offsite_0", "/offsite/?page-action=offsite-sapo&token=BCYj_UL3CkGt0wguU0H6sBK3Q9SGCkxFYaKa_mv5lmFlp-oyIcaafCMGkVDEkzLIYmTmaukQBKMe%0D%0AfGywn0m2PiPhF5uCpHlO2eeEnqgauYxqyZyK3Hh1ox83z0q8a22VOoa5dj1OYBCWUmeQ76wuy5Z6%0D%0A7VmUKWLRWqyKapiWs_xrcMkiD9rNxH36-ikY4IjXlzc9ddNQS7G1dop57sMnYkoZj_Zy2xmKojgF%0D%0AtUPz6NmrsU%0D%0A&ref=tt_pdt_ofs_offsite1", "/country/us?ref=tt_dtdt", "/language/en?ref=tt_dtdt", "/language/fa?ref=tt_dtdt", "/language/ur?ref=tt_dtdt", "/language/ar?ref=tt_dtdt", "/search/title?locations=Palmdale+Regional+Airport,+Palmdale,+California,+USA&ref=tt_dtdt", "business?ref=tt_dt_bus", "http://pro.imdb.com/title/tt0371746/companycredits?rf=cons_tt_cocred_tt&ref_=cons_tt_cocred_tt", "http://pro.imdb.com/signup/index.html?rf=cons_tt_cocred_spl&ref_=cons_tt_cocred_spl", "/search/title?soundmixes=sdds&ref=tt_dt_spec", "/search/title?sound_mixes=dolbydigital&ref=tt_dt_spec", "/search/title?soundmixes=dts&ref=tt_dtspec", "/search/title?colors=color&ref=tt_dtspec", "/plugins?titleId=tt0371746&ref=tt_plgrt", "/plugins?titleId=tt0371746&ref=tt_plgrt", "externalsites?ref=tt_dtdt#official", "releaseinfo?ref=tt_dtdt", "releaseinfo?ref=tt_dtdt#akas", "locations?ref=tt_dtdt", "companycredits?ref=tt_dtco" ], "headline":[ "'Avengers 2' Iron Man Mark Xliii Figure Fully Unveiled", "Hot Toys Reveal Their 'Iron Man Mark Xliii' Avengers: Age Of Ultron Action Figure", "New footage from Avengers: Age of Ultron in Audi commercial" ], "contentRating":[ "12A", "12A" ], "audience":{ "itemType":"http://schema.org/Audience", "url":"/title/tt0371746/parentalguide?ref=tt_strypg" }, "creator":[ { "itemType":"http://schema.org/Organization", "url":"/company/co0023400?ref=tt_dtco", "name":"Paramount Pictures" }, { "itemType":"http://schema.org/Organization", "url":"/company/co0095134?ref=tt_dtco", "name":"Marvel Enterprises" }, { "itemType":"http://schema.org/Organization", "url":"/company/co0051941?ref=tt_dtco", "name":"Marvel Studios" }, { "itemType":"http://schema.org/Person", "url":[ "/name/nm1318843/?ref=tt_ovwr", "/name/nm1319757/?ref=tt_ovwr" ], "name":[ "Mark Fergus", "Hawk Ostby" ] } ], "duration":[ "126 min", "126 min" ], "review":{ "itemType":"http://schema.org/Review", "name":"A Nutshell Review: Iron Man", "reviewRating":{ "itemType":"http://schema.org/Rating", "worstRating":"1", "ratingValue":"10", "bestRating":"10" }, "datePublished":"2008-04-30", "reviewBody":"With a little tinge of shame and regret, my rare dalliances with the Iron Man character stemmed from a few one off comic books, as well as occasions during the teenage years of spending time in the arcade with those Marvel games, where Iron Man was one of my preferred characters because it came together with his incredible arsenal of weapons from repulsor beams to this gigantic cannon which accompanied the execution of some complex combo moves. There's something sexy about the red and gold suit of armour, and having an array of weapons at the disposal of a player, makes perfect sense for variety in dispatching your enemies.This may irk the fervent fans of Iron Man, but face it, the superhero belonged to Tier B where superheroes are concerned, languishing behind easily recognizable peers who already have movie after movie being made. But thanks to the advancement in digital technology, bringing Iron Man to life no longer consisted of the prospect and worrying thought of having a man running about in a rubber suit passing it off as metal, the way Ultraman would have been done, complete with mechanical clicks and whirrs as sound effects to try and fool the visual sensory. Here, we have a very detailed rendering of the entire design from scratch to final modification, and we're in at every step of the way, with many cheeky and sometimes a tad implausible scenes just for cheap laughs thrown in.I thought Iron Man the story worked because of stark (pardon the pun) similarities with Batman Begins, also an origin story which took its time to dwell on the man behind the suit, nevermind at the sacrifice of having less action sequences, or by not giving the fans what they want through the showcase of more than the basic powers. Advanced capabilities can always find room in the sequel, and as the first movie used to establish its characters, I felt that it succeeded, given too that it had a cast of capables (just like Batman Begins had) to pull the movie through without resorting to over the top and campy performances, starting of course with the lead in Robert Downey Jr.In a nutshell, Downey is Tony Stark through and through. His affinity for the character shines, and no doubt it bore some parallels between his own personal, and Stark's life in the narrative future when he hits the bottle. He was allowed to become a Two-Face of sorts, on one hand being and later acting out his flamboyance self whose mission in life was the continuation of his father's legacy of Stark Industries, a weapons conglomerate, versus his personal mission in ridding his own weapons from the hands of the bad guys, now updated to be freedom fighters in the Middle East. The dialogue contained within each scene of Stark's, except perhaps during captivity, is full of one-liners done in double quick time, you probably would think it boiled down to a whole host of natural ad-libbing.But while Starks spends significant amount of time in his unsecured basement building his masterpiece, his human interaction come in the form of faithful secretary Pepper Potts (Gwyneth Paltrow) who actually, for the first time I admit, looked really good on screen as Stark's most trusted aide, bringing about some serious spark of sexual tension and chemistry between the two characters of opposite sex, more so than any other comic book movie I have seen. And good friend from the air force Jim Rhodes (Terrence Howard) complete the circle of trust who knows of Stark's secret identity, and you'd be keeping your fingers crossed at the toss of a teaser of a certain War Machine appearance should the sequel be out. Who's the main villain in the movie? It points the finger at Corporations, or at least here, the weapons manufacturers and the shady deals that go through in the name of profit, the sole objective for any corporation's existence. And Jeff Bridges, in a rare villainous role, got to personify that greed and wrestle for absolute power just like the trailer already suggested. While his performance is refreshing as he disappears behind the ball head and bushy beard, you could see his motivation and how the plot would have been developed to introduced the ultimate fodder for Iron Man to duke it out in a, sad to say, ordinary finale which any audience would probably be able to stay a step ahead.As mentioned earlier, there are plenty of similarities with the Dark Knight of Gotham in Christopher Nolan's reboot, but more so because of properties inherent with the likeness between Bruce Wayne and Tony Stark. Both are incredibly wealthy to devote time outside of the day job to pursue their \"hobby\", both have to suffer personal tragedies in order to wake up to the cruel world, and in the movie, both fall prey to the corporate raider type, spend time perfecting their suit of war, have assistants they would trust their lives with, and of course save them from impending doom, and a finales set at their facilities.But Iron Man is still a special effects extravaganza offering a thrill ride especially when he goes into battle mode, and without a doubt, Robert Downey Jr probably should be credited for raising the profile of this once Tier-B character, to perhaps becoming more recognizable now, and obviously, expanding the fan base of this weaponry filled suit of metal, which of course, in this origin movie, we were only given a glimpse of its potential. can everyone now spell sequel and clamour for more please? Iron Man has set the bar for the other upcoming comic book movies to try and surpass this summer season!", "author":"DICK STEEL" }, "datePublished":[ "1 hours ago", "2 hours ago", "10 hours ago", "2008-05-02" ], "provider":[ "MovieWeb", "ComicBookMovie.com", "Flickeringmyth" ], "actor":[ { "itemType":"http://schema.org/Person", "url":"/name/nm0000375/?ref=tt_clt1", "name":"Robert Downey Jr." }, { "itemType":"http://schema.org/Person", "url":"/name/nm0005024/?ref=tt_clt2", "name":"Terrence Howard" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0000313/?ref=tt_clt3", "name":"Jeff Bridges" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0000569/?ref=tt_clt4", "name":"Gwyneth Paltrow" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0004753/?ref=tt_clt5", "name":"Leslie Bibb" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0869467/?ref=tt_clt6", "name":"Shaun Toub" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0846687/?ref=tt_clt7", "name":"Faran Tahir" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0163988/?ref=tt_clt8", "name":"Clark Gregg" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0810488/?ref=tt_clt9", "name":"Bill Smitrovich" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0046223/?ref=tt_clt10", "name":"Sayed Badreya" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0079273/?ref=tt_clt11", "name":"Paul Bettany" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0269463/?ref=tt_clt12", "name":"Jon Favreau" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0082526/?ref=tt_clt13", "name":"Peter Billingsley" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0347375/?ref=tt_clt14", "name":"Tim Guinee" }, { "itemType":"http://schema.org/Person", "url":"/name/nm0528164/?ref=tt_clt15", "name":"Will Lyman" } ], "thumbnailUrl":[ "/media/rm303336448/tt0371746?ref=tt_pv_md1", "/media/rm2392634368/tt0371746?ref=tt_pv_md2", "/media/rm2409411584/tt0371746?ref=tt_pv_md_3" ], "image":[ "http://ia.media-imdb.com/images/G/01/imdb/images/nopicture/small/unknown-1394846836._CB379391227_.png", "http://ia.media-imdb.com/images/G/01/imdb/images/nopicture/small/unknown-1394846836._CB379391227_.png", "http://ia.media-imdb.com/images/G/01/imdb/images/nopicture/small/unknown-1394846836._CB379391227_.png", "http://ia.media-imdb.com/images/M/MV5BMTczNTI2ODUwOF5BMl5BanBnXkFtZTcwMTU0NTIzMw@@._V1_SX214_AL_.jpg", "http://ia.media-imdb.com/images/M/MV5BMTQxOTA2NDUzOV5BMl5BanBnXkFtZTgwNzY2MTMxMzE@._V1_SX86_CR0,0,86,86_AL_.jpg", "http://ia.media-imdb.com/images/M/MV5BMTU0NTgzNTA1OF5BMl5BanBnXkFtZTgwNDMzMzM4NDE@._V1_SX86_CR0,0,86,86_AL_.jpg", "http://ia.media-imdb.com/images/M/MV5BMjMzODA4NDYzM15BMl5BanBnXkFtZTgwMTc0Mzc0NDE@._V1_SX86_CR0,0,86,86_AL_.jpg" ], "director":{ "itemType":"http://schema.org/Person", "url":"/name/nm0269463/?ref_=tt_ovdr", "name":"Jon Favreau" }, "actors":{ "itemType":"http://schema.org/Person", "url":[ "/name/nm0000375/?ref=tt_ovst", "/name/nm0000569/?ref=tt_ovst", "/name/nm0005024/?ref=tt_ovst", "fullcredits?ref=tt_ov_stsm" ], "name":[ "Robert Downey Jr.", "Gwyneth Paltrow", "Terrence Howard" ] }, "trailer":"/video/imdb/vi447873305/?ref=tt_ov_vi", "name":"Iron Man", "aggregateRating":{ "itemType":"http://schema.org/AggregateRating", "ratingValue":"7.9", "bestRating":"10", "ratingCount":"571,104", "reviewCount":[ "1,010 user", "472 critic" ] } } ]