Closed randykoala closed 9 years ago
Hey randykoala,
Thanks - glad you're finding the scraper useful.
I don't see anything immediately wrong with the data 18 web content on a file I tested on, but the consistency of the site is a mess and they use different formats all over the place, so it's very possible there's something I didn't account for. Can you paste in a few URLs of movies you are trying to scrape from web content that aren't working?
Thanks.
There are a couple here http://www.data18.com/content/1152153 http://www.data18.com/content/1151766 http://www.data18.com/content/1151609
Hope this helps
I'm getting images just fine from these URLs. Try deleting AmalgamationSettings.json and settings.xml from the directory the program is installed in. Do you still notice the issue? If so, can you provide the logs?
You can see the log by going to the "View" menu and selecting "Show Output Panel in New WIndow"
Thanks.
Still not getting the images - this is the log
ead in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Read in amalgamation preferences from AmalgamationSettings.json
Gui Initialized
Read in amalgamation preferences from AmalgamationSettings.json
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at javax.swing.ImageIcon.
just for info, when i scrape no images other than actors come up in main screen or for selection. Empty image files are saved into the directory
Can you download the newest version I just posted from http://www.mediafire.com/download/pm3d2yl49qa99fe/JAVMovieScraper.jar and see if it fixes the problem? If not, can you post the log again?
Thanks.
sorry still not working - see log
Read in amalgamation preferences from AmalgamationSettings.json
currentItemSource = Data18 Web Content with disabled = false
currentItemSource = Default Data Item Source with disabled = false
fileBaseName = Big Wet Butts 14 11 09 Jada Stevens Best Butt In The Biz
searchString = http://www.data18.com/search/?k=Big+Wet+Butts+14+11+09+Jada+Stevens+Best+Butt+In+The+Biz&t=0
Searching google with date replaced file name: Big Wet Butts 2014 November 09 Jada Stevens Best Butt In The Biz
Movie scraped = null
Scraping complete of siteScraper = Data18 Web Content
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at javax.swing.ImageIcon.
Are you scraping a folder or a file? If a folder, is there anything in the folder already besides the movie? If it's a file, are there are any other files in the same directory?
Hi DoctorD - scraping a folder and its empty apart from the video. Thanks for trying to fix. I tried it on another pc to see if its something with my settings and still not working
Latest log Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Read in amalgamation preferences from AmalgamationSettings.json Gui Initialized Read in amalgamation preferences from AmalgamationSettings.json currentItemSource = Data18 Web Content with disabled = false fileBaseName = Best Butt In The Biz currentItemSource = Default Data Item Source with disabled = false searchString = http://www.data18.com/search/?k=Best+Butt+In+The+Biz&t=0 Scraping this webpage for movie: http://www.data18.com/content/1145765 Movie scraped = Movie [title=Title [title="Best Butt in the Biz" source="Data18 Web Content"], originalTitle=OriginalTitle [originalTitle="" source="Data18 Web Content"], sortTitle=SortTitle [sortTitle="" source="Data18 Web Content"], set=Set [set="Big Wet Butts" source="Data18 Web Content"], rating=Rating [maxRating="0.0", rating="" source="Data18 Web Content"], year=Year [year="2014" source="Data18 Web Content"], top250=Top250 [top250="" source="Data18 Web Content"], trailer = Trailer [trailer="" source="Data18 Web Content"], votes=Votes [votes="" source="Data18 Web Content"], outline=Outline [outline="" source="Data18 Web Content"], plot=Plot [plot="Dive balls deep in the best butt in the business with today's BWB scene featuring the one and only Jada Stevens. Today, this bodacious babe let Mick Blue give her thick ass a workout outside. After peeling off her bikini to tease you, Jada twerked and shook her ass on Mick's face, pressing her cheeks on his mouth so he could lick her pussy and crack. After rooster-tailing oil from her pristine asshole, Jada spread her holes open for Mick to fill her up with his dick. Mick pounded Jada's pussy, then he stretched her tight ass open with dirty and intense anal sex, in reverse cowgirl and downward dog, until busting a fat load of jizz across Jada's mouth." source="Data18 Web Content"], tagline=Tagline [tagline="" source="Data18 Web Content"], studio=Studio [studio="Brazzers" source="Data18 Web Content"]releaseDate=ReleaseDate [releasedate="2014-11-09" source="Data18 Web Content"], runtime=Runtime [runtime="" source="Data18 Web Content"], posters=[Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], fanart=[Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], extrafanart = [Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], mpaa=MPAARating [MPAARating="XXX" source="Data18 Web Content"], id=ID [id="1145765" source="Data18 Web Content"], genres=[Genre [genre="Heterosexual" source="Data18 Web Content"], Genre [genre="Hardcore" source="Data18 Web Content"], Genre [genre="Anal" source="Data18 Web Content"], Genre [genre="Boy Girl" source="Data18 Web Content"], Genre [genre="Caucasian" source="Data18 Web Content"], Genre [genre="Caucasian Men" source="Data18 Web Content"], Genre [genre="Big Butts" source="Data18 Web Content"], Genre [genre="Bikini" source="Data18 Web Content"], Genre [genre="Brunettes" source="Data18 Web Content"]], actors=[Actor [role="null", Person [name="Jada Stevens", thumb=Thumb [thumbURL=http://img.data18.com/images/stars/120/662.jpg" source="Default Data Item Source"]] source="Data18 Web Content"], Actor [role="null", Person [name="Mick Blue", thumb=Thumb [thumbURL=http://img.data18.com/images/stars/120/6316.jpg" source="Default Data Item Source"]] source="Data18 Web Content"]], directors=[]] Scraping complete of siteScraper = Data18 Web Content Read in amalgamation preferences from AmalgamationSettings.json Skipping amalgamation process as there is only one movie Setting new movie: Movie [title=Title [title="Best Butt in the Biz" source="Data18 Web Content"], originalTitle=OriginalTitle [originalTitle="" source="Data18 Web Content"], sortTitle=SortTitle [sortTitle="" source="Data18 Web Content"], set=Set [set="Big Wet Butts" source="Data18 Web Content"], rating=Rating [maxRating="0.0", rating="" source="Data18 Web Content"], year=Year [year="2014" source="Data18 Web Content"], top250=Top250 [top250="" source="Data18 Web Content"], trailer = Trailer [trailer="" source="Data18 Web Content"], votes=Votes [votes="" source="Data18 Web Content"], outline=Outline [outline="" source="Data18 Web Content"], plot=Plot [plot="Dive balls deep in the best butt in the business with today's BWB scene featuring the one and only Jada Stevens. Today, this bodacious babe let Mick Blue give her thick ass a workout outside. After peeling off her bikini to tease you, Jada twerked and shook her ass on Mick's face, pressing her cheeks on his mouth so he could lick her pussy and crack. After rooster-tailing oil from her pristine asshole, Jada spread her holes open for Mick to fill her up with his dick. Mick pounded Jada's pussy, then he stretched her tight ass open with dirty and intense anal sex, in reverse cowgirl and downward dog, until busting a fat load of jizz across Jada's mouth." source="Data18 Web Content"], tagline=Tagline [tagline="" source="Data18 Web Content"], studio=Studio [studio="Brazzers" source="Data18 Web Content"]releaseDate=ReleaseDate [releasedate="2014-11-09" source="Data18 Web Content"], runtime=Runtime [runtime="" source="Data18 Web Content"], posters=[Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], fanart=[Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], extrafanart = [Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], mpaa=MPAARating [MPAARating="XXX" source="Data18 Web Content"], id=ID [id="1145765" source="Data18 Web Content"], genres=[Genre [genre="Heterosexual" source="Data18 Web Content"], Genre [genre="Hardcore" source="Data18 Web Content"], Genre [genre="Anal" source="Data18 Web Content"], Genre [genre="Boy Girl" source="Data18 Web Content"], Genre [genre="Caucasian" source="Data18 Web Content"], Genre [genre="Caucasian Men" source="Data18 Web Content"], Genre [genre="Big Butts" source="Data18 Web Content"], Genre [genre="Bikini" source="Data18 Web Content"], Genre [genre="Brunettes" source="Data18 Web Content"]], actors=[Actor [role="null", Person [name="Jada Stevens", thumb=Thumb [thumbURL=http://img.data18.com/images/stars/120/662.jpg" source="Default Data Item Source"]] source="Data18 Web Content"], Actor [role="null", Person [name="Mick Blue", thumb=Thumb [thumbURL=http://img.data18.com/images/stars/120/6316.jpg" source="Default Data Item Source"]] source="Data18 Web Content"]], directors=[]]
OK, it looks like we're making progress. You're not getting random java errors anymore, but I notice that the actual posters and fanarts it's giving you are the wrong URL! Let me look into this some more to see what is going on.
Thanks, DoctorD
The weird thing is that when I scrape that same movie, I get this as the result. Notice the fanart and poster section is different from what you are getting.
Setting new movie: Movie [title=Title [title="Best Butt in the Biz" source="Data18 Web Content"], originalTitle=OriginalTitle [originalTitle="" source="Data18 Web Content"], sortTitle=SortTitle [sortTitle="" source="Data18 Web Content"], set=Set [set="Big Wet Butts" source="Data18 Web Content"], rating=Rating [maxRating="0.0", rating="" source="Data18 Web Content"], year=Year [year="2014" source="Data18 Web Content"], top250=Top250 [top250="" source="Data18 Web Content"], trailer = Trailer [trailer="" source="Data18 Web Content"], votes=Votes [votes="" source="Data18 Web Content"], outline=Outline [outline="" source="Data18 Web Content"], plot=Plot [plot="Dive balls deep in the best butt in the business with today's BWB scene featuring the one and only Jada Stevens. Today, this bodacious babe let Mick Blue give her thick ass a workout outside. After peeling off her bikini to tease you, Jada twerked and shook her ass on Mick's face, pressing her cheeks on his mouth so he could lick her pussy and crack. After rooster-tailing oil from her pristine asshole, Jada spread her holes open for Mick to fill her up with his dick. Mick pounded Jada's pussy, then he stretched her tight ass open with dirty and intense anal sex, in reverse cowgirl and downward dog, until busting a fat load of jizz across Jada's mouth." source="Data18 Web Content"], tagline=Tagline [tagline="" source="Data18 Web Content"], studio=Studio [studio="Brazzers" source="Data18 Web Content"]releaseDate=ReleaseDate [releasedate="2014-11-09" source="Data18 Web Content"], runtime=Runtime [runtime="" source="Data18 Web Content"], posters=[Thumb [thumbURL=http://74.50.117.45/1/1136/145765/01.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/02.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/03.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/04.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/05.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/06.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/07.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/08.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/09.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/10.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/11.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/12.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/13.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/14.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/15.jpg" source="Data18 Web Content"]], fanart=[Thumb [thumbURL=http://74.50.117.45/1/1136/145765/01.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/02.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/03.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/04.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/05.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/06.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/07.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/08.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/09.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/10.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/11.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/12.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/13.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/14.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/15.jpg" source="Data18 Web Content"]], extrafanart = [Thumb [thumbURL=http://74.50.117.45/1/1136/145765/01.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/02.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/03.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/04.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/05.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/06.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/07.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/08.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/09.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/10.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/11.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/12.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/13.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/14.jpg" source="Data18 Web Content"], Thumb [thumbURL=http://74.50.117.45/1/1136/145765/15.jpg" source="Data18 Web Content"]], mpaa=MPAARating [MPAARating="XXX" source="Data18 Web Content"], id=ID [id="1145765" source="Data18 Web Content"], genres=[Genre [genre="Heterosexual" source="Data18 Web Content"], Genre [genre="Hardcore" source="Data18 Web Content"], Genre [genre="Anal" source="Data18 Web Content"], Genre [genre="Boy Girl" source="Data18 Web Content"], Genre [genre="Caucasian" source="Data18 Web Content"], Genre [genre="Caucasian Men" source="Data18 Web Content"], Genre [genre="Big Butts" source="Data18 Web Content"], Genre [genre="Bikini" source="Data18 Web Content"], Genre [genre="Brunettes" source="Data18 Web Content"]], actors=[Actor [role="null", Person [name="Jada Stevens", thumb=Thumb [thumbURL=http://img.data18.com/images/stars/120/662.jpg" source="Default Data Item Source"]] source="Data18 Web Content"], Actor [role="null", Person [name="Mick Blue", thumb=Thumb [thumbURL=http://img.data18.com/images/stars/120/6316.jpg" source="Default Data Item Source"]] source="Data18 Web Content"]], directors=[]]
Thanks DoctorD
I am also getting these exception errors when i first open the application even with the settigs file deleted
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at javax.swing.ImageIcon.
Same problem for me in 2.0.5-alpha. I used the same web content that u used Doc. http://www.data18.com/content/1145765
This is my log file: Setting new movie: Movie [title=Title [title="Best Butt in the Biz" source="Data18 Web Content"], originalTitle=OriginalTitle [originalTitle="" source="Data18 Web Content"], sortTitle=SortTitle [sortTitle="" source="Data18 Web Content"], set=Set [set="Big Wet Butts" source="Data18 Web Content"], rating=Rating [maxRating="0.0", rating="" source="Data18 Web Content"], year=Year [year="2014" source="Data18 Web Content"], top250=Top250 [top250="" source="Data18 Web Content"], trailer = Trailer [trailer="" source="Data18 Web Content"], votes=Votes [votes="" source="Data18 Web Content"], outline=Outline [outline="" source="Data18 Web Content"], plot=Plot [plot="Dive balls deep in the best butt in the business with today's BWB scene featuring the one and only Jada Stevens. Today, this bodacious babe let Mick Blue give her thick ass a workout outside. After peeling off her bikini to tease you, Jada twerked and shook her ass on Mick's face, pressing her cheeks on his mouth so he could lick her pussy and crack. After rooster-tailing oil from her pristine asshole, Jada spread her holes open for Mick to fill her up with his dick. Mick pounded Jada's pussy, then he stretched her tight ass open with dirty and intense anal sex, in reverse cowgirl and downward dog, until busting a fat load of jizz across Jada's mouth." source="Data18 Web Content"], tagline=Tagline [tagline="" source="Data18 Web Content"], studio=Studio [studio="Brazzers" source="Data18 Web Content"]releaseDate=ReleaseDate [releasedate="2014-11-09" source="Data18 Web Content"], runtime=Runtime [runtime="" source="Data18 Web Content"], posters=[Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], fanart=[Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], extrafanart = [Thumb [thumbURL=http://94.229.67.74/1/1136/145765/big.jpg" source="Data18 Web Content"]], mpaa=MPAARating [MPAARating="XXX" source="Data18 Web Content"], id=ID [id="1145765" source="Data18 Web Content"], genres=[Genre [genre="Heterosexual" source="Data18 Web Content"], Genre [genre="Hardcore" source="Data18 Web Content"], Genre [genre="Anal" source="Data18 Web Content"], Genre [genre="Boy Girl" source="Data18 Web Content"], Genre [genre="Caucasian" source="Data18 Web Content"], Genre [genre="Caucasian Men" source="Data18 Web Content"], Genre [genre="Big Butts" source="Data18 Web Content"], Genre [genre="Bikini" source="Data18 Web Content"], Genre [genre="Brunettes" source="Data18 Web Content"]], actors=[Actor [role="null", Person [name="Jada Stevens", thumb=Thumb [thumbURL=http://img.data18.com/images/stars/120/662.jpg" source="Default Data Item Source"]] source="Data18 Web Content"], Actor [role="null", Person [name="Mick Blue", thumb=Thumb [thumbURL=http://img.data18.com/images/stars/120/6316.jpg" source="Default Data Item Source"]] source="Data18 Web Content"]], directors=[]]
The scrape go to http://94.229.67.74/1/1136/145765/big.jpg but this is not an image but a redirect to gallery. Real image are http://94.229.67.74/1/1136/145765/01.jpg like your http://74.50.117.45/1/1136/145765/02.jpg.
I think it's a country problem.. U live in USA Doctor right? From EU (i tried ~10 different connection from 10 different country in EU and always go to 94.229.67.74 when i "view image link" from http://www.data18.com/viewer/1145765/01 (gallery). With a connection from USA the ip is 74.50.117.45
I wish this can be helpful
Ah, that makes a lot of sense if it is giving different results for each country. Yes, I'm in the USA.
I'll see what I can do to get around the country thing, though it could be difficult.
Hi DoctorD, any news about scraping from EU?
@m0uthless i use "Cyber Ghost" service to scan the files
Any chance I can get you to paste the HTML of the entire page for the movie mentioned in this thread that isn't scraping in Europe? Also, the HTML of one of the gallery pages when you click on one of the thumbnails for the gallery. It might be a good idea to use a site like pastebin (or similar) and link to that site in the comment here.
Thanks.
thx @Signumda for the trick. @DoctorD1501 this is html of the page http://pastebin.com/9vMrNz1s and this html of the gallery http://pastebin.com/pNzkwiTe
One final question - Does the 74.50.117.45 IP address work for you in Europe if you try it directly? I may do some replacements with the European IP with the American IP, but if it doesn't load then that won't work I guess.
In other words, does this link load & show an image in Europe?
Yes it works
I just posted a new build on http://www.mediafire.com/download/pm3d2yl49qa99fe/JAVMovieScraper.jar with a potential fix for this issue. Please test it out and let me know if you still notice any issues with the images on this or any other pages. I tried it out using Cyber Ghost on this particular page, so i think we should be OK, but I used manual IP address substitution, so who knows what kind of weirdness we may see :).
Thanks for all the detailed error logs and help with this one everyone!
sorry, it didn't work for Me in germany
Can you post the log? Thanks.
@DoctorD1501 this last build works for me! Thx!
My mistake, the WEB version is working fine, but can you fix the MOVIE version also?
Signumda,
I just uploaded a new version on mediafire (http://www.mediafire.com/download/pm3d2yl49qa99fe/JAVMovieScraper.jar) which I think should fix the movie version. Can you try it out and let me know if it works?
Thanks, DoctorD
no, the extrafanarts are 0Kb and the fanart is the backside
Signumda,
I uploaded a new build on mediafire which fixed an extrafanart problem on one of the pages I could find some on. The problem was they were using yet another IP address for that in Europe which needed to use an American IP address. In general, this approach is going to be problematic since I have no idea how many IP addresses they are using there, so if you notice specific movies not getting art that you believe should have it (you see the art on the page of the movie), let me know and I can take a look at that page.
In addition, your comment about the fanart being the backside gave me another chance to look at how this was handled. It turned out this was actually the same as the art American users were getting, so I made a change to allow the extrafanart to show up as fanart as well. The extrafanart tends to be official promotional gallery images and is likely much better suited to be the movie's fanart anyways.
Please try it out and let me know if you notice any other issues.
Thanks, DoctorD
great work thank you
HI DoctorD Love the new Excalibur scraper - fantastic work. Has there been a change to the Data 18 Website. Its not pulling images for web content other than actors. The data 18 Movie scraper is working fine