DKarap / web-driver

crawler that use the webdriver, ghostdriver/phantomJS
0 stars 1 forks source link

Icims job portal issue with the inline frame.. #32

Open DKarap opened 10 years ago

DKarap commented 10 years ago

The main problem with the icims is that the real content exist in a iframe which when we switch via webdriver we see the content but when we go strsight via url we dont; adding the "in_iframe=1" into url solves the problem but looks like there is a redirection...

https://careers-newmarket.icims.com/jobs/search?branding=live&ss=1&searchKeyword=&searchCategory=&searchCompany=&searchPositionType=&searchLocation=&latitude=&longitude=

https://college-marathonpetroleum.icims.com/jobs/search?ss=1&searchCategory=

https://externalhub-agfa.icims.com/jobs/search?ss=1&searchLocation=&searchCategory=&hashed=0&in_iframe=1

https://jobs3-cenovus.icims.com/jobs/search?ss=1&searchKeyword=&searchLocation=&searchCategory=&searchPositionType=&mobile=false&width=760&height=500&bga=true&needsRedirect=false

https://careers-apexcos.icims.com/jobs/search?ss=1&searchLocation=&searchCategory=&hashed=0

https://careers-wolverineworldwide.icims.com/jobs/intro?mobile=false&width=520&height=500&bga=false&needsRedirect=false

https://careers-dnvgl.icims.com/jobs/search?ss=1&searchKeyword=&searchCategory=&searchPositionType=&searchLocation=&latitude=&longitude=

https://careers-fei.icims.com/jobs/search?ss=1 https://jobs-aegion.icims.com/jobs/search?pr=0&searchRadius=25&branding=live

DKarap commented 10 years ago

Currently this is not possible to be solved because when we go to correct url the browser(firefox too) redirects to page without the parameter &in_frame=1 so we cant retrieve the frame's content. An ugly hack is to use the below function everytime we use the driver.get() function but still during update we retrieve only one semantic per time..

private  void specialCaseOfIcims(String url){
    System.out.println("\ncheck url:"+url);
    if(url.contains("&in_iframe=1")){
        System.out.println("\t contains &in_iframe=1:");
        List<Frame> frames = getFrames(Arrays.asList("iframe"));
        if(frames.size()==1){
            System.out.println("\t contains iframe:");
            switchToFrame(FindFrameBy.index, 0);
        }
    }       
}