search

changhoon-kim / fresh-demo

0 stars 0 forks source link

크롤링으로 데이터 가져올 수 있는지 확인 #1

Open changhoon-kim opened 3 years ago

changhoon-kim commented 3 years ago
changhoon-kim commented 3 years ago

컬리

import requests

if __name__ == '__main__':
    url = 'https://www.kurly.com/shop/goods/goods_search.php?searched=Y&log=1&skey=all&hid_pr_text=&hid_link_url=&edit=Y&sword=%B1%BA%B8%B8%B5%CE&x=19&y=15'

    r = requests.get(url)

    print(r.text)
<body class="goods-goods_search" oncontextmenu="return false" ondragstart="return false" onselectstart="return !disableSelection">

<script src="//developers.kakao.com/sdk/js/kakao.min.js"></script>
<div id="wrap" class="">
<div id="pos_scroll"></div>
<div id="container">
<div id="header"><script>
  // ie10 이하 페이지 이동
  var ieCheckAgent = navigator.userAgent.toLowerCase();
  if ( (navigator.appName === 'Netscape' && navigator.userAgent.search('Trident') !== -1) || (ieCheckAgent.indexOf("msie") !== -1) ) {
    if(navigator.appName !== 'Netscape'){
      location.href = "/shop/event/browserUpdate.php";
    }
  }
</script>
changhoon-kim commented 2 years ago

셀레늄 테스트

requirement

brew install selenium
$ cat capture_test.py
from selenium import webdriver

url = 'http://www.naver.com/'

browser = webdriver.Chrome()
browser.implicitly_wait(3)
browser.get(url)
browser.save_screenshot('Website.png')
browser.quit()
from selenium import webdriver

if __name__ == '__main__':
    url = 'https://www.kurly.com/shop/goods/goods_search.php?searched=Y&log=1&skey=all&hid_pr_text=&hid_link_url=&edit=Y&sword=%B1%BA%B8%B8%B5%CE&x=19&y=15'

    browser = webdriver.Chrome()
    browser.implicitly_wait(3)

    browser.get(url)
    html = browser.page_source

    print(html)