Xonshiz / comic-dl

Comic-dl is a command line tool to download manga and comics from various comic and manga sites. Supported sites : readcomiconline.to, mangafox.me, comic naver and many more.
https://github.com/Xonshiz/comic-dl
MIT License
545 stars 68 forks source link

I am not able export or write my output to csv using python #276

Closed JayashreeKannabiran closed 3 years ago

JayashreeKannabiran commented 3 years ago

I am trying to export or write my output in a CSV file but I am not able to write or export and instead of the output, my error is

Traceback (most recent call last): File "C:/Users/ICONS-TOS-DEV-08012/PycharmProjects/WebScraping/SecNew.py", line 72, in material.loop_sections() File "C:/Users/ICONS-TOS-DEV-08012/PycharmProjects/WebScraping/SecNew.py", line 40, in loop_sections raise e File "C:/Users/ICONS-TOS-DEV-08012/PycharmProjects/WebScraping/SecNew.py", line 34, in loop_sections s += self.product_header(section) TypeError: can only concatenate str (not "list") to str

For the below program output, I have to import in the CSV file.

from bs4 import BeautifulSoup from selenium import webdriver import time, re, csv

class ProductTracker:

def __init__(self, url):
    self.url = url
    self.user_agent = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36'}
    # self.responce = requests.get(url=self.url, headers= self.user_agent).text
    driver = webdriver.Chrome(executable_path="D:\\cromedriver\\chromedriver.exe")
    driver.get(url)
    time.sleep(1)
    self.response = driver.page_source
    driver.close()
    self.soup = BeautifulSoup(self.response, 'lxml')

def product_title(self):
        title = self.soup.find('h1', {'class': 'view-header__primary-heading'})
        if title is not None:
         #return title.text
            return title.text.replace('\n', '')
        return "Tag Not Found"

def loop_sections(self):
    s = ''
    try:
        for section in self.soup.select('.search-table-view__web-parent'):
            print(self.product_header(section))
            print()
            print(self.product_tableheader(section))
            self.product_tablevalues(section)  # you might change this from a list of lists to print during as loop rather than nested list comprehension
            print()
            s += self.product_header(section)
            s += "\n"
            s += self.product_tableheader(section)
            s += self.product_tablevalues(section)  # you might change this from a list of lists to print during as loop rather than nested list comprehension
            s += "\n"
    except Exception as e:
        raise e
    else:
        return s

def product_header(self, section):
    try:
        table_title = [section.select_one('.search-table-view__heading').text]
        return table_title
    except Exception as e:
        print("error", e)
        return

def product_tableheader(self, section):
    try:
        tableheaders = [i.text.strip() for i in section.select_one(' .search-table-view__web-parent-table-head-row').select('th')][:-1]
        return tableheaders
    except Exception as e:
        print("error", e)
        return

def product_tablevalues(self, section):
    try:
        for row in section.select('.search-table-view__web-parent-table-row'):
            print([re.sub('\n+|\s{2,}', ' ', r.text.strip()) for r in row.select('td')][:-1])
        return
    except Exception as e:
        print("error", e)
        return

material = ProductTracker(url="https://www.grainger.com/category/power-transmission/bearings/ball-bearings/ball-screw-support-bearings") print(material.product_title()) print() material.loop_sections()

with open(material.product_title()+'.csv','w') as f: writer = csv.writer(f, delimiter =',') writer.writerows(material.loop_sections())

Xonshiz commented 3 years ago

What? What is this?

Are you looking for a solution to a problem you're having with your code? Stackoverflow would be a better place for this.

Closing this.