pavit939 / Web_Scrapping

A project built for Code Challenge 2020. The project has its main focus towards collecting data from websites and storing the output in csv and text file to induce readability.
MIT License
0 stars 0 forks source link

reduce amount of repeated code #2

Open PandaWhoCodes opened 3 years ago

PandaWhoCodes commented 3 years ago

Use loops or functions wherever possible

df1 = DataFrame({"Meta_Name":m_name})
df2 = DataFrame({"Meta_Content":m_content})
df3 = DataFrame({"Unigram":uni})
df4 = DataFrame({"Frequency Unigram":word_fd.most_common(20)})
df5 = DataFrame({"Bigram":bi_l})
df6 = DataFrame({"Frequency Bigram":bigram_fd.most_common(20)})
df8 = DataFrame({"Image_url":x})
final = pd.concat([df1,df2,df3,df4,df5,df6,df8],ignore_index = "False",axis=1)
def append_df(df,name,data):
    return pd.concat([df,DataFrame({name:datal})],ignore_index = "False",axis=1)

data_list = [("Meta_Name":m_name)...]
df = Dataframe()
for item in data_list:
    df = append_df(df,item[0],item[1])
PandaWhoCodes commented 3 years ago

the same for any same line that performs the same function but with different data

filee.write(f'Total External links: {len(external)}\n')
filee.write(f'Total Internal links: {len(internal)}\n')
filee.write(f'Total links: {len(external) + len(internal)}\n')
pavit939 commented 3 years ago

Great! Thanks a lot for your input!

On Tue, Nov 3, 2020, 4:21 PM Thomas Ashish Cherian notifications@github.com wrote:

the same for any same line that performs the same function but with different data

filee.write(f'Total External links: {len(external)}\n') filee.write(f'Total Internal links: {len(internal)}\n') filee.write(f'Total links: {len(external) + len(internal)}\n')

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pavit939/Web_Scrapping/issues/2#issuecomment-721042470, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJRU7KB2LJXN7F2VIPSEIJTSN7OCJANCNFSM4TISXNEA .