andrelmfarias
commented
5 years ago The scrapped articles have a very complicated and mixed list of paragraphs, some phrases do not have anything to do with the articles such as "Share this page !", "Tools" and even some text that work as links to other pages.
For exemple, the text in the csv for second article is:
"['Category', 'Share', 'Tools', 'BNP Paribas has received the notification by the European Central Bank of the outcome of the 2018 Supervisory Review and Evaluation Process (SREP) confirming the Common Equity Tier 1 (CET1) requirement that the Group has to respect on a consolidated basis.', 'The Common Equity Tier 1 (CET1) requirement that the Group has to respect on a consolidated basis is 9.83% since 1st January 2019 (excluding the Pillar 2 guidance, not public), which includes 1.50% for the G-SIB buffer, 2.50% for the Conservation buffer, 1.25% for the Pillar 2 requirement (unchanged vs. last year) and 0.08% of countercyclical buffer. The requirement for the Tier 1 Capital is 11.33% and that for the Total Capital is 13.33%.', 'With a CET1 ratio at 11.8%, a Tier 1 ratio at 13.1% and a Total Capital ratio at 15.0% as at 31 December 2018, the Group is well above the regulatory requirements.', '', 'BNP Paribas is a leading bank in Europe with an international reach. It has a presence in 73 countries, with more than 196,000 employees, including around 149,000 in Europe. The Group has key positions in its three main activities: Domestic Markets and International Financial Services (whose retail-banking networks and financial services are covered by Retail Banking & Services) and Corporate & Institutional Banking, which serves two client franchises: corporate clients and institutional investors. The Group helps all its clients (individuals, community associations, entrepreneurs, SMEs, corporates and institutional clients) to realise their projects through solutions spanning financing, investment, savings and protection insurance. ', 'In Europe, the Group has four domestic markets (Belgium, France, Italy and Luxembourg) and BNP Paribas Personal Finance is the European leader in consumer lending.', 'BNP Paribas is rolling out its integrated retail-banking model in Mediterranean countries, in Turkey, in Eastern Europe and a large network in the western part of the United States. In its Corporate & Institutional Banking and International Financial Services activities, BNP Paribas also enjoys top positions in Europe, a strong presence in the Americas as well as a solid and fast-growing business in Asia-Pacific.', 'Share this page !', 'Tools']"I understand that we should filter that, however do you expect us to do the analysis including these "trashy" pieces of text?
fmikaelian
Generate some KPIs on the dataset we collected. Ideas of KPIs:
Before and after filtering.