search

cvpaperchallenge-alumni-community / Babel

Babel is an advanced tool that scrapes conference paper information from various academic conferences to analyze trends in academic fields based on keyword analysis.
MIT License
1 stars 0 forks source link

Extract Keywords of CVPR 2022 #3

Closed gatheluck closed 6 months ago

gatheluck commented 6 months ago

Why

CVPR 2022の論文情報の一覧を取得する

Definition of Done

How

以下のコマンドを実行する

% poetry run python src/scripts/scrape_conference_page.py -c cvpr -y 2022

CVPR 2022の論文採択数をWebページで確認して、JSONファイル内の数と一致しているか確認する

Hina39 commented 6 months ago 
INFO:src.cvf:Processing 98/2074: https://openaccess.thecvf.com/content/CVPR2022/html/Kim_E2V-SDE_From_Asynchronous_Events_to_Fast_and_Continuous_Video_Reconstruction_CVPR_2022_paper.html
Traceback (most recent call last):
  File "/home/challenger/babel/src/scripts/scrape_conference_page.py", line 78, in <module>
    scrape_conference_page(
  File "/home/challenger/babel/src/scripts/scrape_conference_page.py", line 33, in scrape_conference_page
    papers = cvf.get_papers(conference=conference, year=year)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/challenger/babel/src/cvf.py", line 34, in get_papers
    paper = parse_paper_page(url)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/challenger/babel/src/cvf.py", line 109, in parse_paper_page
    title: Final[str] = bs.select_one("#papertitle").text.strip()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'text'
Hina39 commented 6 months ago

2067件採択　https://aip.riken.jp/news/202203_cvpr/

Hina39 commented 6 months ago

INFO:main:Successfully parsed 2074 papers.

Hina39 commented 6 months ago

三件空欄あり

# 98番目
{
        "title": "",
        "author": "",
        "abstract": "",
        "page": "https://openaccess.thecvf.com/content/CVPR2022/html/Kim_E2V-SDE_From_Asynchronous_Events_to_Fast_and_Continuous_Video_Reconstruction_CVPR_2022_paper.html",
        "pdf": "https://openaccess.thecvf.com/content/CVPR2022/papers/Kim_E2V-SDE_From_Asynchronous_Events_to_Fast_and_Continuous_Video_Reconstruction_CVPR_2022_paper.pdf"
    },
# 554番目
{
        "title": "",
        "author": "",
        "abstract": "",
        "page": "https://openaccess.thecvf.com/content/CVPR2022/html/Wang_Accelerating_Neural_Network_Optimization_Through_an_Automated_Control_Theory_Lens_CVPR_2022_paper.html",
        "pdf": "https://openaccess.thecvf.com/content/CVPR2022/papers/Wang_Accelerating_Neural_Network_Optimization_Through_an_Automated_Control_Theory_Lens_CVPR_2022_paper.pdf"
    },
# 1701番目
    {
        "title": "",
        "author": "",
        "abstract": "",
        "page": "https://openaccess.thecvf.com/content/CVPR2022/html/Qin_A_Graph_Matching_Perspective_With_Transformers_on_Video_Instance_Segmentation_CVPR_2022_paper.html",
        "pdf": "https://openaccess.thecvf.com/content/CVPR2022/papers/Qin_A_Graph_Matching_Perspective_With_Transformers_on_Video_Instance_Segmentation_CVPR_2022_paper.pdf"
    },