boostcampaitech2 / mrc-level2-nlp-04

mrc-level2-nlp-04 created by GitHub Classroom
4 stars 5 forks source link

[DRAFT]Elasticsearch #28

Closed SinGonKim closed 2 years ago

SinGonKim commented 2 years ago

이슈에 내용은 올렸습니다. 이슈

이해하는데 오래걸리고 시행하는데 많은 시간이 걸려서 도움을 요청합니다. 48시간

제가 지금까지 진행한 코드 코드

elasticsearch-7.15.1 설치

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.1-linux-x86_64.tar.gz -q -P ../etc/ tar -xzf ../etc/elasticsearch-7.15.1-linux-x86_64.tar.gz -C ../etc/ chown -R daemon:daemon ../etc/elasticsearch-7.15.1

Python Library 설치

pip install elasticsearch pip install tqdm

nori Tokenizer 설치

../etc/elasticsearch-7.15.1/bin/elasticsearch-plugin install analysis-nori

elastic search stop word 설정

mkdir ../etc/elasticsearch-7.15.1/config/user_dic cp ../etc/my_stop_dic.txt ../etc/elasticsearch-7.15.1/config/user_dic/.

python script file 실행

python run_elasticsearch.py --path_to_elastic ../etc/elasticsearch-7.15.1/bin/elasticsearch --index_name wiki-index python run_elasticsearch.py --path_to_elastic ../etc/elasticsearch-7.15.1/bin/elasticsearch --index_name wiki-index-split-400 python run_elasticsearch.py --path_to_elastic ../etc/elasticsearch-7.15.1/bin/elasticsearch --index_name wiki-index-split-800 python run_elasticsearch.py --path_to_elastic ../etc/elasticsearch-7.15.1/bin/elasticsearch --index_name wiki-index-split-1000

elastic search 실행 여부 확인

ps -ef | grep elastic

참고코드

시간남으시면 한번 도전해주시면 좋을 것 같습니다.