search

HollowPrincess / Data-Analysis

This repo was created for writing a diploma.
1 stars 0 forks source link

Загрузить логи в базу #105

Closed HollowPrincess closed 5 years ago

HollowPrincess commented 5 years ago

https://medium.com/nuances-of-programming/начало-работы-с-elasticsearch-в-python-часть-1-73d6aa6be215

HollowPrincess commented 5 years ago

https://medium.com/nuances-of-programming/начало-работы-с-elasticsearch-в-python-часть-2-412796dcb213

HollowPrincess commented 5 years ago

https://nuancesprog.ru/p/1818/

HollowPrincess commented 5 years ago

https://habr.com/ru/post/280488/

HollowPrincess commented 5 years ago

https://ruseller.com/lessons.php?rub=37&id=2409

HollowPrincess commented 5 years ago

https://mkdev.me/posts/vvedenie-v-elk-sobiraem-filtruem-i-analiziruem-bolshie-dannye

HollowPrincess commented 5 years ago

Про json: http://qaru.site/questions/108532/importindex-a-json-file-into-elasticsearch

HollowPrincess commented 5 years ago

На python: http://carrefax.com/new-blog/2018/3/12/load-json-files-into-elasticsearch

HollowPrincess commented 5 years ago

Documentation: https://www.elastic.co/guide/index.html https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html

HollowPrincess commented 5 years ago

Mapping: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

HollowPrincess commented 5 years ago

https://tryolabs.com/blog/2015/02/17/python-elasticsearch-first-steps/

HollowPrincess commented 5 years ago

http://qaru.site/questions/6634367/upload-json-file-to-elastic-search-using-python

HollowPrincess commented 5 years ago

https://elasticsearch-py.readthedocs.io/en/master/

HollowPrincess commented 5 years ago

https://docs.objectrocket.com/elastic_python_examples.html

HollowPrincess commented 5 years ago

https://www.youtube.com/watch?v=s2U9mAnqH50

HollowPrincess commented 5 years ago

https://www.logicalfeed.com/posts/1182/upload-bulk-json-data-to-elasticsearch-using-python https://discuss.elastic.co/t/import-json-through-python/113766 https://stackoverflow.com/questions/43981275/index-json-files-in-elasticsearch-using-python

HollowPrincess commented 5 years ago

JSON файл должен соответствовать bulk формату, а именно: { "index": {"_index": "my_index", "_type":"mytype","_id":"my_id1"} } {1 документ-объект json} { "index": {"_index": "my_index", "_type":"mytype","_id":"my_id2"} } {2 документ-объект json} ...

HollowPrincess commented 5 years ago

Для того, чтобы написать файл, подходящий под Bulk установила jq

HollowPrincess commented 5 years ago

Создание файла с индексом по-умолчанию: jq-win64 -c ". | {"index": {}}, ." spbutest0.json > bulk.json

По-умолчанию индекс mydb тип _doc id случайный

HollowPrincess commented 5 years ago

Создание базы: curl -H "Content-Type: application/json" -XPOST "localhost:9200/mydb/_doc/_bulk?pretty&refresh" --data-binary "@bulk.json"

HollowPrincess commented 5 years ago

Посмотреть индексы: curl "localhost:9200/_cat/indices?v"

HollowPrincess commented 5 years ago

Удалить индекс: curl -X DELETE "localhost:9200/имя_индекса"

HollowPrincess commented 5 years ago

Ход работы: https://docs.google.com/document/d/1JGJz8bHL_RGRy7Glf4Gqc7yCwRxJ1-sJplIeUYURWn4/edit#