search

cyberdyne-ventures / skynet-data

Data Layer For the Skynet Project, Including Sample Data
Other
3 stars 0 forks source link

Refactored and optimized code to avoid duplicate loops #10

Closed rewanthtammana closed 3 months ago

rewanthtammana commented 4 months ago

Optimized code to avoid duplicate loops and unwanted function calls.

Managed to reduce the neo4j data ingestion time from 26+ minutes to 6 minutes for 40k+ alerts.

image
rewanthtammana commented 4 months ago

Podman commands to spin up a neo4j container.

podman run -d \
    -p 7474:7474 -p 7687:7687 \
    --env NEO4J_AUTH=neo4j/password \
    -e NEO4J_server_memory_heap_initial__size=2G \
    -e NEO4J_server_memory_heap_max__size=4G \
    -e NEO4J_apoc_export_file_enabled=true \
    -e NEO4J_apoc_import_file_enabled=true \
    -e NEO4J_apoc_import_file_use__neo4j__config=true \
    -e NEO4J_PLUGINS='["apoc"]' \
    --name neo4j-apoc \
    neo4j