Add the option to persist or not the queue

dice-group / Squirrel

Squirrel searches and collects Linked Data

Other

22 stars 19 forks source link

Closed gsjunior86 closed 5 years ago

gsjunior86 commented 5 years ago

In the frontier, it would be nice to have an option to persist or not the queue on disk after the frontier terminator.

Suggestion: create an environment variable for it and drop or not the collections when the queue is being closed.

gsjunior86 commented 5 years ago

added the QUEUE_FILTER_PERSIST env variable on the frontier, which receives true or false for queue persistance