Use array jobs to generate keywords

ericleasemorgan / reader

Distant Reader, a tool for using & understanding a corpus

GNU General Public License v2.0

20 stars 7 forks source link

Use array jobs to generate keywords #134

Closed dbrower closed 4 years ago

dbrower commented 4 years ago

Change the keywords processing for CORD to

run as an array job
process more than one source file with each python invocation
output stdout and stderr to same file

ericleasemorgan commented 4 years ago

Thank you, and earlier today I did the same thing for parts-of-speech. Now, using all there different feature extractions, I will up date the carrel-building process so our newly extracted feature sets are copied to a carrel as opposed to computed again. Duh!