Long processing time - Githubissues

CLARIAH / COW

Integrated CSV to RDF converter, using CSVW and nanopublications

MIT License

46 stars 9 forks source link

Long processing time #112

Closed Bramvdhout closed 3 years ago

Bramvdhout commented 3 years ago

The transformation process of a large dataset can take up a looong time (ANDB dataset >2 hours).

This issue is extracted from Issue #108 opened by @MiconSchorsij

Bramvdhout commented 3 years ago

Surprisingly, COW is pretty darn fast when it comes to handling large datasets. But large datasets, containing a large number of transformations, will just take a lot of processing time. Especially on user-computers. We recommend running COW on a server to speed up this process.

MiconSchorsij commented 3 years ago

Question: how can I use CoW on a server? I would like to try that.

rlzijdeman commented 3 years ago

Not sure how you useful this is, but the shopping list would be:

buy a server or get some server space (e.g. via amazon);
install python3 if not already installed
get going with cow! see this new quick tutorial