search

swerik-project / pyriksdagen

MIT License
0 stars 0 forks source link

Read metadata db from file #7

Closed BobBorges closed 1 month ago

BobBorges commented 6 months ago

load_Corpus_metadata takes a long time, here we can load the medatata db from a previously saved pickle

ninpnin commented 6 months ago

Do we need to pickle? CSV would be fine by me for our purposes.

BobBorges commented 6 months ago

pickle reads in faster, and you can't accidentally change the delimiter by opening it in libre office

ninpnin commented 6 months ago

I have slowly become afraid of binary data types. You never know what they are hiding in them.

But if this does make your workflow faster / easier then we can do it.

BobBorges commented 6 months ago

could always throw in a try: read_csv except: read_pickle if you want to use csv.