search

gtadigital / ProfileParser

Other
1 stars 2 forks source link

new export data for ETL available (v3) #50

Closed thomashaensli closed 4 years ago

thomashaensli commented 4 years ago

HI Matteo,

here are the new place export data from EDB for ETL: https://shared.ethz.ch/s/4WNcfQ3YQJ5QD8y

Note: with raising complexity, EDB has troubles to dump all in one file. Thus, here you find batches of 100 records (7 folder à 100 xml à 100 records = approx 70k records). Can you work with this?

(I'll keep trying a one-piece dump meawhile)

Th.

thomashaensli commented 4 years ago

PS needless to write, this dump reflects the fixed typos in #49

thomashaensli commented 4 years ago

Hoi Matteo,

all-in-one export worked this time. Can you check it against the .XSL?

Thanks, Th.

BTW: new export record in line numbers... 59Mio! Technically, that's the new definition of verboseness...

matteoLorenzini commented 4 years ago

Processing the all in one export I got this error Exception in thread "main" java.lang.OutOfMemoryError: Java heap space so I used the other dump looping through folders.

New dump "EdB_dump" available here

matteoLorenzini commented 4 years ago

@thomashaensli here the new dump