search

dogsheep / healthkit-to-sqlite

Convert an Apple Healthkit export zip to a SQLite database
https://datasette.io/tools/healthkit-to-sqlite
Apache License 2.0
191 stars 9 forks source link

Duplicate Column #21

Open FabianHertwig opened 2 years ago

FabianHertwig commented 2 years ago

Hey, thank you for this repo!

When I try to convert my export, I get a multiple column error. Here is the stack trace:

(.venv) (base) computer:bodyweight_app user$ healthkit-to-sqlite ./data/Health_export.zip ./data/healthkit.db
Importing from HealthKit  [###############################-----]   87%  00:00:22
Traceback (most recent call last):
  File "/MyProject/.venv/bin/healthkit-to-sqlite", line 10, in <module>
    sys.exit(cli())
  File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/cli.py", line 57, in cli
    convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)
  File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py", line 41, in convert_xml_to_sqlite
    write_records(records, db)
  File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py", line 146, in write_records
    batch_size=50,
  File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 2579, in insert_all
    extracts=extracts,
  File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 1246, in create
    extracts=extracts,
  File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 767, in create_table
    self.execute(sql)
  File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 421, in execute
    return self.conn.execute(sql)
sqlite3.OperationalError: duplicate column name: metadata_Meal
FabianHertwig commented 2 years ago

I think the issue is that I have records like these:

 <Record type="HKQuantityTypeIdentifierDietaryCholesterol" sourceName="MyFitnessPal" sourceVersion="35120" unit="mg" creationDate="2021-07-04 20:55:27 +0200" startDate="2021-07-04 20:55:00 +0200" endDate="2021-07-04 20:55:00 +0200" value="124">
  <MetadataEntry key="meal" value="Dinner"/>
  <MetadataEntry key="Meal" value="Dinner"/>
 </Record>

And if sqlite is case insensitive, then metadata_meal and metadata_Meal result in the same column.