search

csimplestring / delta-go

Native Delta Lake Implementation in Go
37 stars 7 forks source link

json: cannot unmarshal bool into Go struct field CommitInfo.commitInfo.operationParameters of type string #50

Closed maxrem closed 3 months ago

maxrem commented 4 months ago

Hello!

I'm getting this error:

delta.ForTable: fail to get protocol and metadata when initializing snapshots: : json: cannot unmarshal bool into Go struct field CommitInfo.commitInfo.operationParameters of type string

In the _delta_log json:

{"operationParameters":{"mode":"Overwrite","statsOnLoad":false,"partitionBy":"[]"}}

Looks like statsOnLoad can't be unmarshalled.

csimplestring commented 4 months ago

Which version of the delta are you using ? I suspect the table is a newer version that this library does not support.

maxrem commented 4 months ago

I'm not sure which version we're using. I have to ask my team, but I think it's a recent one.

It looks like the operation parameters is a Map[String, Any] in the current Delta Lake code. Can I make a PR to change it to that in this library too?

Thanks!

maxrem commented 4 months ago

We're using Delta Lake v2.4.0

csimplestring commented 4 months ago

I'm not sure which version we're using. I have to ask my team, but I think it's a recent one.

It looks like the operation parameters is a Map[String, Any] in the current Delta Lake code. Can I make a PR to change it to that in this library too?

Thanks!

sure, waiting for your contribution.