Getting ValueError (sdv-pii-25szo) while sampling synthesizer on SDV==1.13.0

sdv-dev / SDV

Synthetic data generation for tabular data

https://docs.sdv.dev/sdv

Other

2.32k stars 304 forks source link

Getting ValueError (sdv-pii-25szo) while sampling synthesizer on SDV==1.13.0 #2023

Closed Dharmik2510 closed 4 months ago

Dharmik2510 commented 4 months ago

I am using SDV version 1.13.0 with the simplify_schema option.

The model trains and saves successfully to the specified location. However, when I call the sample method to generate synthetic data, I encounter the following error:

=================================================

ValueError: invalid literal for int() with base 10: 'sdv-pii-25szo'

=================================================

srinify commented 4 months ago

Hi there @Dharmik2510 do you mind sharing a bit more context here for us to try to reproduce this error?

What did your metadata look like before and after running simplify_schema()?
Were you using HMASynthesizer (just to double check)?
By "saves successfully" do you mean exporting the model using save()?
Does the synthesizer model work after training but before you saved / exported it? Is the error only happening when you re-import the model and try to sample?
Do you mind sharing the full code you ran and sharing the full stack trace? You can hide your dataset if its sensitive

Let us know and we can start digging into this! Thanks!

srinify commented 4 months ago

Hi @Dharmik2510 just following up :) Let me know if you're still blocked by this issue

srinify commented 4 months ago

Hi @Dharmik2510 I haven't heard from you in a while so I'm closing this issue out for now. If you still need help, feel free to comment and tag me and I can re-open!

npatki commented 3 months ago

Hi @Dharmik2510 -- it looks like we've replicated the issue in #2064. It seems to be happening if the metadata has detected a column as 'unknown' even though it's supposed to be 'numerical'. The metadata auto-detection is not guaranteed to be accurate, so it's important to check it to ensure it accurately describes your data. A quick fix here would be to update the metadata for the column to be 'numerical'.

For more information about inspecting and updating your metadata, please see the Metadata API docs. Thanks.