fjxmlzn / DoppelGANger

[IMC 2020 (Best Paper Finalist)] Using GANs for Sharing Networked Time Series Data: Challenges, Initial Promise, and Open Questions
http://arxiv.org/abs/1909.13403
BSD 3-Clause Clear License
296 stars 75 forks source link

Training does not run although the input is of the required form #37

Closed dgtriantis closed 1 year ago

dgtriantis commented 1 year ago

Hello, I'm trying to run the TF2 version of DoppleGANger but I'm running into some issues. I have transformed the input data in the form that is required (according to README). When trying to train the GAN I run into the following error:

Traceback (most recent call last): File "C:\Users\dgtri\Desktop\DoppelGANger-TF2\example_training(wt)\main.py", line 29, in normalize_per_sample( File "C:\Users\dgtri\Desktop\DoppelGANger-TF2\example_training(wt)..\gan\util.py", line 182, in normalize_per_sample additional_attribute = np.stack(additional_attribute, axis=1) File "<__array_function__ internals>", line 5, in stack File "C:\Users\dgtri\anaconda3\lib\site-packages\numpy\core\shape_base.py", line 423, in stack raise ValueError('need at least one array to stack') ValueError: need at least one array to stack

I am wondering what is the cause of this error. It's worth noting that I have not pre-normalized the data inside of the numpy arrays to be inputed. I'm also attaching the numpy arrays in the following screenshots in order to make sure that they're in the right form. Their shapes are: (3682, 120, 4) (3682, 5) (3682, 120)

Screenshot (406) Screenshot (407) Screenshot (408)

Thank you!

fjxmlzn commented 1 year ago

Could you please provide the content inside "data_feature_output.pkl" that you prepared?

dgtriantis commented 1 year ago

Thanks a lot for the quick response, much appreciated! Turns out there was a minor mistake in the pickle file. It runs now!

I am also having a little trouble with the training as the following error occurs:

"Failed to create a NewWriteableFile: ./test/checkpoint\epoch_id-4\model-179.data-00000-of-00001.tempstate10424354672339237629 : The system cannot find the path specified."

I checked what was said here (https://github.com/fjxmlzn/DoppelGANger/issues/30) and I tried out the solution of chameleonzz but it caused problems in the following generation of data.

I see that you provided a possible solution in the case where the GPUTaskScheduler is used. I wonder if you could provide a solution to the said problem when using the example training WITHOUT the GPUTaskScheduler.

Thanks again!

fjxmlzn commented 1 year ago

I just merged the fixes from the master branch to TF2 branch. Can you re-download the code and try again? Thanks!

dgtriantis commented 1 year ago

After the changes you made it runs, thanks!

When generating samples, I am running in two problems:

  1. The same as https://github.com/fjxmlzn/DoppelGANger/issues/28 Do you have any input on this?

  2. The results are not logical, all attributes have 0 or 1 value and the features do not seem to have the required behavior. What could I do about that? Would running the training for more epochs and a lower sample_len help?

Again thank you for your help!

fjxmlzn commented 1 year ago

To use DoppelGANger, you will need to normalize the features into 0~1 or -1~1 and indicate the normalization method in the pkl files. From the screenshot you showed, the data is not normalized yet. It will cause problems in training.

dgtriantis commented 1 year ago

Ok, thank you for all the help!