Can not reproduce the results.

[mfaisal59]

Hi, I have trained the writer-dependent classifiers using your pre-trained models. But EER score is not the same as reported in your papers.

[luizgh]

Hi, can you please inform :

• The EER values that you obtained
• The dataset
• Which commands you used for preprocessing
• Which command you used for training the WD classifiers

[mfaisal59]

Hi, I used this command to preprocess the GPDS dataset python -m sigver.preprocessing.process_dataset --dataset gpds --path ./datasets/gpds --save-path gpds_170_242.npz

And I trained the writer dependent classifiers using the pre-trained models provided by you using the following command: python -m sigver.wd.test -m signet --model-path <path/to/trained_model> \ --data-path <path/to/data> --save-path <path/to/save> \ --exp-users 0 300 --dev-users 300 881 --gen-for-train 12

I obtained these results: For signet.pth

For signet_f_lambda_0.95.pth

[luizgh]

@mfaisal59 thanks for the info. These results are way off. I just re-ran these experiments with the GPDS dataset, and here is what I get (left is sigver; right is sigver-f)

The first I think it is worth checking is the data. To be sure, these experiments use the Grayscale version of the GPDS dataset - can you confirm that this is the case for your dataset? (I know that some version of GPDS was shared as binary images).

Please check the histogram of the first image in the dataset:

import skimage
import numpy as np
import matplotlib.pyplot as plt 
img = skimage.io.imread('<path>/002/c-002-01.png') # replace the data path here
plt.hist(img.reshape(-1))
plt.yscale('log')
plt.savefig('histogram.png')
print(img.shape)

I got this (image of shape 519, 1142):

[mfaisal59]

Hi, The dataset, that I downloaded, have images in jpg format whereas you have in png format. Did you convert the images to png? I got this histogram for the same image:

[luizgh]

Hi @mfaisal59,

I think you are using the GPDS Synthetic dataset instead of the GPDS-960 dataset. I also have a copy of the GPDS synthetic dataset, and 1) they are jpg images (whereas GPDS-960 has png images) 2) I generated the histogram for c-002-01.jpg (on the synthetic dataset) and it is exacly the same as the one you posted above.

Please note that all experiments in the paper were done with GPDS-960, MCYT-75, CEDAR and the Brazilian PUC-PR. We did not perform any experiments with the GPDS synthetic dataset. In my opinion, you should take results on the synthetic dataset with a grain of salt: while the procedure to generate the genuine signatures is reasonable, I don't quite agree with the authors on how they generate the skilled forgeries (see the paper that introduced the dataset: [1]). In practice, we observe some particularities in skilled forgeries (such as limp strokes, caused by slow hand movements) that were not modeled in the synthetic dataset.

[1] Miguel A. Ferrer, Mois\E9s Diaz-Cabrera, Aythami Morales, "Static Signature Synthesis: A Neuromotor Inspired Approach for Biometrics," IEEE Transactions on Pattern Analysis and Machine Intelligence, ISSN: 0162-8828, vol. 37, no. 3, pp.1- 667-680, March 2015.

[luizgh]

@mfaisal59 I am closing this issue since the results you obtained were from a different dataset than those used in the paper. If you have other questions / find any issues, please create a new issue.

[mfaisal59]

Hi, I have one confusion, I downloaded GPDS Synthetic Signature database from this link http://www.gpds.ulpgc.es/ under the license agreement, this dataset consists of signatures of 4000 users. Whereas in your paper you used GPDS-960 dataset, I can not find this dataset link. From where should I download this dataset. Is GPDS-960 the subset of GPDS Synthetic? Thanks in advance.

[luizgh]

Hi @mfaisal59,

The GPDS-960 is a different dataset - it is not a synthetic dataset, but rather was collected from 960 individuals. It used to be available in the same page you listed above (search for the term "GPDS960GRAYsignature database"), but unfortunately it seems that it is no longer being distributed.

You can still use the CEDAR (https://cedar.buffalo.edu/signature/) and MCYT-75 (https://atvs.ii.uam.es/atvs/mcyt75so.html) datasets.

[gonultasbu]

@mfaisal59 here is the SigNet model trained on GPDS4000.

https://drive.google.com/open?id=1GbvMf1V7_r6CajO4UQMh4myQ3qTG6xpt