There are various handshape datasets for Sign Language. However:
This library aims to provide two main features:
This library is a work in progress. Contributions are welcome. If you wish to add a dataset you can make a push request or open an issue.
You can install handshape_datasets
via pip with:
pip install handshape_datasets
Simply import the module and load a dataset. The following downloads, preprocesses and load to memory the LSA16 dataset:
import handshape_datasets as hd
images,metadata = hd.load("lsa16")
Afterwards you can, for example, plot the first images of the dataset
import matplotlib.pyplot as plt
plt.imshow(images[0,:,:,:]) # show the first sample of the dataset
import handshape_datasets as hd
hd.list_datasets() # List available datasets
hd.load("lsa16",version="color",delete=True) # use the color version, delete temporary files
hd.delete_temporary_files("lsa16")# Delete temporary files (if any)
hd.clear("lsa16") # Delete all the local files for dataset LSA16
hd.info("lsa16") # Shows detailed info of the dataset, including url, data format, fields, etc.
Dataset id | Download size | Size on disk | Samples | Classes |
---|---|---|---|---|
lsa16 | 640.6 Kb | 1.2 Mb | 800 | 16 |
rwth | 44.8 Mb | 206.8 Mb | 3359 | 45 |
Irish | 173.4 Mb | 515.0 Mb | 58114 | 26 |
Ciarp | 10.6 Mb | 18.6 Mb | 7127 | 10 |
PugeaultASL_A | 2.1 Gb | 4.3 Gb | 65774 | 24 |
PugeaultASL_B | 317.4 Mb | 717.9 Mb | 72676 | 26 |
indianA | 1.7 Gb | 1.9 Gb | 5040 | 140 |
indianB | 320.5 Mb | 8.6 Gb | 5000 | 140 |
Nus1 | 2.8 Mb | 3.6 Mb | 479 | 10 |
Nus2 | 73.7 Mb | 106.1 Mb | 2750 | 10 |
jsl | 4.5 Mb | 7.9 Mb | 8055 | 41 |
psl | 285.2 Mb | 1.2 Gb | 960 | 16 |
You can find more information about the datasets in the following sign language dataset survey
Load the dataset:
x,metadata = handshape_datasets.load("lsa16")
y = metadata["y"]
Get the input_shape and number of classes:
input_shape = x[0].shape
classes = y.max() + 1
Define a model (using a pretrained MobileNet here):
base_model = keras.applications.mobilenet.MobileNet(input_shape=(input_shape[0],input_shape[1],3),
weights='imagenet', include_top=False)
output = keras.layers.GlobalAveragePooling2D()(base_model.output)
output = keras.layers.Dense(32, activation='relu')(output)
output = keras.layers.Dense(classes, activation='softmax')(output)
model = Model(inputs=base_model.input, outputs=output)
model.compile(optimizer='Adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
Split the dataset intro train/test sets:
X_train, X_test, Y_train, Y_test = sklearn.model_selection.train_test_split(x,y,
test_size=0.9,
stratify=y)
Fit the model
history = model.fit(X_train, Y_train, batch_size=self.batch_size, epochs=self.epochs, validation_data=(X_test, Y_test))
https://colab.research.google.com/drive/1kY-YrbegGFVT7NqVaeA4RjXYRVlZiISR?usp=sharing