Continuing experiment to capture the feature for different length of Sign Language video datasets
Problem= the video length is ranging from 0.36 to 8.12 seconds
Preprocess the videos before processing [1]
Fix frame [2]
Fix frame and repeat the videos if the videos has lesser frame [3]
Randomly select the consecutive frame on the videos [4]
Sign language divided into two categories [2]:
Isolated sign language [1], [2], [3], [4]
Continuous sign language[5]
Reference
[1] G. H. Samaan … Y. I. Cho, “MediaPipe’s Landmarks with RNN for Dynamic Sign Language Recognition,” Electron., vol. 11, no. 19, pp. 1–15, 2022, doi: 10.3390/electronics11193228.
[2] Y. C. Bilge, R. G. Cinbis, and N. Ikizler-Cinbis, “Towards Zero-Shot Sign Language Recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 1, pp. 1217–1232, 2022, doi: 10.1109/TPAMI.2022.3143074.
[3] S. Jiang, B. Sun, L. Wang, Y. Bai, K. Li, and Y. Fu, “Skeleton aware multi-modal sign language recognition,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2021, pp. 3408–3418. doi: 10.1109/CVPRW53098.2021.00380.
[4] P. Selvaraj, G. Nc, P. Kumar, and M. Khapra, “OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 2114–2133. doi: 10.18653/v1/2022.acl-
[5] F. Wen, Z. Zhang, T. He, and C. Lee, “AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove,” Nat. Commun., vol. 12, no. 1, pp. 1–13, 2021, doi: 10.1038/s41467-021-25637-w.
Next Plan
Creating research posters for symposiums - Deadline 8th December 2022
Continuing to explore multivariate reservoir computing
Implementing VAE or another dimensional reduction method
Drafting journal or conference paper for reservoir computing
Trying out existing implementations to gain insight into gesture recognition
Seminar 2023-01-18
ariers22
Progress
Continuing experiment to capture the feature for different length of Sign Language video datasets
Problem= the video length is ranging from 0.36 to 8.12 seconds
Preprocess the videos before processing [1]
Fix frame [2]
Fix frame and repeat the videos if the videos has lesser frame [3]
Randomly select the consecutive frame on the videos [4]
Sign language divided into two categories [2]:
Reference [1] G. H. Samaan … Y. I. Cho, “MediaPipe’s Landmarks with RNN for Dynamic Sign Language Recognition,” Electron., vol. 11, no. 19, pp. 1–15, 2022, doi: 10.3390/electronics11193228. [2] Y. C. Bilge, R. G. Cinbis, and N. Ikizler-Cinbis, “Towards Zero-Shot Sign Language Recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 1, pp. 1217–1232, 2022, doi: 10.1109/TPAMI.2022.3143074. [3] S. Jiang, B. Sun, L. Wang, Y. Bai, K. Li, and Y. Fu, “Skeleton aware multi-modal sign language recognition,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2021, pp. 3408–3418. doi: 10.1109/CVPRW53098.2021.00380. [4] P. Selvaraj, G. Nc, P. Kumar, and M. Khapra, “OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 2114–2133. doi: 10.18653/v1/2022.acl- [5] F. Wen, Z. Zhang, T. He, and C. Lee, “AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove,” Nat. Commun., vol. 12, no. 1, pp. 1–13, 2021, doi: 10.1038/s41467-021-25637-w.
Next Plan
Creating research posters for symposiums - Deadline 8th December 2022Trying out existing implementations to gain insight into gesture recognitionTry to implement Reservoir ComputingDrafting symposium paperAdding literature study