Human Activity Recognistion using CNN

[io23632]

How does this fit in the overall project?

The tutorial I found uses the WISDM dataset and a CNN deep learning model for activity recognistion. I can learn potentiallly new feature engineering techniques and evaluate a CNN model's potential in the smart watch application. In Fang Wangs dissertation in the Related Works section there is a section whcich talks about the potentiality of using CNN and its very accurate prediction capability for activity recognistion. Therefore this will be a section to include in my dissertation.

[io23632]

Notes:

1. Impersonal Model (Logistic Regression trained on 75% of user data, tested on 25%)

Observations:

•   Walking has the highest number of correct classifications (1376), as well as the highest number of misclassifications, especially as Upstairs (629).
•   Jogging is classified correctly most of the time (1553) but gets confused occasionally with Walking (185 times).
•   Activities like Downstairs and Upstairs are particularly misclassified, with Downstairs being confused with Walking (75 times) and Jogging (185 times).
•   The model seems to have difficulty distinguishing between activities with similar movement patterns, like Walking, Jogging, and Upstairs.

Conclusion:

•   The model works best for activities with distinct features like Jogging and Walking, but struggles to differentiate between similar activities such as Upstairs and Downstairs.

2. Personal Model (Logistic Regression on User 33 - 80% Train, 20% Test)

Observations:

•   The model performs exceptionally well on Walking (60 correct predictions) and Sitting (13), as well as Standing (6).
•   Misclassifications are few, with Downstairs being confused with Upstairs (4 times) and Jogging with Walking (1 time).
•   It achieves perfect accuracy on Sitting, Standing, and Walking, and high accuracy on Jogging.

Conclusion:

•   The personalized model performs extremely well on User 33, with near-perfect accuracy across most activities. This suggests the model effectively learns from and generalizes well on the user’s specific movement patterns.

3. Personal Model on Other Users (Logistic Regression on User 33’s Data Tested on 27 Other Users)

Observations:

•   The performance drops significantly when the model trained on User 33’s data is applied to other users.
•   Jogging (3784) is still predicted well, but misclassifications for other activities like Sitting, Standing, and Walking are high. For instance, Sitting is confused with Walking (390 times).
•   The model has particular trouble with activities like Upstairs and Walking, with Upstairs being classified as Jogging and Walking being classified as Jogging.

Conclusion:

•   The personal model overfits to User 33’s data, making it unsuitable for other users. The high misclassification rate demonstrates that the model fails to generalize across different users. This supports the notion of using personalized models for specific users but highlights the challenge of generalizing to broader populations.

4. Impersonal Model (CNN 2D Model on All Users with Limited Data and Without Additional Features)

Observations:

•   Downstairs and Upstairs activities are particularly challenging for this model, with Upstairs having low classification accuracy (only 13 correct predictions and many misclassifications as Downstairs).
•   Walking and Sitting perform well, with Walking having the highest accuracy.
•   The overall performance shows a fair amount of confusion between different activities, especially between Downstairs, Upstairs, and Jogging.

Conclusion:

•   The CNN model is somewhat effective but shows significant overfitting after a few epochs, as seen from the training and test curves. Additionally, the lack of hand-engineered features seems to contribute to the weaker performance, as the model fails to distinguish between some activities. Class imbalances also likely affect the model’s ability to accurately predict the rarer classes.

Overall Summary:

•   Personalized Models (like the one trained on User 33’s data) perform exceptionally well on the user they are trained on but fail when tested on other users.
•   Impersonal Models (trained on data from multiple users) offer better generalization but struggle with class imbalance and activities with similar movement patterns.
•   Feature Engineering plays a crucial role in improving model accuracy, as shown by the better performance of models with additional features compared to CNN models without them.
•   Cross-validation and confusion matrices are essential tools for understanding where models misclassify, offering insights for future improvements.

[io23632]

To train a CNN with time-series data like accelerometer readings, you shouldn’t aggregate statistical features across a window. Instead, you’ll want to keep the raw x-axis, y-axis, z-axis readings for each window, so that the CNN can learn patterns from the raw time-series data.

[io23632]

Non-CNN Models:

The models you mentioned that performed well on personalized data are typically traditional machine learning algorithms, such as:

•   Decision Trees
•   Logistic Regression
•   Random Forests
•   Gradient Boosting

These models are lightweight and rely on engineered features (such as the statistical features you calculated earlier). They are simpler to train and deploy but often need feature engineering to work effectively. These models are great for personalized settings where the model is trained on a single user’s data.

CNNs on Impersonal Data:

CNNs (Convolutional Neural Networks) are designed to automatically extract patterns from the raw data without much feature engineering. From your experimentation, it seems that CNNs trained on impersonal data perform just as well as traditional models trained on personal data.

Key Considerations for Smartwatches:

1.  Power Consumption:
•   CNNs are generally more computationally intensive due to the large number of parameters and the convolutional layers that perform matrix operations.
•   Decision Trees and other traditional models are much lighter and can be trained and executed with much less power and memory.
2.  Model Complexity:
•   CNNs will need more processing time and memory, which can be taxing on devices with limited resources like smartwatches.
•   Decision Trees and Logistic Regression are simpler models that can run efficiently with lower computational overhead.
3.  Personalized vs Impersonal Models:
•   Personalized models (like Decision Trees on personal data) seem to offer better performance because they are tailored to the user’s movement patterns.
•   Impersonal models (like CNNs) would work better in a generalized setting but might need more data and processing power.
4.  Data Preprocessing:
•   CNNs would require raw data windows (in 3D format), which the smartwatch needs to handle.
•   Traditional models can work with pre-engineered features, which might be easier to compute and store.
5.  Training on Device (On-Device Learning):
•   Traditional models like Decision Trees can be incrementally updated with new data, meaning the smartwatch can continuously personalize the model for the user without needing large amounts of data or computation.
•   CNNs are harder to retrain on-device due to their complexity and may require training on a server and then deploying the model.

Should You Use CNN for Smartwatches?

1.  Advantages of CNN:
•   Automatic feature extraction from raw data.
•   Good performance on impersonal (generalized) data.
•   Robust to noise and can handle larger and more complex data patterns.
2.  Disadvantages of CNN for Smartwatches:
•   High computational cost: More parameters, complex operations, and memory usage.
•   Battery drain: Continuous use of CNN models can quickly deplete the smartwatch’s battery.
•   Complex deployment: Harder to implement and fine-tune on limited hardware compared to Decision Trees or Logistic Regression.

Conclusion:

•   Decision Trees or Random Forests on personalized data might be more practical for real-time activity prediction on smartwatches due to their low power consumption and ability to adapt to a single user.
•   CNNs are powerful and can handle generalized data, but they may be overkill for a smartwatch, especially if you can achieve similar results with simpler models like Decision Trees on personalized data.
•   If you can train a personalized Decision Tree on a user’s data, it might provide high accuracy while being more efficient on a smartwatch.

In summary, CNNs are excellent for general-purpose applications, but for a battery-constrained device like a smartwatch, a personalized Decision Tree or traditional machine learning model may provide a better balance between accuracy and efficiency.

[io23632]

Overview of Using 2D CNN to predict Human Activity from smart phone accelerometer Data

---

1. Model Architecture

I implemented a 2D Convolutional Neural Network (CNN) to predict HAR using the WISDM data. The CNN model was built using Keras adn TensorFlow adn trained on dataset of user activities.

Model Layers:

Input Layer: The inputs to the model is a 2D accelerometer data (x, y and z axis) segmented into 5 second windows (100 data points) with a 50 data point overlap. Input Shape: (100, 3, 1) 100 is the number of samples, 3 is the number of features, 1 is for grayscale depth. What is grayscale depth?

Concolutional layers: First Layer: 16 filters of size (2x2) with ReLU activation to extract spatial patterns from input data. This layer captures local dependencies between nearby acclerometer values. Dropout (0.1): This prevents overfitting by randomly ignoring 10% of neurons during training.

Second Layer: 32 filters of size (2x2) with ReLU activation for deeper feature extraction Dropout (0.2): 20% of neurons are ignored, further improving model generalization.

Flatten Layer: The 2D feature maps are flattened into a 1D vector to be processed by fully connected layers.

Dense Layers: Dense (64 units): A fully connected layer with ReLU activation to learn complex patterns. Dropout (0.5): Half of the neurons are randomly ignored to avoid overfitting.

Output Layer: Dense (6 unit) The final layer has 6 output neurons (one for each activity) with a softmax actication that produces the probabilty distribution across the activity classes What is softmax activation?

4. Conclusion and Next Steps

The CNN model shows promising results with high accuracy (86%) on real-time activity prediction using accelerometer data. The use of convolutional layers allows the model to capture local patterns in the time-series data, making it effective for distinguishing activities based on motion.

Personalization vs. Generalization: A CNN model trained on multiple users can generalize well across users, but simpler models (like Decision Trees) may perform similarly for personalized models. Next Steps: • Optimize the model for deployment on resource-constrained devices like smartwatches (e.g., quantization, model pruning). • Consider training models on personalized data to explore model accuracy vs. computational cost trade-offs for on-device inference.