To ensure the CPS’s safe and proper operation, attacks must be rapidly recognized, identified, and pinpointed, and fast action must be taken to defend the entire system. Due to the unpredictability of attacks, the first step in strengthening resilience during the attack period and/or post-attack phase is to detect cyber attacks successfully. Regarding Anomaly Detection Systems(ADS), there are certain challenges. First, to determine the system’s typical behavior, the majority of cyber attack detection approaches, such as state estimation methods, build a model based on the available data. The system then assesses whether or not the system’s behavior is normal by comparing the estimated outputs of the model with the actual process outputs. In relation to the second problem, some intrusion detection techniques, such as Machine Learning (ML), can automatically build the model based on the training data set, which contains data instances that can be characterized using a set of attributes (features) and associated labels. Massive amounts of data—often referred to as "big data"—are needed but handling them is challenging. To tackle all these issues in this study, an online model-free algorithm is proposed to detect False Data Injection (FDI) and jamming attack on cyber-physical systems. Using Principal Component Analysis (PCA) in the observation space, the proposed method reconstructs expected observations in a reduced dimension space based on the most effective principle components. To begin with, the existence of an attack or normal operation is decided based on the measurement residual by using the Euclidean detector and the cosine-similarity metric detector. The proposed method has been evaluated by performing simulations on an IEEE-14 bus power system with 23 smart meters. In addition, the results have been compared with the model based Kalman estimation method which shows the outperformance of the proposed method in terms of statistical measures of binary classification problems such as precision, recall, and F-score. To extend our research work we use the paradigm of model-free reinforcement learning (RL) to describe the online attack/anomaly detection problem as a Partially Observable Markov Decision Process (POMDP) problem using the measurement provided by our new PCA-based technique. Numerical studies using Matlab show that the suggested RL-based algorithm is successful in detecting cyber attacks on the smart grid.