Among radiological imaging data, Chest X-rays (CXRs) are of great use in observing COVID-19 manifestations. For mass screening, using CXRs, a computationally efficient AI-driven tool is the must to detect COVID-19-positive cases from non-COVID ones. For this purpose, we proposed a light-weight Convolutional Neural Network (CNN)-tailored shallow architecture that can automatically detect COVID-19-positive cases using CXRs, with no false negatives. The shallow CNN-tailored architecture was designed with fewer parameters as compared to other deep learning models. The shallow CNN-tailored architecture was validated using 321 COVID-19-positive CXRs. In addition to COVID-19-positive cases, another set of non-COVID-19 5856 cases (publicly available, source: Kaggle) was taken into account, consisting of normal, viral, and bacterial pneumonia cases. In our experimental tests, to avoid possible bias, 5-fold cross-validation was followed, and both balanced and imbalanced datasets were used. The proposed model achieved the highest possible accuracy of 99.69%, sensitivity of 1.0, where AUC was 0.9995. Furthermore, the reported false positive rate was only 0.0015 for 5856 COVID-19-negative cases. Our results stated that the proposed CNN could possibly be used for mass screening. Using the exact same set of CXR collection, the current results were better than other deep learning models and major state-of-the-art works.