Chest X-Ray (CXR) imaging has developed as an important technique for identifying lung diseases, especially in low- and middle-income nations where tuberculosis and pneumonia are serious health problems. With the onset of the COVID-19 pandemic, the need for early and accurate diagnosis has become even more pressing. This research presents a hybrid segmentation and classification for the multiclass lung disease classification using CXR images. The authors use Deep Atrous Attention U-Net (DAA-UNet), specifically designed for lung segmentation, enhancing the Region of Interest (RoI) for classification. The segmented lung regions are then classified using fine-tuned transfer learning on pre-trained models (ResNet101, ChexNet, DenseNet201, and InceptionV3). This hybrid segmentation and classification method achieves an average accuracy of 96.87%, significantly outperforming other classification models, as evidenced by metrics such as precision, sensitivity, specificity, and F1-score. This method exemplifies the potential for integrating deep learning classifiers with image segmentation to improve the diagnosis of lung disease, enabling early intervention and improved patient outcomes.