Sentiment Analysis of Sirekap Tweets Using CNN Algorithm
Abstract
Background: The research investigates the application of deep learning models for sentiment analysis on Twitter data related to Indonesia's Sirekap system. Sentiment analysis is crucial for understanding public opinion and enhancing the transparency and reliability of election result recapitulation processes. Objective: The objective of this study is to compare the performance of Convolutional Neural Networks (CNN) and CNN-LSTM models in analyzing sentiments from tweets about the Sirekap system. The study aims to identify the most effective model and preprocessing techniques to improve sentiment classification accuracy. Methods: A comprehensive data preprocessing pipeline was implemented, including cleansing, case folding, tokenizing, normalization, stopword removal, and stemming. To address class imbalance, the SMOTE technique was applied. The models were trained and evaluated using accuracy, precision, recall, and F1-score metrics. Pre-trained word embeddings were used to enhance model performance. Results: The CNN model achieved an accuracy of 85.90%, outperforming the CNN-LSTM model, which achieved 79.91% accuracy. Additionally, the CNN model demonstrated superior precision, recall, and F1-score metrics compared to the CNN-LSTM model. The thorough preprocessing and handling of class imbalance significantly contributed to the enhanced performance of the CNN model. Conclusion: The research emphasizes the effectiveness of deep learning approaches, particularly CNNs, in sentiment analysis tasks. The findings highlight the importance of comprehensive preprocessing and class imbalance handling. The use of pre-trained word embeddings and various evaluation metrics ensures robust model performance. These insights contribute to improving the accuracy and efficiency of sentiment classification, thereby enhancing the reliability and transparency of election result recapitulation processes.
Downloads
References
“KPU - PAGE.” Accessed: Mar. 07, 2024. [Online]. Available: https://www.kpu.go.id/page/read/12/pemilu-dalam-sejarah
“KPU - PAGE.” Accessed: Mar. 10, 2024. [Online]. Available: https://www.kpu.go.id/page/read/5/tugas-dan-kewenangan
“Manfaatkan Sirekap, Transparan dan Kemudahan untuk Masyarakat - KPU.” Accessed: Mar. 10, 2024. [Online]. Available: https://www.kpu.go.id/berita/baca/10143/manfaatkan-sirekap-transparan-dan-kemudahan-untuk-masyarakat
M. Rodríguez-Ibánez, A. Casánez-Ventura, F. Castejón-Mateos, and P. M. Cuenca-Jiménez, “A review on sentiment analysis from social media platforms,” Expert Syst. Appl., vol. 223, no. March, 2023, doi: 10.1016/j.eswa.2023.119862.
M. K. Anam, M. I. Mahendra, W. Agustin, R. Rahmaddeni, and N. Nurjayadi, “Framework for Analyzing Netizen Opinions on BPJS Using Sentiment Analysis and Social Network Analysis (SNA),” INTENSIF J. Ilm. Penelit. dan Penerapan Teknol. Sist. Inf., vol. 6, no. 1, pp. 11–28, 2022, doi: 10.29407/intensif.v6i1.15870.
H. T. Phan, V. C. Tran, N. T. Nguyen, and D. Hwang, “Improving the Performance of Sentiment Analysis of Tweets Containing Fuzzy Sentiment Using the Feature Ensemble Model,” IEEE Access, vol. 8, pp. 14630–14641, 2020, doi: 10.1109/ACCESS.2019.2963702.
A. Yadav and D. K. Vishwakarma, “Sentiment analysis using deep learning architectures: a review,” Artif. Intell. Rev., vol. 53, no. 6, pp. 4335–4385, 2020, doi: 10.1007/s10462-019-09794-5.
Y. Cheng et al., “Sentiment Analysis Using Multi-Head Attention Capsules with Multi-Channel CNN and Bidirectional GRU,” IEEE Access, vol. 9, pp. 60383–60395, 2021, doi: 10.1109/ACCESS.2021.3073988.
M. A. Kausar, A. Soosaimanickam, and M. Nasar, “Public Sentiment Analysis on Twitter Data during COVID-19 Outbreak,” Int. J. Adv. Comput. Sci. Appl., vol. 12, no. 2, pp. 415–422, 2021, doi: 10.14569/IJACSA.2021.0120252.
P. O. A. Sunarya, R. Refianti, A. B. Mutiara, and W. Octaviani, “Comparison of accuracy between convolutional neural networks and Naïve Bayes Classifiers in sentiment analysis on Twitter,” Int. J. Adv. Comput. Sci. Appl., vol. 10, no. 5, pp. 77–86, 2019, doi: 10.14569/ijacsa.2019.0100511.
G. A. M. Kukuh Jaluwana, Gusti Made Arya Sasmita, and I Made Agus Dwi Suarjaya, “Analysis of Public Sentiment Towards Goverment Efforts to Break the Chain of Covid-19 Transmission in Indonesia Using CNN and Bidirectional LSTM,” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 6, no. 4, pp. 511–520, 2022, doi: 10.29207/resti.v6i4.4055.
M. Dehghani and Z. Yazdanparast, “Political Sentiment Analysis of Persian Tweets Using CNN-LSTM Model,” 2023, doi: 10.48550/arXiv.2307.07740.
M. Daffa Dhiyaulhaq, S. Jurnalis Pipin, F. Mikael Sinaga, S. Winardi, and M. Noor Hakim, “Sentiment Analysis Classification of ChatGPT on Twitter Big Data in Indonesia Using Fast R-CNN,” Media Inform. Budidarma, vol. 7, no. 4, pp. 2137–2148, 2023, doi: 10.30865/mib.v7i4.6816.
H. Yanagimoto and S. Isaji, “Incident related tweet extraction with density ratio estimation,” Procedia Comput. Sci., vol. 35, no. C, pp. 456–463, 2014, doi: 10.1016/j.procs.2014.08.126.
R. Wongso, F. A. Luwinda, B. C. Trisnajaya, O. Rusli, and Rudy, “News Article Text Classification in Indonesian Language,” Procedia Comput. Sci., vol. 116, pp. 137–143, 2017, doi: 10.1016/j.procs.2017.10.039.
M. Rodriguez-Ibanez, F. J. Gimeno-Blanes, P. M. Cuenca-Jimenez, C. Soguero-Ruiz, and J. L. Rojo-Alvarez, “Sentiment Analysis of Political Tweets from the 2019 Spanish Elections,” IEEE Access, vol. 9, pp. 101847–101862, 2021, doi: 10.1109/ACCESS.2021.3097492.
V. A. Fitri, R. Andreswari, and M. A. Hasibuan, “Sentiment analysis of social media Twitter with case of Anti-LGBT campaign in Indonesia using Naïve Bayes, decision tree, and random forest algorithm,” Procedia Comput. Sci., vol. 161, pp. 765–772, 2019, doi: 10.1016/j.procs.2019.11.181.
M. J. Tan and C. H. Guan, “Are people happier in locations of high property value? Spatial temporal analytics of activity frequency, public sentiment and housing price using twitter data,” Appl. Geogr., vol. 132, p. 102474, 2021, doi: 10.1016/j.apgeog.2021.102474.
B. Kanwal et al., “Opinion Mining from Online Travel Reviews: An Exploratory Investigation on Pakistan Major Online Travel Services Using Natural Language Processing,” IEEE Access, vol. 11, no. March, pp. 29934–29945, 2023, doi: 10.1109/ACCESS.2023.3260114.
A. G. Gozal, H. Pranoto, and M. F. Hasani, “Sentiment analysis of the Indonesian community toward face-to-face learning during the Covid-19 pandemic,” Procedia Comput. Sci., vol. 227, pp. 398–405, 2023, doi: 10.1016/j.procs.2023.10.539.
F. Zuhad and N. Wilantika, “Perbandingan Penggunaan Kamus Normalisasi dalam Analisis Sentimen Berbahasa Indonesia,” J. Linguist. Komputasional, vol. 5, no. 1, pp. 13–23, 2022, doi: 10.26418/jlk.v5i1.60.
R. Ahuja, A. Chug, S. Kohli, S. Gupta, and P. Ahuja, “The Impact of Features Extraction on the Sentiment Analysis,” Procedia Comput. Sci., vol. 152, pp. 341–348, Jan. 2019, doi: 10.1016/J.PROCS.2019.05.008.
M. Demircan, A. Seller, F. Abut, and M. F. Akay, “Developing Turkish sentiment analysis models using machine learning and e-commerce data,” Int. J. Cogn. Comput. Eng., vol. 2, no. November, pp. 202–207, 2021, doi: 10.1016/j.ijcce.2021.11.003.
A. Erkan and T. Gungor, “Analysis of Deep Learning Model Combinations and Tokenization Approaches in Sentiment Classification,” IEEE Access, vol. 11, no. December, pp. 134951–134968, 2023, doi: 10.1109/ACCESS.2023.3337354.
D. Ramachandran and R. Parvathi, “Analysis of Twitter Specific Preprocessing Technique for Tweets,” Procedia Comput. Sci., vol. 165, pp. 245–251, 2019, doi: 10.1016/j.procs.2020.01.083.
M. Young et al., “Natural language processing to assess the epidemiology of delirium-suggestive behavioural disturbances in critically ill patients,” Crit. Care Resusc., vol. 23, no. 2, pp. 144–153, 2021, doi: 10.51893/2021.2.oa1.
A. Jabbar, S. Iqbal, M. I. Tamimy, A. Rehman, S. A. Bahaj, and T. Saba, “An Analytical Analysis of Text Stemming Methodologies in Information Retrieval and Natural Language Processing Systems,” IEEE Access, vol. 11, no. December, pp. 133681–133702, 2023, doi: 10.1109/ACCESS.2023.3332710.
“Sastrawi · PyPI.” Accessed: Aug. 28, 2024. [Online]. Available: https://pypi.org/project/Sastrawi/
H. S. Hota, D. K. Sharma, and N. Verma, Lexicon-based sentiment analysis using Twitter data. Elsevier Inc., 2021. doi: 10.1016/B978-0-12-824536-1.00015-0.
C. Padurariu and M. E. Breaban, “Dealing with data imbalance in text classification,” Procedia Comput. Sci., vol. 159, pp. 736–745, 2019, doi: 10.1016/j.procs.2019.09.229.
V. Rupapara, F. Rustam, H. F. Shahzad, A. Mehmood, I. Ashraf, and G. S. Choi, “Impact of SMOTE on Imbalanced Text Features for Toxic Comments Classification Using RVVC Model,” IEEE Access, vol. 9, pp. 78621–78634, 2021, doi: 10.1109/ACCESS.2021.3083638.
A. Arafa, N. El-Fishawy, M. Badawy, and M. Radad, “RN-SMOTE: Reduced Noise SMOTE based on DBSCAN for enhancing imbalanced data classification,” J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 8, pp. 5059–5074, 2022, doi: 10.1016/j.jksuci.2022.06.005.
“Splitting Datasets With scikit-learn and train_test_split() (Overview) – Real Python.” Accessed: May 25, 2024. [Online]. Available: https://realpython.com/lessons/splitting-datasets-overview/
S. Tammina and S. Annareddy, “Sentiment analysis on customer reviews using convolutional neural network,” 2020 Int. Conf. Comput. Commun. Informatics, ICCCI 2020, pp. 20–25, 2020, doi: 10.1109/ICCCI48352.2020.9104086.
S. Tam, R. Ben Said, and Ö. Tanriöver, “A ConvBiLSTM Deep Learning Model-Based Approach for Twitter Sentiment Classification,” IEEE Access, vol. 9, pp. 41283–41293, 2021, doi: 10.1109/ACCESS.2021.3064830.
W. Meng, Y. Wei, P. Liu, Z. Zhu, and H. Yin, “Aspect Based Sentiment Analysis with Feature Enhanced Attention CNN-BiLSTM,” IEEE Access, vol. 7, pp. 167240–167249, 2019, doi: 10.1109/ACCESS.2019.2952888.
J. Wang, L. C. Yu, K. R. Lai, and X. Zhang, “Tree-Structured Regional CNN-LSTM Model for Dimensional Sentiment Analysis,” IEEE/ACM Trans. Audio Speech Lang. Process., vol. 28, pp. 581–591, 2020, doi: 10.1109/TASLP.2019.2959251.
Y. Bao and S. Yang, “Two Novel SMOTE Methods for Solving Imbalanced Classification Problems,” IEEE Access, vol. 11, no. December 2022, pp. 5816–5823, 2023, doi: 10.1109/ACCESS.2023.3236794.
M. Heydarian, T. E. Doyle, and R. Samavi, “MLCM: Multi-Label Confusion Matrix,” IEEE Access, vol. 10, pp. 19083–19095, 2022, doi: 10.1109/ACCESS.2022.3151048.
D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, no. 1, pp. 1–13, 2020, doi: 10.1186/s12864-019-6413-7.
L. Yang, Y. Li, J. Wang, and R. S. Sherratt, “Sentiment Analysis for E-Commerce Product Reviews in Chinese Based on Sentiment Lexicon and Deep Learning,” IEEE Access, vol. 8, pp. 23522–23530, 2020, doi: 10.1109/ACCESS.2020.2969854.
Ç. Oğuz and M. Yağanoğlu, “Detection of COVID-19 using deep learning techniques and classification methods,” Inf. Process. Manag., vol. 59, no. 5, pp. 1–18, 2022, doi: 10.1016/j.ipm.2022.103025.
Copyright (c) 2024 Handoko Handoko, Ahmad Asrofiq, Junadhi Junadhi, Ari Sukma Negara
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
1. Copyright on any article is retained by the author(s).
2. The author grants the journal, right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work’s authorship and initial publication in this journal.
3. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal’s published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
4. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
5. The article and any associated published material is distributed under the Creative Commons Attribution-ShareAlike 4.0 International License
