A Comprehensive Study on Sentiment Analysis of Twitter Data: Techniques, Challenges, and Future Prospects

Khwaish Tiwari; Shreya *

Khwaish Tiwari Bennett University, Geater Noida, India
Shreya * Bennett University, Geater Noida, India

Keywords: Sentiment Analysis, Twitter Data, Machine Learning, Deep Learning, Sarcasm Detection, Multilingual Sentiment Analysis

Abstract

The increasing prevalence of social media platforms, especially Twitter, has led to an abundance of user-generated content that offers valuable insights into public opinion and sentiment. This paper presents a comprehensive study on sentiment analysis of Twitter data, emphasizing the significance of analyzing sentiments expressed in short, often informal, textual data. The study reviews and compares existing methods, including machine learning and dictionary-based approaches, for extracting and classifying sentiments. Advanced techniques such as Naive Bayes, Support Vector Machines, logistic regression, and deep learning models like Recurrent Neural Networks (RNN) are evaluated for their performance and applicability to Twitter sentiment analysis. The challenges of analyzing Twitter data, such as handling informal language, abbreviations, sarcasm, and context dependency, are explored. Preprocessing steps, including tokenization, removal of noise, and feature extraction techniques like N-grams and part-of-speech tagging, are implemented to enhance data quality. The study also integrates hybrid approaches to leverage the strengths of various models for improved accuracy and scalability. Experimental results demonstrate the effectiveness of combining traditional machine learning techniques with deep learning models to achieve higher sentiment classification accuracy. The findings highlight the potential of advanced natural language processing methods and multimodal content analysis in overcoming challenges unique to Twitter. This study contributes to the field by addressing critical research gaps and providing practical insights for real-time sentiment analysis applications. The implications span multiple domains, including business, politics, and public health, enabling organizations to make informed decisions based on dynamic user sentiments.

Downloads

Download data is not yet available.

References

[1]. Sarkar, Dipanjan, and Dipanjan Sarkar. "Sentiment analysis." Text Analytics with Python: A Practitioner's Guide to Natural Language Processing (2019): 567-629.
[2]. Mewada, A., Dewang, R.K. SA-ASBA: a hybrid model for aspect-based sentiment analysis using synthetic attention in pre-trained language BERT model with extreme gradient boosting. J Supercomput 79, 5516–5551 (2023).
[3]. Mewada, Arvind, et al. "SentiBERT: A Novel Approach for Fake Review Detection Incorporating Sentiment Features with Contextual Features." Proceedings of the 2023 Fifteenth International Conference on Contemporary Computing. 2023.
[4]. Jain, Swati, et al. "Event Detection through Lexical Chain Based Semantic Similarity Algorithm." IOP Conference Series: Materials Science and Engineering. Vol. 1166. No. 1. IOP Publishing, 2021.
[5]. Juneja, Saloni, et al. "Spam Review Detection Using Okapi Relevance Method for Negative Reviews." Data, Engineering and Applications: Select Proceedings of IDEA 2021. Singapore: Springer Nature Singapore, 2022. 493-504.
[6]. Bordoloi, M., Biswas, S.K. Sentiment analysis: A survey on design framework, applications and future scopes. Artif Intell Rev 56, 12505–12560 (2023).
[7]. Mohammad SM, Turney PD (2013) Crowdsourcing a word-emotion association lexicon. Comput Intell 29(3):436–465.
[8]. Pang, B., and L. Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2 (1–2):1–135.
[9]. Go, Alec, Lei Huang, and Richa Bhayani. "Twitter sentiment analysis." Entropy 17 (2009): 252.
[10]. Kim, Yoon, "Convolutional Neural Networks for Sentence Classification." arXiv, 2014, arxiv.org/abs/1408.5882.
[11]. Tang, Duyu, Bing Qin, and Ting Liu. "Document modelling with gated recurrent neural network for sentiment classification." Proceedings of the 2015 conference on empirical methods in natural language processing. 2015.
[12]. F. Barbieri, J. Camacho-Collados, L. E. Anke, and L. Neves, “Tweeteval: Unified benchmark and comparative evaluation for tweet classification,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, 2020, pp. 1644–1650.
[13]. Pennington, Jeffrey, Socher, Richard, Manning, Christopher D (2014). ”GloVe: Global vectors for word representation.” In Proceedings of the 2014 Conference on EMNLP Doha, Qatar (pp. 1532–1543).
[14]. Chiche, Alebachew, and Betselot Yitagesu. "Part of speech tagging: a systematic review of deep learning and machine learning approaches." Journal of Big Data 9.1 (2022): 10.
[15]. Fersini, Elisabetta, Enza Messina, and Federico Alberto Pozzi. "Sentiment analysis: Bayesian ensemble learning." Decision Support Systems 68 (2014): 26-38.
[16]. Das, Ringki, and Thoudam Doren Singh. "Multimodal sentiment analysis: a survey of methods, trends, and challenges." ACM Computing Surveys 55.13s (2023): 1-38.
[17]. Cui, Jingfeng, et al. "Survey on sentiment analysis: evolution of research methods and topics." Artificial Intelligence Review 56.8 (2023): 8469-8510.
[18]. Zhu, Linan, et al. "Multimodal sentiment analysis based on fusion methods: A survey." Information Fusion 95 (2023): 306-325.
[19]. Zhang, Wenxuan, et al. "A survey on aspect-based sentiment analysis: Tasks, methods, and challenges." IEEE Transactions on Knowledge and Data Engineering 35.11 (2022): 11019-11038.
[20]. Chan, Jireh Yi-Le, et al. "State of the art: a review of sentiment analysis based on sequential transfer learning." Artificial Intelligence Review 56.1 (2023): 749-780.
[21]. Bordoloi, Monali, and Saroj Kumar Biswas. "Sentiment analysis: A survey on design framework, applications and future scopes." Artificial intelligence review 56.11 (2023): 12505-12560.
[22]. Xu, Qianwen Ariel, Victor Chang, and Chrisina Jayne. "A systematic review of social media-based sentiment analysis: Emerging trends and challenges." Decision Analytics Journal 3 (2022): 100073.
[23]. Dake, Delali Kwasi, and Esther Gyimah. "Using sentiment analysis to evaluate qualitative students’ responses." Education and Information Technologies 28.4 (2023): 4629-4647.
[24]. Huang, Bo, et al. "Aspect-level sentiment analysis with aspect-specific context position information." Knowledge-Based Systems 243 (2022): 108473.