Feature Selection and Performance Evaluation of Buzzer Classification Model
Main Article Content
Keywords
buzzer classification, feature selection, spearman correlation, pearson correlation, chi-square test, social media
Abstract
In the rapidly evolving digital age, social media platforms have transformed into battleground for shaping public opinion. Among these platforms, X has been particularly susceptible to the phenomenon of 'buzzers', paid or coordinated actors who manipulate online discussions and influence public sentiment. This manipulation poses significant challenges for users, researchers, and policymakers alike, necessitating robust detection measures and strategic feature selection for accurate classification models. This research explores the utilization of various feature selection techniques to identify the most influential features among the 24 features employed in the classification modeling using Support Vector Machine. This study found that selecting 11 key features yields a remarkably effective classification model, achieving an impressive F1-score of 87.54 in distinguishing between buzzer and non-buzzer accounts. These results suggest that focusing on the relevant features can improve the accuracy and efficiency of buzzer detection models. By providing a more robust and adaptable solution to buzzer detection, our research has the potential to advance social media research and policy. This enabling researchers and policymakers to devise strategies aimed at mitigating misinformation dissemination and cultivating an environment of trust and integrity within social media platforms, thus fostering healthier online interactions and discourse.
References
[2] M. Grandjean, “A social network analysis of Twitter: Mapping the digital humanities community,” Cogent Arts & Humanity, vol. 3, no. 1, 2016, doi: 10.1080/23311983.2016.1171458.
[3] L. K. Kaye, “Exploring the ‘socialness’ of social media,” Computers in Human Behavior Reports, vol. 3. Elsevier Ltd, Jan. 01, 2021. doi: 10.1016/j.chbr.2021.100083.
[4] D. M. Boyd and N. B. Ellison, “Social network sites: Definition, history, and scholarship,” Journal of Computer-Mediated Communication, vol. 13, no. 1, pp. 210–230, Oct. 2007, doi: 10.1111/j.1083-6101.2007.00393.x.
[5] G. Yavetz and N. Aharony, “Social media in government offices: usage and strategies,” Aslib Journal of Information Management, vol. 72, no. 4, pp. 445–462, Nov. 2020, doi: 10.1108/AJIM-11-2019-0313.
[6] I. Mergel, “Open innovation in the public sector: drivers and barriers for the adoption of Challenge.gov,” Public Management Review, vol. 20, no. 5, pp. 726–745, 2018, doi: 10.1080/14719037.2017.1320044.
[7] E. Rosenberg et al., “Sentiment analysis on Twitter data towards climate action,” Results in Engineering, vol. 19, Sep. 2023, doi: 10.1016/j.rineng.2023.101287.
[8] O. Czeranowska et al., “Migrants vs. stayers in the pandemic – A sentiment analysis of Twitter content,” Telematics and Informatics Reports, vol. 10, Jun. 2023, doi: 10.1016/j.teler.2023.100059.
[9] L. Ilias and I. Roussaki, “Detecting malicious activity in Twitter using deep learning techniques,” Appl Soft Comput, vol. 107, Aug. 2021, doi: 10.1016/j.asoc.2021.107360.
[10] M. T. Juzar and S. Akbar, “Buzzer Detection on Twitter Using Modified Eigenvector Centrality,” in 2018 5th International Conference on Data and Software Engineering (ICoDSE), 2018, pp. 1–5. doi: 10.1109/ICODSE.2018.8705788.
[11] M. Ibrahim, O. Abdillah, A. F. Wicaksono, and M. Adriani, “Buzzer Detection and Sentiment Analysis for Predicting Presidential Election Results in a Twitter Nation,” in Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015, Institute of Electrical and Electronics Engineers Inc., Jan. 2016, pp. 1348–1353. doi: 10.1109/ICDMW.2015.113.
[12] M. O. Ibrohim and I. Budi, “Hate speech and abusive language detection in Indonesian social media: Progress and challenges,” Heliyon, vol. 9, no. 8. Elsevier Ltd, Aug. 01, 2023. doi: 10.1016/j.heliyon.2023.e18647.
[13] M. Kantepe and M. C. Ganiz, “Preprocessing framework for Twitter bot detection,” in 2017 International Conference on Computer Science and Engineering (UBMK), 2017, pp. 630–634. doi: 10.1109/UBMK.2017.8093483.
[14] S. Wang, J. Tang, and H. Liu, “Feature Selection,” in Encyclopedia of Machine Learning and Data Mining, Boston, MA: Springer US, 2016, pp. 1–9. doi: 10.1007/978-1-4899-7502-7_101-1.
[15] H. I. Kuru, A. E. Cicek, and O. Tastan, “From Cell-Lines to Cancer Patients: Personalized Drug Synergy Prediction,” 2023, doi: 10.1101/2023.02.13.528276.
[16] R. Rodríguez-Pérez and J. Bajorath, “Feature importance correlation from machine learning indicates functional relationships between proteins and similar compound binding characteristics,” Sci Rep, vol. 11, no. 1, p. 14245, Jul. 2021, doi: 10.1038/s41598-021-93771-y.
[17] A. J. Panatra, F. B. Chandra, W. Darmawan, H. L. H. S. Warnars, W. H. Utomo, and T. Matsuo, “Buzzer Detection to Maintain Information Neutrality in 2019 Indonesia Presidential Election,” in Proceedings – 2019 8th International Congress on Advanced Applied Informatics, IIAI-AAI 2019, Institute of Electrical and Electronics Engineers Inc., Jul. 2019, pp. 873–876. doi: 10.1109/IIAI-AAI.2019.00177.
[18] A. Suciati, A. Wibisono, and P. Mursanto, “Twitter Buzzer Detection for Indonesian Presidential Election,” in 2019 3rd International Conference on Informatics and Computational Sciences (ICICoS), 2019, pp. 1–5. doi: 10.1109/ICICoS48119.2019.8982529.
[19] S. Pebiana et al., “Experimentation of Various Preprocessing Pipelines for Sentiment Analysis on Twitter Data about New Indonesia’s Capital City Using SVM and CNN,” in 2022 25th Conference of the Oriental COCOSDA International Committee for the Co-Ordination and Standardisation of Speech Databases and Assessment Techniques, O-COCOSDA 2022 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 2022. doi: 10.1109/O-COCOSDA202257103.2022.9997982.
[20] M. Lobo and R. D. Guntur, “Spearman’s rank correlation analysis on public perception toward health partnership projects between Indonesia and Australia in East Nusa Tenggara Province,” J Phys Conf Ser, vol. 1116, p. 022020, Dec. 2018, doi: 10.1088/1742-6596/1116/2/022020.
[21] P. Sedgwick, “Spearman’s rank correlation coefficient,” BMJ, p. g7327, Nov. 2014, doi: 10.1136/bmj.g7327.
[22] F. Zinzendoff Okwonu, B. Laro Asaju, and F. Irimisose Arunaye, “Breakdown Analysis of Pearson Correlation Coefficient and Robust Correlation Methods,” IOP Conf Ser Mater Sci Eng, vol. 917, no. 1, p. 012065, Sep. 2020, doi: 10.1088/1757-899X/917/1/012065.
[23] P. Schober, C. Boer, and L. A. Schwarte, “Correlation Coefficients: Appropriate Use and Interpretation,” Anesth Analg, vol. 126, no. 5, pp. 1763–1768, May 2018, doi: 10.1213/ANE.0000000000002864.
[24] S. T. Nihan, “Karl Pearsons chi-square tests,” Educational Research and Reviews, vol. 15, no. 9, pp. 575–580, Sep. 2020, doi: 10.5897/ERR2019.3817.
[25] R. Singhal and R. Rana, “Chi-square test and its application in hypothesis testing,” Journal of the Practice of Cardiovascular Sciences, vol. 1, no. 1, p. 69, 2015, doi: 10.4103/2395-5414.157577.
[26] M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, and B. Scholkopf, “Support vector machines,” IEEE Intelligent Systems and their Applications, vol. 13, no. 4, pp. 18–28, 1998, doi: 10.1109/5254.708428.
[27] M. Kantepe and M. C. Ganiz, “Preprocessing framework for Twitter bot detection,” in 2017 International Conference on Computer Science and Engineering (UBMK), IEEE, Oct. 2017, pp. 630–634. doi: 10.1109/UBMK.2017.8093483.
[28] A. J. Panatra, F. B. Chandra, W. Darmawan, H. L. H. S. Warnars, W. H. Utomo, and T. Matsuo, “Buzzer Detection to Maintain Information Neutrality in 2019 Indonesia Presidential Election,” in 2019 8th International Congress on Advanced Applied Informatics (IIAI-AAI), IEEE, Jul. 2019, pp. 873–876. doi: 10.1109/IIAI-AAI.2019.00177.
[29] M. Ibrahim, O. Abdillah, A. F. Wicaksono, and M. Adriani, “Buzzer Detection and Sentiment Analysis for Predicting Presidential Election Results in a Twitter Nation,” in 2015 IEEE International Conference on Data Mining Workshop (ICDMW), IEEE, Nov. 2015, pp. 1348–1353. doi: 10.1109/ICDMW.2015.113.
[30] C. Goutte and E. Gaussier, “A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation,” 2005, pp. 345–359. doi: 10.1007/978-3-540-31865-1_25.
Article Sidebar
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Authors' Retention of Copyright
Authors retain the copyright of their original work published in Jurnal Optimasi Sistem Industri (JOSI). By submitting their manuscript to the journal, authors grant Jurnal Optimasi Sistem Industri (JOSI) a non-exclusive license to publish the work under the CC BY-NC-SA 4.0 International License. This means that others are permitted to:
- Share and adapt the work for non-commercial purposes, provided proper attribution is given to the original authors and source.
- Create derivative works based on the original article, as long as the new work is non-commercial and under the same CC BY-NC-SA 4.0 International License.
Non-Commercial Usage
The CC BY-NC-SA 4.0 International License restricts commercial use of the published work and its derivatives. Commercial use includes generating revenue, financial gain, or any form of commercial exploitation. This limitation aims to preserve the scholarly and educational nature of the content.
ShareAlike Requirement
If you remix, adapt, or build upon the work, you must distribute your contributions under the same CC BY-NC-SA 4.0 International License as the original work. This "ShareAlike" requirement ensures that any new works based on the original content adhere to the same non-commercial, share-alike principles.
Usage, Attribution, and Citations
Authors and readers are allowed to:
- Share and distribute the published article in any medium or format.
- Adapt, remix, transform, and build upon the article for non-commercial purposes.
- Provide appropriate attribution to the original authors and the journal.
When using content from Jurnal Optimasi Sistem Industri (JOSI), proper attribution must be provided through citation of the original source and authors.
Copyright Infringement and Ethical Considerations
If you believe that any material published in Jurnal Optimasi Sistem Industri (JOSI) infringes upon your copyright or the copyright of someone you represent, please contact the editorial team promptly. The journal is committed to addressing copyright concerns in accordance with established ethical guidelines set forth by the Committee on Publication Ethics (COPE).
For any questions or inquiries related to copyright and licensing, please contact the editorial team at editor_josi[at]eng.unand.ac.id.