An Exploration of Various Image and Video Deepfake Detection Methods and Models
Keywords:
Deepfake, Face Synthesis, Attribute manipulation, Identity Swap, Face Reenactment, Lip-SyncingAbstract
Deepfake has become the most challenging attack in this digital age. It has become source of spreading misinformation and forging people through social media content. Deepfake attacks like face synthesis, attribute manipulation, identity swap, face reenactment and lips syncing are discussed in this paper. After deepfake is acquainted as a threat to human beings, various detection methods and models have developed based on machine learning, deep learning, transfer learning and digital forensics. In this research paper survey of deepfake detection techniques based on CNN, RNN, XceptionNet, EfficientNet, ResNext, MesoNet and, CLRNet, LSTM and other methods are provided. Different ensemble models are also developed by combining various algorithms. This paper provides an in-depth analysis of evolution, types, impacts and deepfake detection methods by considering visual artifacts, temporal features and handcrafted features. This track record of deep-fake and evolving methods in the deepfake detection process could be helpful for researchers in their future research.
Downloads
References
Rachael Brooks, Yefeng Yuan, Yuhong Liu and Haiquan Chen, “DeepFake and its Enabling Techniques: A Review”, APSIPA Transactions on Signal and Information Processing, 2022, Received 24 April 2022; Revised 23 June 2022, ISSN 2048-7703; DOI 10.1561/116.00000024, 2022.
Teng Zhang, Lirui Deng, Liang Zhang, Xianglei Dang, “Deep Learning in Face Synthesis: A Survey on Deepfakes”, 2020 IEEE 3rd International Conference on Computer and Communication Engineering Technology, 978-1-7281-8811-9/20, ©2020 IEEE, 2020.
S. Awah Buo, ‘‘The emerging threats of deepfake attacks and countermeasures,’’ 2020, arXiv:2012.07989.
Yogesh Patel, Sudeep Tanwar, Rajesh Gupta, Pronaya Bhattacharya, Innocent Ewean Davidson, Royi Nyameko, Srinivas Aluvala, And Vrince Vimal, “Deepfake Generation and Detection: Case Study and Challenges”, Digital Object Identifier 10.1109/ACCESS.2023.3342107.
Sami Alanazi, Seemal Asif, and Irene Moulitsas, “Examining the Societal Impact and Legislative Requirements of Deepfake Technology: A Comprehensive Study”, International Journal of Social Science and Humanity, Vol. 14, No. 2, 2024, DOI: 10.18178/ijssh.2024.14.2.1194, https://www.researchgate.net/publication/379615642, 2024.
Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales and Javier Ortega-Garcia, “DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection”, arXiv:2001.00179v3 [cs.CV] 18 Jun 2020.
Fakhar Abbas, Araz Taeihagh, “Unmasking deepfakes: A systematic review of deepfake detection and generation techniques using artificial intelligence”, Expert Systems With Applications 252 (2024) 124260, www.elsevier.com/locate/eswa, https://doi.org/10.1016/j.eswa.2024.124260, Accepted 14 May 2024, 2024.
Rachel Selva Dhanaraj, M Sridevi, “Face Warping Deepfake Detection and Localization in a Digital Video using Transfer Learning Approach”, Journal of Metaverse. 4 (1), 11-20. Doi: 10.57019/jmv.1338907, 2024.
Yogesh Patel, Sudeep Tanwar, Rajesh Gupta, Pronaya Bhattacharya, Innocent Ewean Davidson, Royi Nyameko, Srinivas Aluvala, And Vrince Vimal, “Deepfake Generation and Detection: Case Study and Challenges”, Digital Object Identifier 10.1109/ACCESS.2023.3342107, IEEE, Received 21 October 2023, accepted 4 December 2023, date of publication 12 December 2023.
Alexander Godulla, Christian P. Hoffmann, & Daniel Seibert, “Dealing with deepfakes – an interdisciplinary examination of the state of research and implications for communication studies”, https://doi.org/10.5771/2192-4007-2021-1-72, am 20.02.2022, 04:03:42, 2022.
Samuel Henrique Silva, Mazal Bethany, Alexis Megan Votto, Ian Henry Scarff, Nicole Beebe, Peyman Najafirad, “Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models”, Published by Elsevier B.V., https://doi.org/10.1016/j.fsisyn.2022.100217
Vurimi Veera Venkata Naga Sai Vamsi, Sukanya S. Shet, Sodum Sai Mohan Reddy, Sharon S. Rose, Sona R. Shetty, S. Sathvika, Supriya M. S., Sahana P. Shankar, “Deepfake detection in digital media forensics”, Publishing Services by Elsevier B.V., https://doi.org/10.1016/j.gltp.2022.04.017
Abdullah Ayub Khan, Yen-Lin Chen, Fahima Hajjej, Aftab Ahmed Shaikh, Jing Yang, Chin Soon Ku, Lip Yee Por, “Digital forensics for the socio-cyber world (DF-SCW): A novel framework for deepfake multimedia investigation on social media platforms”, Published by Elsevier BV, https://doi.org/10.1016/j.eij.2024.100502
Ruben Tolosana, Sergio Romero-Tapiador, Ruben Vera-Rodriguez, Ester Gonzalez-Sosa, Julian Fierrez, “DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation”, Published by Elsevier Ltd., https://doi.org/10.1016/j.engappai.2022.104673
Abdul Qadir, Rabbia Mahum, Mohammed A. El-Meligy, Adham E. Ragab, Abdulmalik AlSalman, Muhammad Awais, “An efficient deepfake video detection using robust deep learning”, Published by Elsevier Ltd., https://doi.org/10.1016/j.heliyon.2024.e25757
Ahmed Sedik, Osama S. Faragallah, Hala S. El-sayed, Ghada M. El-Banby, Fathi E. Abd El-Samie, Ashraf A. M. Khalaf, Walid El-Shafai, “An efficient cybersecurity framework for facial video forensics detection based on multimodal deep learning”, Neural Computing and Applications, Springer Nature 2021, https://doi.org/10.1007/s00521-021-06416-6
Zhiqing Guo, Gaobo Yang, Jiyou Chen, Xingming Sun, “Fake face detection via adaptive manipulation traces extraction network”, arXiv:2005.04945v2 [cs.CV] 16 Dec 2020.
Yuval Nirkin, Lior Wolf, Yosi Keller, and Tal Hassner, “DeepFake Detection Based on Discrepancies Between Faces and their Context”, arXiv:2008.12262v1 [cs.CV] 27 Aug 2020.
Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Dong Chen, Fang Wen, Baining Guo, “Identity-Driven DeepFake Detection”, arXiv:2012.03930v1 [cs.CV] 7 Dec 2020.
Li Zhang, Dezong Zhao, Chee Peng Lim, Houshyar Asadi, Haoqian Huang, Yonghong Yu, Rong Gao, “Video Deepfake classification using particle swarm optimization-based evolving ensemble models”, Published by Elsevier B.V., https://doi.org/10.1016/j.knosys.2024.11141
Daniel Xie, Prosenjit Chatterjee, Zhipeng Liu, Kaushik Roy, Edoh Kossi, “DeepFake Detection on Publicly Available Datasets using Modified AlexNet”, 2020 IEEE Symposium Series on Computational Intelligence (SSCI), 978-1-7281-2547-3/20/ ©2020 IEEE
Aminollah Khormali and Jiann-Shiun Yuan, “ADD: Attention-Based DeepFake Detection Approach”, Big Data Cognitive Computing 2021, 5, 49., https:// doi.org/10.3390/bdcc5040049, 2021.
Shahroz Tariq, Sangyup Lee, Simon S. Woo, “One Detector to Rule Them All Towards a General Deepfake Attack Detection Framework”, arXiv:2105.00187v1 [cs.CV] 1 May 2021, 2021.
Anubhav Jain, Pavel Korshunov, and S´ebastien Marcel, “Improving Generalization of Deepfake Detection by Training for Attribution”, 978-1-6654-3288-7/21/ 2021 IEEE, 2021.
Chih-Chung Hsu, Chia-Yen Lee, Yi-Xiu Zhuang, "Learning to Detect Fake Face Images in the Wild," 2018 International Symposium on Computer, Consumer and Control (IS3C), Taichung, Taiwan, 2018, pp. 388-391, doi: 10.1109/IS3C.2018.00104, 2018.
Nhu-Tai Do, In-Seop Na, Soo-Hyung Kim,“ Forensics Face Detection From GANs Using Convolutional Neural Network”, publication at: https://www.researchgate.net/-publication/327905310 , 2018.
Shilpa Pant, Chhaya Gosavi, Sheetal Barekar, “ Deep Fake Detection using LSTM and Survey of Deep Fake Creation Technologies”, International Journal of Intelligent Systems And Applications In Engineering (IJISE), ISSN:2147-6799, Submitted: 28/09/2023 Revised: 14/11/2023 Accepted: 26/11/2023, 2023.
Muhammad Mussadiq Rafiqee, Zahid Hussain Qaiser, Muhammad Fuzail, Naeem Aslam, and Muhammad Sajid Maqbool, “ Implementation of Efficient Deep Fake Detection Technique on Videos Dataset Using Deep Learning Method ”, Journal of Computing & Biomedical Informatics, https://doi.org/10.56979/501/2023, ISSN: 2710 – 1606, Volume 05 Issue 01, 2023.
Sreeraj Ramachandran, Aakash Varma Nadimpalli, Ajita Rattani, ” An Experimental Evaluation on Deepfake Detection using Deep Face Recognition”, arXiv:2110.01640v1 [cs.CV] 4 Oct 2021.
Ipek Ganiyusufoglu, L. Minh Ngˆo, Nedko Savov, Sezer Karaoglu, Theo Gevers, “Spatio-temporal Features for Generalized Detection of Deepfake Videos”, arXiv:2010.11844v1 [cs.CV] 22 Oct 2020, 2020.
Amal Naitali, Mohammed Ridouani, Fatima Salahdine and Naima Kaabouch, “Deepfake Attacks: Generation, Detection, Datasets, Challenges, and Research Directions”, Computers 2023, 12, 216., https://doi.org/10.3390/ computers12100216.
Joao C. Neves, Ruben Tolosana, Ruben Vera-Rodriguez, Vasco Lopes and Hugo Proenc¸”Real or Fake? Spoofing State-Of-The-Art Face Synthesis Detection Systems”, JOURNAL OF LATEX CLASS FILES, VOL. 13, NO. 9, MARCH 2016.
Xiangtao Meng, Li Wang, Shanqing Guo, Lei Ju, and Qingchuan Zhao, “AVA: Inconspicuous Attribute Variation-based Adversarial Attack bypassing DeepFake Detection”, arXiv:2312.08675v1 [cs.CV] 14 Dec 2023.
Zahid Akhtar, Murshida Rahman Mouree , Dipankar Dasgupta, “Utility of Deep Learning Features for Facial Attributes Manipulation Detection”, In 2020 IEEE International Conference on Humanized Computing and Communication with Artificial Intelligence (HCCAI) (pp. 55-60). IEEE.
Walczyna, Tomasz, and Zbigniew Piotrowski, "Quick Overview of Face Swap Deep Fakes", Applied Sciences 13, no. 11: 6711. https://doi.org/10.3390/app13116711, 2023.
Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, and Javier Ortega-Garcia, “An Introduction to Digital Face Manipulation”, Handbook of Digital Face Manipulation and Detection, Advances in Computer Vision and Pattern Recognition, 2022. https://doi.org/10.1007/978-3-030-87664-7_1.
Ana Pantelić and Ana Gavrovska, “From puppet-master creation to false detection”, PROCEEDINGS, IX INTERNATIONAL CONFERENCE IcETRAN, Novi Pazar, Serbia, 6 - 9. june 2022. ISBN 978-86-7466-930-3, 2022.
Ana Pantelić and Ana Gavrovska, “From puppet-master creation to false detection”, PROCEEDINGS, IX INTERNATIONAL CONFERENCE IcETRAN, Novi Pazar, Serbia, 6 - 9. june 2022., ISBN 978-86-7466-930-3.
Soumyya Kanti Datta, Shan Jia, Siwei Lyu, “EXPOSING LIP-SYNCING DEEPFAKES FROM MOUTH INCONSISTENCIES”, arXiv: 2401.10113v1 [cs.CV] 18 Jan 2024, 2024.
Jie Gao, Marco Micheletto, Giulia Orrù, Sara Concas, Xiaoyi Feng, Gian Luca Marcialis, Fabio Roli, “Texture and artifact decomposition for improving generalization in deep-learning-based deepfake detection”, Engineering Applications of Artificial Intelligence 133 (2024) 108450, https://doi.org/10.1016/j.engappai.2024.108450.
Falko Matern Christian Riess Marc Stamminger, “Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations”, 2019 IEEE Winter Applications of Computer Vision Workshops, 978-1-7281-1392-0/19/ ©2019 IEEE, DOI 10.1109/WACVW.2019.00020
Ashifur Rahman, Md. Mazharul Islam, Mohasina Jannat Moon, Tahera Tasnim, Nipo Siddique, Md. Shahiduzzaman, and Samsuddin Ahmed, “A Qualitative Survey on Deep Learning Based Deep Fake Video Creation and Detection Method”, Australian Journal of Engineering and Innovative Technology, https://doi.org/10.34104/ajeit.022.013026, 2022.
HaiweiWu, Jiantao Zhou, Shile Zhang, Jinyu Tian, “Exploring Spatial-Temporal Features for Deepfake Detection and Localization”, arXiv:2210.15872v1 [cs.CV] 28 Oct 2022.
Zhihao Gu, Yang Chen, Taiping Yao, Shouhong Ding, Jilin Li, Feiyue Huang, Lizhuang Ma, “Spatiotemporal Inconsistency Learning for DeepFake Video Detection”, arXiv:2109.01860v3 [cs.CV] 11 Oct 2021.
Jialiang Li, Haoyue Wang, Sheng Li, Zhenxing Qian, Xinpeng Zhang, Athanasios V. Vasilakos, “Are handcrafted filters helpful for attributing AI-generated images?”, ACM ISBN 979-8-4007-0686-8/24/10 https://doi.org/10.1145/3664647.3680945.
Ying Xu, Sule Yildirim Yayilgan, “When Handcrafted Features and Deep Features Meet Mismatched Training and Test Sets for Deepfake Detection”, arXiv:2209.13289v1 [cs.CV] 27 Sep 2022.
Gourab Naskar, Sk Mohiuddin, Samir Malakar, Erik Cuevas, Ram Sarkar, “Deepfake detection using deep feature stacking and meta-learning”, https://doi.org/10.1016/-j.heliyon.2024.e25933, Published by Elsevier Ltd
