Scaling the Future: Tackling the Challenges of Big Data and Analytics
Keywords:
Big Data, Edge Computing, Machine Learning, Scalability ChallengesAbstract
Big data and analytics are crucial for determining actions in diverse domains, including scientific research, business, and healthcare. Scalability, however, becomes a major impediment in efficiently managing and analyzing these vast databases as data volumes proliferate at an unprecedented rate. The pace at which the data is generated, the complexity and diversity of the data types involved, and the sheer volume of the accumulated data are all factors that affect scalability.
We also touched on the increasing demand for scalable solutions to manage unstructured and diverse data types, such as text, photos, video, and sensor data.
The scalability issues in big data and analytics are investigated in this study, with particular emphasis given to vital aspects such as data processing, storage, management, and real-time analysis. We had additionally examined the most recent advances in this area and suggested alternative approaches using edge computing, cloud computing, distributed systems, and artificial intelligence (AI). In this paper, we examined the scalability issues that contemporary analytics platforms and big data systems confront, as well as the methods and tools that have recently been used to solve these challenges. Additionally, this study highlights fresh strategies that provide an avenue forward for resolving big data scalability issues.
Downloads
References
https://www.i-scoop.eu/big-data-action-value-context/#google_vignette
S. Ghemawat, H. Gobioff, and S.-T. Leung, "The Google File System," ACM SIGOPS Operating Systems Review, vol. 37, no. 5, pp. 29-43, 2003.
R. Ranjan, and A. D. P. Gupta, "Cloud Computing for Big Data Processing: A Survey," IEEE Transactions on Cloud Computing, vol. 7, no. 1, pp. 22-39, Jan.-Mar. 2019.
M. Zaharia et al., "Spark: Cluster Computing with Working Sets," in Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, 2010.
B. White et al., "Hadoop: The Definitive Guide," O'Reilly Media, 2015.
M. T. Ozsu and P. Valduriez, Principles of Distributed Database Systems, Springer, 2011.
D. R. Koutsonikolas, C. V. Trimmel, and M. H. C. J. Rittman, "Real-time Stream Processing for Big Data Analytics: A Survey," Journal of Big Data, vol. 2, no. 1, p. 13, 2015.
M. P. Papazoglou, D. Georgakopoulos, and M. D. Mokbel, "Cloud Computing and Big Data," Springer, 2016.
F. Bonomi, R. Milito, P. Natarajan, and J. Zhang, "Edge Computing: A Survey and Research Directions," Proceedings of the 2012 ACM/IEEE 5th International Conference on Future Internet of Things and Cloud, 2012.
https://jelvix.com/blog/what-is-edge-computing
L. Wang, "Big Data Analytics and Machine Learning: A Survey," IEEE Access, vol. 8, pp. 24053-24072, 2020.
D. McMahan et al., "Federated Learning of Deep Networks using Model Averaging," in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017.
