AUTOMATION OF THE INFRASTRUCTURE FOR CONTROLEING THE BIG DATA TRANSFER PROCESS

Abstract

The aim of this work is to explore the main challenges associated with big data migration and the continuous integration process. As a result, an automated infrastructure was developed that manages resources through code. This approach ensures traceability, simplifies replication and updates, and increases overall efficiency. Automation reduces the likelihood of human error, enabling fast and reliable infrastructure changes.

Containerization and the use of Kubernetes provide load balancing, automatic scaling, and service stability. These technologies also streamline the management and scaling of services in distributed environments.

The outcome of this work includes an improvement in service quality and a reduction in maintenance time, enabling the creation of reliable, scalable, and efficient systems that meet modern technological demands.

The implementation of data collection, automation, and infrastructure management processes not only simplifies the handling of complex systems but also opens up new opportunities in the technology sector.