![]() In consultation with our customer, we decided a proof of concept (PoC) with specific success criteria would be the next step. Our variables were data types (structured, unstructured), data scale, and frequency (volume, variety, and velocity of data). To build a production migration strategy, it was important for us to perform qualitative and quantitative analysis using variables. This challenge was aggravated by queries related to business operations that tripled the execution time. It also led to a six-fold increase in the user load.Īs a result, the customer’s hourly ETL jobs took more time to execute than the SLA for the business. The need for real-time data increased the workload on the cluster by about 100x, mostly from complex operations. In some cases, they used machine learning (ML) to generate predictive outcomes. They needed the data for reporting, historical data, and periodic analysis. It gathered information from different systems, including marketing, sales, supply chain, production, manufacturing, and human resources.ĭuring the last few years before they began their migration, the customer’s business units had to access the data in real-time. Our customer’s data warehouse served as a central source of information and data analytics platform for the entire company. How the Customer Used Oracle Data Warehouse In this post, we explain how we selected the optimal migration process for the customer’s needs, and how we carried it out. We also tripled the performance for dependent business processes and reports. The result of our efforts was a 30 percent lower total cost of ownership (TCO) for the customer’s data platform on AWS. Mactores is an AWS Partner Network (APN) Advanced Consulting Partner with the AWS Data & Analytics Competency. Mactores Cognition worked with Amazon Web Services (AWS) to migrate the customer from Oracle to their MPP data platform. By alleviating ETL performance challenges and improving query execution time, they hoped an MPP data platform would result in faster business operations. ![]() And, to control the ETL process, they had to continuously purchase hardware every quarter.īecause of these factors, the customer decided to migrate to a cloud-based Massively Parallel Processing (MPP) data platform. To accommodate the volume of incremental data, the data warehouse team had no choice but to breach the ETL service-level agreement (SLA) defined by the business. To sustain the size of their operations, they had to scale their infrastructure up to 1,500 cores and 20 TB. The company’s operations included extract, transfer, and load (ETL) operations and business processes. Over the next 10 years, the application portfolio scaled, bringing the data volumes up to 512 TB, supporting more than 5,000 business processes. By Nandan Umarji, COO and VP of Engineering at Mactores Cognitionīy Anil Patel, Data Engineer at Mactores Cognitionīy Sanjay Nighojkar, Manager, Partner Solutions Architect at AWSĪ large manufacturing company faced many challenges with their Oracle data warehouse, which was deployed on-premises in 2009 with about 120 GB of data.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |