A migration to AWS may involve migrating tens or hundreds of applications from on-premise to AWS. Migrating such number of applications needs to follow a process where we can plan start to finish, for a successful outcome.
In today’s blog, I am going to focus on the end of the process, the actual migration, and specifically on an AWS Solution released recently, the AWS CloudEndure Migration Factory Solution.
_AWS CloudEndure Migration Factory Solution is an AWS Solutions Implementation that helps migrate a large number of servers with CloudEndure in a simplified and expedited way at scale. The solution automates many of the manual, time-consuming tasks that enterprises commonly face in migrating servers from on-premise to the cloud; for example, checking prerequisites on the source machine, installing/uninstalling software on the source and target machine. _
CloudEndure is an agent-based solution that helps migrate Windows and Linux applications to AWS by continuously replicating the source machines into AWS, drastically reducing cut over times and manual tasks. This takes place in the background, without application disruption or performance impact, which makes it easy to deploy in production environments without worrying about impacting our application users. CloudEndure was acquired by AWS in January 2019 and provides Cloud Migration at no cost. To see CloudEndure in action you can watch a demo I recently did for the AWS Melbourne User Group.
The CloudEndure Migration Factory Solution from AWS takes CloudEndure at its core and goes one step further in the process of simplification through automation of many repetitive, manual tasks when facing large scale migrations.
Three migration phases
This AWS Solution comes with prescriptive guidance on how to use the Migration Factory, divided in three phases: Pre-migration, migration execution and post-migration.
The pre-migration phase is all about setting up the environment, deploying the CloudEndure Migration Factory Solution and setting up a CloudEndure account. AWS provides a detailed guide to deploy the CloudEndure Migration Factory.
The migration execution phase is all about running through a set of steps (more on these later) for each of the migration waves. A migration wave consists of a group of servers that are migrated at the same time. Dividing large scale migrations in waves helps a team focus on a smaller number of servers at any given time for troubleshooting and verification, and several waves can run in parallel.
The post-migration phase may vary from project to project. Some of the most usual tasks will be decommissioning source servers that are no longer required and begin rearchitecting the applications with the goal of implementing cloud native patterns by leveraging AWS services like Amazon EC2 Autoscaling, Amazon Relational Database Service, and many more to free the teams from undifferentiated tasks and let them focus on building new products and features.
The migration execution phase
Coming back to the migration execution phase, this is divided in 5 main steps that we will repeat for each wave:
- Import wave data into the CloudEndure Migration Factory
- Prepare the source servers
- Verify server replication status
- Perform a test migration
- Cut over to AWS
The CloudEndure Migration Factory helps with automating a lot of these steps, reducing or eliminating time-consuming manual tasks.
Import wave data into the CloudEndure Migration Factory
In this step we prepare all server and application data into a csv file. This file will have information about the application name, operating system, wave it belongs for the migration execution, and more. This data will come from previous analysis and portfolio tools like TSO Logic and AWS Application Discovery Service and provides the necessary information to identify the source servers and applications and define the target state post migration to AWS.
The CloudEndure Migration Factory provides automation scripts to import the CSV file(s) with the server information into the solution, avoiding manual entry through the CloudEndure Console for each server
The attributes for each server are as follows:
- wave_id: A unique wave id. All servers that need to be migrated on the same cut over date will have the same wave id
- app_name: The name of the application
- cloudendure_projectname: CloudEndure migrations are organised in Projects. One or more projects will be created for this migration
- aws_accountid: the 12-digit AWS account id where this server will be migrated to
- server_name: The name of the server on premises before migration
- server_os: The Operating system (Windows, Linux) of the server on premises before migration
- server_os_version: The Operating system version of the server on premises before migration
- server_fqdn: The Fully qualified domain name (FQDN) of the server on premises before migration
- server_tier: The type of server, such as web, application, or database server
- server_environment: The environment of the server, such as development, test, production, etc
- subnet_IDs: The AWS Subnet ids for provisioning the server once migrated to AWS in the target AWS account
- securitygroup_IDs: The AWS Security Group ids for provisioning the server once migrated to AWS in the target AWS account
- subnet_IDs_test: The AWS Subnet ids for provisioning the server once migrated to AWS in the target AWS account for test mode migration
- securitygroup_IDs_test: The AWS Security Group ids for provisioning the server once migrated to AWS in the target AWS account for test mode migration
- instanceType: The Amazon EC2 instance type and size to use for provisioning the server once migrated to AWS in the target AWS account
- Tenancy: The AMAZON EC2 tenancy (default, dedicated, host) to use for provisioning the server once migrated to AWS in the target AWS account.
Prepare the source servers
This step focuses on ensuring the source servers have the required network connectivity to CloudEndure Replication Server, enough disk space is available, and the appropriate packages are installed for the agent to run.
The CloudEndure Migration Factory again provides automation scripts to verify all requirements are met for all Windows and Linux servers imported in the previous step, avoiding manual verification per server, which can be an error prone task.
Once all servers have passed the verification the CloudEndure Migration Factory provides another automation script that can push the CloudEndure agent (responsible for continuous replication of the source servers to the staging area) to all servers, injecting the appropriate tokens for authentication with CloudEndure and establishing the relationship with the appropriate CloudEndure project.
The last preparation step for the source servers will be to push the post-launch script or set of scripts. Post-launch scripts are executed by CloudEndure in the target EC2 instance once the server has been migrated, and help automating tasks such as removing no longer necessary software, and installing new software specifically required for running this application in AWS.
Once again, the CloudEndure Migration Factory provides scripts to push these post-launch scripts to the source servers. These post-launch scripts can be tested once we perform the migration in test mode, prior to the final cut over.
Verify server replication status
This step is responsible for verifying the status of the server replication. Once the CloudEndure agent is deployed to all source servers the replication to the staging area begins. This initial replication can take some time depending on the number of servers and the size of the volumes to replicate.
The CloudEndure console provides a view on initial replication progress but verifying this status across various projects and waves can be time consuming and require manual checklists. The CloudEndure Migration Factory provides automated scripts to monitor the replication status and report back when all servers are in Continuous Data Replication status.
Perform a test migration
Before progressing with the final cut over we want to test the migration, ensure the blueprint for all servers works, software is removed and installed as required once EC2 instances are deployed in the target environment and that applications performed as expected.
The CloudEndure Migration Factory helps us with launching test migrations across many servers and verify all EC2 instances boot up correctly. All this without requiring us to manually go into the CloudEndure and AWS Consoles and manually check progress and status for many servers.
Once we have run a test and identified any issues that require fixes like reviewing a post-launch script or changing an egress Security Group rule, we can apply the change and rerun the test quickly and with certainty the steps will be taken in the same order and with the same configuration we provided in the previous test, plus the applied fix.
Cut over to AWS
Cut over is the final migration step, and at the end of a successful cut over the application will be running in AWS. Previous tests give us certainty the migration will perform as expected.
The CloudEndure Migration Factory helps us verifying that all source servers are in Continuous Data Replication state, meaning there is no lag and we have a copy in the staging area of the latest state of the source servers, and it does so with a script that checks across all servers within a wave.
Once all servers are in Continuous Data Replication state, we shut down the source servers to avoid the changes that would occur in the source if transactions kept being processed. This can be automated via the CloudEndure Migration Factory, shutting down all servers with a single script and gaining critical time during the cut over window.
With all source servers shut down it is time to launch the cut over migration, which provisions EC2 instances from the data in the staging area and executes the post-launch scripts to remove and install software as tested during the previous step. The CloudEndure Migration Factory will assist again with confirming that all EC2 instances have booted up correctly.
At this point the application or applications in the wave have been migrated and final configurations including DNS resolution, networking can be done, before performing the final tests on the applications to verify that they behave as expected and can start serving real users.
Summary
The CloudEndure Migration Factory helps simplify and accelerate migrations to AWS leveraging CloudEndure. It provides automated mechanisms to perform the necessary steps from start to finish for a successful migration.
The CloudEndure Migration Factory Solution can be reviewed and deployed from the AWS Solutions Library. AWS APN Partners like AC3 can help customers prepare and migrate their workloads to AWS leveraging solutions like the CloudEndure Migration Factory and others like TSO Logic, the AWS Application Discovery Service and the AWS Database Migration Service in a secure and consistent manner to accelerate your move to AWS.
This article was written by Jesus Rodriguez, Cloud Solutions Architect at AC3. Jesus has over 8 years of experience in IT, Certified AWS Solutions Architect Professional, Certified AWS DevOps Professional, and Certified AWS Big Data - Specialty, with a strong focus on implementing, deploying and provisioning secure, high available, scalable and cost optimized applications utilizing the best AWS services for each use case.