In the old days, “data migration” was a straightforward task. If you wanted to go from four-drawer gray file cabinets to five-draw tan ones, you simply lifted the folders from the old cabinets and dropped them into the new ones. Unfortunately, some people view the migration of computer data in a similar manner, when in truth it is a much more complicated process. Adhering to the following best practices and strategies can facilitate the process and increase your chances of a successful migration.
- Define the reasons your data exists. Why is the data important? Who needs the data? Is it likely that the data will be used in the future for a different purpose, such as business analytics or metrics? Once you know why precisely why you are collecting and storing data, you can formulate a better plan for the migration.
- Assess the current state of your data. How dirty is it? How clean does it need to be after migration? What can be done to keep your data clean in the future? Establish the business rules and standards you will use to scrub your data and keep it clean in the future.
- Examine your current business rules for data and redefine them if necessary.Your data must be compatible and in compliance with your validation rules. Regulatory compliance may also be an issue. It may be necessary to apply one set of rules for the migration and then switch to a more complex set for the data after migration. This is all part of preparing for migration and managing your data afterwards.
- Establish your team’s responsibilities and the governing rules early in the process. Who has the final authority over data? Who has the authority to determine whether the migration has been successfully completed? Who is responsible for validating the data after migration? Without a clear definition of responsibilities and rules, your team can quickly begin to struggle, increasing the risk of a delayed or failed migration.
- Determine the schedule for the migration. Once you have determined the rules and areas of responsibility, prepare a schedule that shows start dates and completion dates. If necessary, include dates for progress checks or milestones.
- Use a data migration management tool, but do not rely only on the tool. Remember that a tool, by definition, is an instrument used to make manual work easier or complete an operation. Leverage the tool, but tailor it to match your enterprise’s specific needs and business rules. Furthermore, do not assume that the data migrated has not been corrupted just because you used a tool. You will still need to validate it.
- Validate data periodically throughout the migration. Waiting until all data has been migrated to validate it is a potentially expensive mistake. Schedule testing and validation procedures throughout the entire process. This makes it faster and less expensive to correct issues as they can be addressed as they occur.
- Be aware of the potential for security leaks. When migrating data, it is easy to overlook security settings or permissions. If security gets left behind, your data is vulnerable to misuse or corruption.
- Schedule in some “Murphy” time. There are many variations of “Murphy’s Law,” but the one that seems most appropriate is, “Anything that can go wrong, will go wrong — and at the worst possible time.” Allow time in your schedule — especially for the most critical aspects — to address problems, revalidate data or learn what went wrong.
Data migrations are seldom as pleasant as a springtime walk in the park, but by following these best practices, your task should be easier. Although migrations are usually divided into “extract, transfer and load,” a better approach might be “evaluate, plan, prepare, extract, test, transfer, validate, load, validate.”