Rapid Migration Services

The need for Rapid Migration

The migration of data from a legacy PACS to a new system can be a time-consuming process when conventional data extraction methods are employed. While PACS data can in principle be extracted from the legacy system through a DICOM query/retrieve interface, or pushed out from the legacy system through a DICOM export interface, legacy PACS archives rarely possess the power or the optimized tools to perform this process rapidly. This is especially the case in the period prior to “go-live” of the replacement system, when the legacy PACS must also support clinical operations. As a result, data migration projects commonly stretch out many months to more than a year, prolonging an already-painful transition period for the customer, lengthening the sales cycle for the replacement PACS vendor, and adding costs for both.

Achieving the highest data transfer rates generally necessitates circumventing the central processor of the legacy PACS and transferring data directly. Such direct transfers begin by reading off-line storage media. In cases where image data is stored in SAN or Network Attached Storage systems, raw image data may be extracted directly from these storage systems by network file access to their host computer. Image data thus read from the storage media must then be transformed and updated as required by information in the legacy PACS database, and delivered to the replacement PACS in multiple DICOM streams.

Migratek offers fast data migration services by employing its dicomBLAST™ Migration Appliance (“the DMA”). The DMA is a group of 2-6 computer processors and peripherals assembled from modular software and hardware components and configured for each migration job. The DMA can be configured to support data transfer rates exceeding one terabyte per day. At this rate, a mid-sized department with 20 Terabytes of archival data could be migrated in 20 days, rather than the 200-400 days required using “standard” data migration methods. Migratek has demonstrated in practice the transfer of over 45,000 DICOM Studies per day.

How it works

It should be no surprise that such transfer rates are possible, even when reading off-line storage media. The common 5.25 inch MO disks can be read at transfer rates ranging from about 3 to 6 megabytes per second, depending on drive manufacturer and disk capacity. Assuming 3 megabytes per second and allowing 30 seconds per disk change, each drive can transfer 10.5 gigabytes per hour, or 250 gigabytes in 24 hours. An Appliance with four drives could thus transfer one terabyte in a 24-hour day. The DMA can stream disks or tapes into media image files that are interpreted as files in subsequent processing within the DMA, allowing the drive’s full speed to be utilized without slowdowns from file system overhead. Also, one terabyte per day, equaling 93 megabits per second, is well within the capability of today’s networks.

In many cases, manual media handling will be required to feed the input drives a continuous stream of media. Migratek can provide this labor or work with PACS vendor or hospital personnel to perform these functions. The media may be manually loaded by the staff, or in some cases a robotic library may be used to aid in media handling. Where duplicate offline backup media are available, there will be no conflict with the operating PACS. Often, however, sole copies of optical disk media are kept in near-line robotic libraries in the legacy PACS. In such cases, the staff will need to use legacy PACS functions to temporarily eject media, and then re-import them after they have been read by the DMA.

The organization and use of the DMA is illustrated in Figure 1. Media Readers input image data files from optical disks, magnetic tape or network-connected magnetic disk in the legacy PACS. Stream Processors modify and format the individual image information objects using information extracted from the legacy PACS database, and forward the resulting objects to DICOM Sender processes that transmit them to DICOM inputs on the new PACS. Objects for which there are errors are stored in the Exception Repository. The DMA architecture allows variable degrees of parallel processing at each stage, distributed over a number of host machines within the DMA. The basic export mechanism for the DMA is through multiple DICOM Storage Class streams. In most cases the new PACS will be able to handle sufficiently high speed input through its standard DICOM interfaces. Migratek has tested target archives able to support sustained input rates of 1 terabyte per day. If needed, special output interfaces for faster bulk loading into the replacement PACS may be developed on a custom basis.

The DMA takes its processing instructions from a Stream Processor Instruction File (SPIF), which contains all the transformation and modification rules for each migration job. The SPIF is generated from legacy PACS database extracts by the Source Extract Compiler, which is customized for each migration job.

Figure 1: How it works. The migration appliance combines database extracts and directly-ready disk or tape storage media in parallel streams without loading the legacy PACS computers or network.

The DMA transfers all DICOM composite objects, updated with order numbers and demographic data coercions or updates stored in the legacy PACS database. Other information, such as Window/Level settings, proprietary annotations, custom annotations and reports, may also be transferred if the information is available and the customer deems their additional costs to be justified. As in all complex real-world settings, exception conditions are expected in handling the legacy PACS data. These include missing or corrupt data, obsolete or non-DICOM forms, and inconsistent use of identifiers in the stored image data. Many of these can be easily fixed, but others may require costs that could exceed their value to the customer. Migratek will provide estimates and options to enable the customer to make the most practical choices. The DMA for a recent project migrating from DLT tapes is pictured at right. This configuration has six processors and two DLT320 drives (one with an autoloader).

System Requirements for Rapid Migration

The DMA can be used for all legacy PACS that store images in files, or in data packages decomposable to files (such as tar or zip archives), provided that stored images have sufficient identifying information to link them to legacy PACS database entries. This condition will be satisfied if the images are stored a DICOM format, which has unique instance identifiers in header fields, or if the image file name and volume number are sufficient to link the image data to its entry in the database of the legacy PACS. Access to the legacy PACS database information is usually required. This information may be extracted by execution of SQL scripts, or from flat files provided by the source PACS vendor, or by using a Migratek tool that queries the legacy PACS DICOM Query (C-FIND) interface.

The economics of custom engineering for a range of source systems have discouraged others from the Rapid Migration approach. This is particularly true for migration service groups of PACS vendors, who must serve all source PACS systems and only their own target PACS systems. In contrast, a neutral third party service provider can spread the engineering investment in each source system over a number of target systems. Also, the independent migration vendor is able to target R&D resources on specific - generally the most numerous - source PACS systems, without the requirement to serve all from the beginning.

Migratek Rapid Data Migration services combine know-how, technology and people to transfer legacy PACS data rapidly with minimum disruption of clinical operations. The short period of data transfer dramatically shortens installation project timelines, offering value and cost savings for the vendor and customer alike.