PACS

What is PACS 3.0 and why is it the next thing in Enterprise Medical Imaging?

I recently wrote a rather lengthy white paper on the subject of PACS 3.0.  It was published as a three-part series on AuntMinnie.com.  I strongly urge anyone considering a radiology PACS replacement to read the series.

In Part 1 of this series, I share some of the problems and “myths” associated with the conventional, current generation of radiology PACS, referred to in the series as R-PACS 2.0.

In Part 2 of this series, I focus on the impact of vendor neutral archive and universal viewers (UniViewers) on the current radiology PACS paradigm and examine the opportunities of embracing these technological advancements.

In Part 3 of this series, I describe the PACS 3.0 concept and explore the benefits of making the switch to this new PACS paradigm.

At present, the PACS 3.0 market is clearly in the “innovators” stage of Geoffrey Moore’s technology adoption curve, but it’s important to gain an understanding of the concepts covered in this series if you are preparing for a change of PACS.  A move to the PACS 3.0 construct requires a “best-of-class” mentality, but a best-of-class approach to PACS should no longer be a cautionary strategy for a healthcare organization, especially those with specialized imaging departments, because many traditional R-PACS 2.0 configurations have already become best-of-class due to the various third-party applications that are currently required to meet the stated requirements.  The end of the R-PACS 2.0 paradigm has begun.

 

Current Generation of Radiology PACS is Ending

Radiology PACS is changing.  Something of a paradigm shift is occurring; from a current generation radiology PACS supporting interfaces to multiple third-party specialty applications, to a Vendor Neutral Archive (VNA) configured with multiple specialty display and workflow application plug-ins. In the former case, the PACS application is at the center of data management, and the PACS vendor continues to own the data.  In the later case, the VNA is at the center of data management, and the healthcare organization owns the data.

Slide1Ten years ago when the current generation of radiology PACS (termed R-PACS 2.0) was first introduced, the core PACS application provided all of the features and functions that a radiology department needed. This single-source, turnkey package included the tools for image acquisition and QC, both the diagnostic and the clinical display applications, and the worklist and workflow applications. The system focused almost exclusively on whatever was required to acquire, display and interpret radiology image data.  DICOM and HL7 were the ubiquitous data exchange interfaces.  The radiology PACS was a standalone system and data exchange with any other system was not considered a requirement (much to the delight of the vendors).

Slide2Over the ensuing ten years, most of the current generation PACS fell further and further behind in meeting the ever-changing needs of the evolving radiology department.  Unable to meet the requirements with in-house development, the easiest solution for many PACS vendors was to simply bolt on to their PACS the third-party applications that could meet those new requirements.  Examples include specialty processing applications for Nuclear Medicine, Digital Breast Imaging, 3D, and Fusion, as well as Discrepancy Reporting, Peer Review, and Analytics.  The latest zero-client, server-side rendering clinical viewers that so many organizations wanted to use to image-enable their EMR systems also became a bolt-on to the PACS.  The department PACS was no longer a single source solution.  Of necessity it had become a best of breed, multi-vendor solution.

Unfortunately the ten-year old current generation PACS is not even close to best-of-breed.  [1] A fat display client that requires a full pixel set download to the display platform is old technology and incapable of meeting performance expectations.  [2] The dependence on DICOM as both a data format and a transfer interface is severely limiting.  [3] The requirement that any third-party application be bound to the idiosyncrasies of the PACS is simply inefficient.  [4] The inability to support a dual-sited, fully mirrored configuration means there is no Business Continuity.   For these and many other reasons, the era of R-PACS 2.0 is coming to an end.

The PACS paradigm has shifted Slide3to PACS 3.0, which is definitely a best-of-breed ensemble of [1] VNA, [2] various diagnostic and clinical display applications, and [3] an enterprise workflow/worklist application, all of which simply plug into the neutral VNA.

A Best-Of-Breed solution implies a more difficult support paradigm, and that rightfully suggests risk.  However one must consider the fact that the R-PACS 2.0 solution that requires multiple third-party applications effectively makes those systems a multi-part solution as well…and most of the PACS vendors are not going to include those third-party apps in their SLA or their system monitoring solution.  It seems to me that meeting all of the requirements of today’s radiology department virtually demands a best-of breed solution.  Since most of the PACS vendors have clearly stated that their service support packages do not cover many of those third-party applications, these hybrid R-PACS 2.0 solutions present some degree of risk as well.

The greater risk however is the fact that these R-PACS 2.0 solutions will most likely have to substantially evolve in the next few years in order to meet the market challenges being presented by those companies whose display applications already feature the more advanced server-side rendering display technology and true BC configurations.  Such a dramatic overhaul to the R-PACS 2.0 architecture will most likely cost significant upgrade dollars…costs that are unlikely to be covered by the yearly software maintenance fees.  Unfortunately, if the PACS vendor that the healthcare system chooses to partner with does not make these dramatic upgrades, that organization will effectively be saddled with old technology for many years to come.

I invite the reader to learn more about the paradigm shift from R-PACS 2.0 to PACS 3.0 by reading my latest White Paper on the subject.  Aunt Minnie is publishing the paper as a three-part series, with the first part appearing on September 25, 2014.  Part 1 can be found at http://www.auntminnie.com/index.aspx?sec=sup&sub=pac&pag=dis&ItemID=108618

 

Today’s Radiology PACS – Shiny But Not New

It may seem a little late for commenting on observations from RSNA 2013, but then nothing has changed in the Radiology PACS market since then, so I think observations from three months ago are still valid.  Come to think of it, nothing really significant seems to have changed in Radiology PACS for several years now.  Pesky little things like bug fixes get attention each year, and functions that should have been standard years ago have finally shown up, but there have been very few major changes in system architecture.  Some semblance of Disaster Recovery has always been there, but a true Business Continuity configuration is still a reach for many of the PACS vendors.

Lots of radiology PACS on the market today are old.

havana3Interesting to see some of the PACS vendors developing a clinical display application based on server-side rendering and a zero (or at least near-zero) client. Why is the diagnostic application suite still a beefy client that requires delivery of the full lossless dataset to the remote display platform?  While the EMR user can access and view data on their Windows or Mac laptop, mobile tablet or phone, why is the radiologist still limited to a glorified PC?  Clinical image distribution and display has historically played second fiddle to the diagnostic application suite.  Why this sudden shift in focus from diagnostic to clinical?   Could it be that image enabling the EMR is the “new thing”, or simply the easier and less expensive engineering effort?  In the absence of meaningful change in system architecture and diagnostic display technology, perhaps the thinking is that a shiny new clinical viewer will serve as a differentiator among a handful of radiology PACS solutions that are otherwise old, and effectively the same core systems they were nearly ten years ago.

From my perspective, it seems that today’s Radiology PACS market is a zero sum game.  While some vendors with old technology are clearly losing market share, vendors with even “decent, old PACS” tend to lose as many existing customers as they gain new customers each year.  At 95+% market penetration, it’s a replacement market, where vendors attempt to take market share from each other.  Price pressure takes its toll on system sales revenue, so the real money is in service contracts.  But once again, if there is no net increase in customer base from year to year, that revenue stream is not growing. Only so much R&D funding is available each year, and apparently there is not enough revenue to fund “the next big thing” for Radiology PACS.   When revenues are flat or falling, profits also fall unless costs are reduced.  In addition to cutting R&D, that typically means vendors cut costs by eliminating staff through layoffs, or euphemistically “rightsizing”, “reorganizing”, spinning off or shutting down under-performing parts of their businesses.  When the Radiology PACS market went cold, the size of Radiology PACS companies shrunk.

In addition to standing pat with their existing technology, the Radiology PACS vendors have fallen behind in enhancing their solutions with those features and functions that radiologists and radiology departments need today to provide good patient care and stay competitive in their market. By and large, today’s Radiology PACS solutions do not support advance Breast Imaging packages to cover Full Field  Digital Mammography and Breast Tomosynthesis, as well as the ability to display multi-modality breast imaging presentations.  They do not include advanced diagnostic Nuclear Medicine packages to cover all the variations on mixed modality image fusion.  There is no advanced worklist functionality that can escalate studies on the read list to meet TAT requirements, no ED preliminary findings and discrepancy reporting, no call functionality and follow-up.  3D is basic if available at all, and remote access is typically poor. Today’s Radiology PACS frequently have no business analytics or data mining that would enable the department to discover and monitor the drivers of their business.

All of these highly desirable, if not necessary, features and functions are third party plugs-ins developed by smaller, more nimble and innovative vendors.  In too many cases, however, the interfaces required to plug these tools into the core Radiology PACS are not yet developed for a specific combination of vendor solutions.  It’s as though cooperating with the third party vendors to expand the core PACS through the addition of these plug-ins would be seen as an open admission that the PACS vendor has fallen behind.

Based on last year’s RSNA observations, it’s time for the PACS vendors to admit their system deficiencies, and start working on the solutions.  If keeping up with current requirements through in-house development is not feasible, then the PACS vendor must embrace the concept of third party plug-ins and get on with the development of the interfaces.  They should think of this strategy as a business necessity.  The VNA is steadily pulling image management from the department PACS.  Advanced clinical viewers that address both DICOM and non-DICOM objects will easily outperform those new PACS clinical viewers.  If a PACS vendor can’t keep up with all of the new radiology department application requirements, it will be a struggle to sell any new systems a few years from now, and that is the beginning of the end for the installed base.

Havana2As I departed Chicago last December, one strong image came to mind.  Pick any upscale neighborhood in Havana and you’re likely to see shiny cars parked at the curb.  The interesting thing is, none of these are new cars, they’re shiny old cars.  Some have new wheels, but they’re still old cars.  Today’s Radiology PACS conjured up images of Havana streets…shiny cars that are not really new, with underlying layers of faded paint covered in super-high-gloss wax.

Upgrading Department PACS with Entry-Level Vendor Neutral Archive Components

Organizations that appreciate the benefits that the Vendor Neutral Archive (VNA) will bring to the organization, sometimes find it difficult to finance the complete VNA solution in a single budget cycle. It is most unfortunate if the decision comes down to deploying none of the solution, due to a lack of funding for all of the solution. In my opinion, continued investment in a proprietary Picture Archiving and Communications System (PACS) archive and the continued accumulation of proprietary data is a flawed strategy.  In a paper sponsored by an unrestricted grant from IBM Systems and Technology I propose an alternative to the full VNA deployment, and discuss an affordable entry-level phase to a multi-phase VNA deployment strategy.

Any IT initiative that is focused on an upgrade to a department PACS storage solution, a refresh of the existing storage solution or a replacement of the disaster recovery (DR) storage solution initiated by an end of life letter, is an opportunity to carefully consider the value to the organization in continuing the proprietary PACS archive paradigm. Migrating the image data from the PACS to a properly configured entry-level VNA is the organization’s opportunity to once and for all normalize its image data and end the cycle of expensive and time-consuming data migrations that are required with every PACS replacement project. The hardware agnostic, entry-level VNA gives the organization the opportunity to apply advanced storage technology to all of its data management applications. All of these benefits can be realized at a fraction of the cost of a fully configured VNA, and in many instances for a price very close to that being quoted for the proprietary PACS archive solution.

Searching for that Next Generation PACS

I made the annual trek to Chicago this year, as usual right after Thanksgiving Day, to participate in the Radiological Society of North America meeting.  I don’t attend RSNA.  I “participate” in RSNA.  It’s like that adage that goes something like this “There are people who play the game, while there are people merely watching the game being played, and then there are those people who have no idea that a game is being played”.

I’m bringing up the adage, because I have seen it written and heard it said numerous times since the meeting that “There was nothing really new at this year’s RSNA”.  Anyone who actually holds that opinion must fall into that third group of people who have no idea that there is a game being played.

I had a number of reasons for checking out the current state of Radiology PACS.  I have clients with newly installed Vendor Neutral Archives that are now looking to replace their current PACS with a department-focused PACS that is a better fit with the VNA.  I had also recently read a very thought provoking article written by Lisa Fratt and published on line by Health Imaging on Nov 2, 2012, in which Lisa quoted lengthy comments on the state of Radiology PACS by Drs. Chang, Dreyer and Siegel.  I highly recommend reading that article, because my take away was that most of the current generation Radiology PACS, especially those in the larger booths, are not meeting the new list of expectations.

I think one of the problems that PACS vendors are having is investing development dollars at the same time they are being asked to reduce the sales price.  Today’s enlightened customer is asking for new features like multi-facility and multi-PACS workflow, powerful business analytics, and EMR integration to be packaged into a department PACS that should simply focus on what goes on inside the Radiology department.  A customer that has invested in a VNA and has already “image-enabled” the EMR with a universal viewer doesn’t need the department PACS to manage the data for its lifecycle, or provide the clinical viewer for the referring physicians.  We want the PACS vendors to add in the new workflow and analytics goodies, but reduce the overall price because of the removal of those applications that are no longer needed.  Apparently the call for a price reduction is falling on deaf ears.

Unfortunately that is the growing expectation among healthcare organizations that have invested in VNA and Universal Viewer solutions.  A growing number of providers believes that a department focused PACS is supposed to be significantly less expensive than the stand-alone, do-it-all model, that Dr. Chang refers to in the article as PACS 2.0.

Not only was I disappointed to learn after two days of RSNA booth visits that a department-focused PACS was merely 10% or so less expensive than the stand-alone model, but that the version of the PACS being demoed in the booths still does not include all the new goodies that are required.  That reminded me of yet another adage that was recently attributed to the new smaller packaging we see on the grocery shelves.  “Statements on the label such as New and Improved, really mean Smaller but just as Expensive.”

Based on what I saw at RSNA 2012, the major PACS vendors are failing to keep up with current market requirements.  So where is that next generation PACS, the one that sticks to the knitting in the department, includes the new Workflow, Analytics and EMR integration requirements, interfaces to the VNA, yet costs a good 20% less than the PACS 2.0 models?

They were there, on the floor, but you had to know where to look.  You had to be one of the people actually playing the game.

I’m going to do something I normally don’t do in my posts.  I’m going to name a few names along with the following disclaimer: I didn’t do an exhaustive search so I’m certain that I missed a few vendors and equally applicable solutions.   Therefore I am only commenting on the few that I did review.

If one were to describe the technology requirements for what we shall call the ideal next generation PACS, those requirements could be divided into three distinct subsystems: Workflow/Analytics, Visualization/Diagnostics, and Enterprise Archive/Distribution.  The adoption of the VNA and the Universal Viewer are well underway, and the arguments in favor of their deployment are well understood.  As for the other two subsystems that comprise what we think of as the department PACS, the three companies that I reviewed that offer commercially available Work Flow and Analytics Packages are: Compressus, Medicalis, and Primordial.  There are numerous companies that offer attractive Visualization/Diagnostics packages, but the application suite that I reviewed was developed by Visage Imaging.  Based on what I have seen and learned at RSNA 2012, in my opinion it is now possible to construct a next generation PACS by layering one of the three Workflow/Analytics packages on top of the Visage Imaging Visualization/Diagnostics package and interface this combination with a true VNA and the Universal Viewer package.  Once again, based on what I learned and in my opinion, that combination is substantially more potent than any of the PACS 2.0 solutions that the traditional PACS vendors are offering.

In my next blog I want to address numerous technology issues that had to be addressed by the participating vendors to make this best of bread combination work (like Directory database synchronization), and discuss some of the Service Level Agreement and Support issues that still need to be addressed.

Please stay tuned.  I think the Radiology PACS replacement market just got a whole lot more interesting.

The Real Reason for Deploying a Vendor Neutral Archive

I came across a PowerPoint presentation that I created in July 2006 that referenced a DICOM Migration Server, a term at that time referring to an “open” DICOM Part 10 Storage Solution.  I vaguely remembered the subject, so I opened the file and reviewed the slides.  I felt as though I had traveled back in time to the very earliest days of the paradigm shift that would one day be referred to as the Vendor Neutral Archive.  That’s six years ago last month.  Slide after slide contained bulleted descriptions of the numerous problems facing an organization that had managed to accumulate no fewer than five different department PACS.  Five separate silos of data that could not be exchanged between the PACS. Five different image viewers that the referring physicians had to toggle between.  The last few slides in the deck laid out a rather optimistic (at the time) plan for a strategic solution to the mess.  A grin spread across my face.

I closed the slide deck, assigned a loud red label to the file so I could easily find it again, and fast-forwarded to the present thinking, “You’ve come a long way baby!”

I have been intensely focused on both the concept and the reality of Vendor Neutral Archive for those last six years.  Perhaps that is why it seems so obvious to me why a healthcare organization should make the switch.  They should “take the A out of PACS”, move the data to a VNA, associate a universal viewer to the VNA and use this combo system to manage the distribution of that data to every other system and caregiver throughout the organization.  These are things that even the best of today’s department PACS are simply incapable of effectively doing in a multi-vendor environment.

Based on the questions I continue to see quoted in the print and electronic publications, posted on-line in the focus groups, and raised at the end of many of my webinars, there still appear to be a large percentage of both PACS admins and IT systems analysts that don’t “get it”.  They seem hung up on the technical features of the VNA and all of the potential snags that they fear are bound to occur when two systems, more importantly two vendors, are forced to work together.  The litany of both identified and suspected complications goes on and on.  No doubt the incumbent PACS vendors skillfully placed many of the items on these lists.

OK, it’s time to step back from the techy stuff for a minute.

It’s true.  Many currently installed department PACS are incapable of efficiently inter-operating with a foreign archive without help, simply because they were not designed to work with a foreign archive.  The installed base of VNA solutions would be a pitiful fraction of the real number, if the VNA guys had not developed some very clever workarounds to the inadequacies of many PACS.  Without help, most PACS could not be paired with a VNA. They lack the ability to store images to a foreign archive and then remember where they stored those images.  They are incapable of propagating ADT updates or Merge and Splits to an outside archive.  They have no concept of a deletion policy and therefore have no mechanism for reacting to an externally executed Purge Strategy.  Some PACS have no concept of a relevant prior coming from another PACS, and if the VNA suddenly delivers the study to its doorstep, the PACS thinks it’s a new study and puts it in a reading list.  As I have said, the litany of both identified and suspected complications goes on and on.  The naysayers apparently have not taken the time to read up and learn how all of these problems have been resolved.  As a consequence of those workarounds, the actual installed base of VNA solutions is actually a pretty impressive number.

My advice to those that still don’t get it is don’t get hung up on the technology.  The real argument for deploying the VNA is CONTROL.  It’s time for the organization to take control of its data.  Every day that goes by, another “x” gigabytes is forwarded to the department PACS where it is converted to an effectively proprietary DICOM format that the organization will eventually have to pay in time and money to move to yet another PACS with its proprietary format.  Regardless how soon the organization can afford to replace the incumbent PACS, it’s time to start migrating the data to a VNA…in effect it’s time to mitigate the cost of that future data migration.

What about future VNA migrations, when the first VNA has to be replaced with another VNA?  That’s a really good question.

The answer is actually quite simple.  The real objective in negotiating the contract for the VNA is to gain access outside of any confidentiality stipulation in the contract to the VNA database and all of the tools required to allow the organization to move its data from VNA 1 to VNA 2, at no cost.  Without that arrangement, you’ve missed the point.

Bottom line…initiate a pro-active DICOM migration of the PACS data to a entry-level VNA.  Take control of your data.  As soon as possible, replace the uncooperative PACS with a real PACS, one that fully interoperates with a VNA.

PACS / VNA Compatibility Issues

While much has been written and stated on behalf of the Vendor Neutral Archive being the ideal strategy for managing medical image data across the enterprise, little has been said about PACS Compatibility with the VNA Solution.

There’s a good deal more to this compatibility issue than the PACS being able to communicate with and exchange data with the VNA using DICOM.

Most department PACS, including Radiology and Cardiology solutions, were not designed to inter-operate with a foreign archive.  This is not to say that PACS systems were not designed to occasionally share study files with an external DICOM conformant system.  Most PACS can accomplish this using the DICOM communications protocol.  What I mean is that most PACS system designs are predicated on the assumption that the PACS will be the sole manager of the study data for the lifetime of the system.  And because of this design assumption, many of the current generation of department PACS are ill suited to the tasks required to fully inter-operate with a VNA.

Since most organizations probably did not include this compatibility issue in their PACS selection process, it may come as some surprise to learn that interfacing their existing PACS to a VNA is going to require solving a number of significant in-compatibility issues.

I thought it would be useful to present a summary of the more significant interoperability issues, because organizations need to be aware of the potential problems when they begin the process of planning for a VNA deployment.  The right VNA solution will have to be able to address these incompatibility issues.  It might also be prudent to consider these issues when planning for the next department PACS purchase, because sooner or later, odds are the organization is going to see value in data consolidation, and system interoperability.

Here is a list of the more critical PACS / VNA compatibility issues.

DICOM and IHE…The PACS should support a full suite of DICOM SOP Classes covering the full array of image objects that belong in the patient’s longitudinal record, not just those objects created in the imaging department.  This would include most of the DICOM Structured Report objects, image objects from Ophthalmology, Endoscopy, Pathology, and some of the non-image Cardiology objects like Waveforms and Hemodynamic data.  The PACS should also support image-related objects like Presentation States and Key Image Notes.  The system should also support a few key IHE profiles including Consistent Presentation of Images Profile, Presentation of Grouped Procedures Profile, Key Object Notes Profile, Simple Image and Numeric Reports Profile, and Transparent Query/Retrieve.

Foreign Study Support…The PACS should support the import and representation of Foreign Studies.  Ideally the PACS would directly accept from the VNA, studies originally acquired/processed by another (disparate) PACS or Image Source that are being managed by the VNA.  At the very least, the PACS would be able to receive from the VNA a non-billable order and use that order to aid in the acceptance of studies originally acquired/processed by another (disparate) PACS.

Store and Remember…The PACS should be able to “Store” (archive) DICOM objects originally acquired by itself to a foreign archive (VNA), and then “Remember” that the objects are stored on the VNA when the time comes to retrieve them.

Study Aggregation…The PACS should have the ability to automatically and pro-actively search for studies in the VNA that were originally acquired by another PACS and stored in the VNA (i.e. search the VNA for relevant priors originally acquired by another PACS).

HL7 Updates…the PACS should not only be able to accept HL7 updates from the local RIS or HIS and apply those updates to the metadata in the PACS that is associated with the image data the PACS is managing, but it should also be able to forward the same metadata updates to the VNA.

Object Versioning…The PACS should have the ability to forward to the VNA any updates or changes made to the study data (both pixel and meta data) after the initial “archiving” of the study data, effectively “re-archiving” the image or study.

Retention Messaging…The PACS should have the ability to accept and utilize the messaging from the VNA that is designed to communicate what image and study files have successfully been purged by the VNA’s Information Lifecycle Management (ILM) application.

The subject of PACS / VNA Messaging is actually the most critical of the PACS compatibility issues.  Perhaps one of the more challenging aspects of PACS / VNA interoperability is keeping the two disparate systems synchronized with each other.  Most but not all PACS accept and utilize HL7 updates from the HIS or RIS.  Many PACS do not have a reciprocal mechanism for updating a foreign archive (VNA) with changes that were made to metadata or pixel data in the PACS.

An even more challenging issue is presented by the advent of the purging mechanism that is supported by many VNA solutions.  This issue is referred to in the above list as Retention Messaging.  If a PACS is configured with a small local cache, and it is programmed to allow the oldest studies to fall off of that cache when the watermark is reached, how does it communicate to the VNA that it no longer has that study?  Correspondingly, if the VNA purges a study that has reached its retention limit, how does the VNA communicate to the PACS that the study no longer exists?

A number of the more advanced VNA solutions are attempting to resolve this “synchronization issue” through the use of a number of standard and not-so standard messaging techniques, because the VNA vendors recognize that few PACS vendors have considered VNA compatibility in their PACS designs and fewer still have implemented the appropriate IHE profiles.  Some of those solutions include:

  1. Private DICOM messaging
  2. Custom HL7 messaging
  3. The new IHE Profile called Imaging Object Change Management (IOCM), which is still in development

The Imaging Object Change Management Integration Profile will specify how one actor communicates local changes applied on existing imaging objects to other actors that manage copies of the modified imaging objects in their own local systems. The supported changes will include (1) object rejection due to quality or patient safety reasons, (2) correction of incorrect modality work list entry selection, and (3) expiration of objects due to data retention requirements. It will define how changes are to be captured and how to communicate these changes.

The successful assimilation of disparate PACS into an enterprise Vendor Neutral Archive configuration will have its challenges.  I think it is better to fully understand these challenges in order to better prepare for them, and I suggest that this knowledge play a key role in the VNA selection process.   It also makes sense to include the knowledge of these issues in the next PACS selection process, and thereby eliminate as many future interoperability issues as possible.

True Business Continuity Requires Two Separated Instances of all Key Applications

I sat in on a webcast Wednesday, July 20, 2011, and noticed how the concepts of Disaster Recovery and Business Continuity were frequently paired…in the bullets and in the presentation…with the inference that if you had a DR solution, you had a Business Continuity solution.  That’s not the case.  A Disaster Recovery solution is basically a second copy of the data, usually stored remotely, so it can be accessed whenever it becomes necessary to replace the first copy of the data that might be lost or temporarily unavailable.  A DR solution could be as simple as an off-line digital tape cartridge or a DVD disk stored in a safe room.  It could be a near-line solution such as a tape library system or an on-line solution like a separate spinning disk system, either of which is directly attached to the PACS.  A second SAN or NAS storage solution paired with its front end gateway server could be located in a remote data center in order to increase its chances of avoiding whatever disaster might take down the PACS and/or its primary copy of the data.  The DR solution could be as elaborate as a Cloud-based storage solution, which often feature multiple data centers located in distant states.

In all of the above examples however, the original PACS application or a standalone display application like a web server is required to access and display that back-up image data. If the PACS application or any of the display applications are down, or in any way unavailable, the second copy of the data cannot be accesses and it cannot be displayed.  In this case, there is no Business Continuity.

A significant number of installed Radiology PACS are based on a single instance of the data management and display application, but they are configured with both a primary and a secondary storage solution, each located in separate data centers.  Depending on how geographically separate those data centers are, the secondary storage solution represents a solid Disaster Recovery solution.

On the other hand, very few Radiology PACS are configured with two instances of the data management application or the display applications.  And because there is only one instance of these applications, such a configuration does not have a Business Continuity solution.  With few exceptions, if the PACS application is unavailable, neither the primary nor secondary copy of the data is accessible.  If the display applications are unavailable, neither the primary nor the secondary copy of the data can be displayed.

A true Business Continuity solution requires two instances of the data management application, which can access either the primary or secondary storage solutions, and two instances of whatever display application the user prefers for accessing and displaying the data.  These two paired applications…data management and data display…should be geographically separated, so either of them can survive the disaster that might befall the other.

Since few PACS can be configured with multiple instances of its data management and display application, the current strategy for building a Business Continuity solution has shifted to deploying a dual-sited Vendor Neutral Archive and a dual-sited UniViewer.  The two separate instances of the VNA and the two separate instances of the UniViewer back each other up in the event of a disaster.  In the event that the only instance of the PACS and/or its only instance of the display application becomes unavailable, the new studies are probably interpreted at the modalities, and the priors are retrieved from whichever VNA is available and displayed using whichever UniViewer is available.  That is a true Business Continuity solution.  Take a look at my previous post regarding Failover Strategies, if you want to get a better idea of how primary and secondary subsystems back each other up.

Failover Strategies in Mirrored Configurations of Medical Image Management Systems

The subject of Failover and Resynchronization is near and dear, as I’ve been configuring mirrored systems for years.  I have become quite familiar with how various vendors address this requirement.  The principal reason for building a mirrored Picture Archiving and Communication System (PACS) or a mirrored Vendor Neutral Archive (VNA) solution is Business Continuity.  Most healthcare organizations realize that they cannot afford to lose the functionality of a mission critical system like a PACS and an Enterprise Archive, so they need more than a Disaster Recovery strategy, they need a functional Business Continuity strategy.   Unfortunately, it’s really tough to build a dual-sited, mirrored PACS that actually works.  The sync and re-sync process drives most PACS vendors nuts.  There are very few PACS that can support multiple Directory databases.  I think this shortcoming of most PACS systems is why we have been configuring mirrored VNA solutions from the beginning…if you can’t configure the PACS with a BC solution, then you should at least configure the enterprise archive with a BC solution.

In the dual-sited, mirrored image management system, there are two nearly identical subsystems, often referred to as a Primary and a Secondary.  The two subsystems are comprised of an instance of all of the application software components, the required servers, load balancers, and the storage solutions.  Ideally these two subsystems are deployed in geographically separate data centers.  While it is possible to make both subsystems Active, so half of the organization directs its image data to the Primary subsystem and the other half directs its data to the secondary subsystem, the more common configuration is Active/Passive.  In the Active/Passive configuration, the organization directs all of its data to the Primary subsystem and the Primary backs that data up on the Passive Secondary subsystem.

When the Primary subsystem fails or is off-line for any reason, there should be a largely automated “failover” process that shifts all operations from the Primary subsystem to the Secondary subsystem, effectively making it the Active subsystem, until the primary subsystem is brought back on-line.  When the Primary subsystem comes back on-line, there should be a largely automated “resynchronization” process that copies all of the data transactions and operational events that occurred during the outage from the Secondary back to the Primary.

Business Continuity operations can be even more complicated in an environment where there is a single instance of the PACS and a dual-sited, mirrored VNA configuration. In this environment, the failover and resynchronization processes can be somewhat complicated, giving rise to numerous questions that should be asked when evaluating either a PACS or a VNA.  I thought it would be beneficial to pose a few of those questions and my associated answers.

Q-1: If the hospital-based PACS and Primary VNA are down, how does the administrator access the offsite Secondary VNA and subsequently the data from the offsite VNA? Is the failover automated, or manual?  If manual, what exactly does the admin do to initiate the failover?”

A: The response depends very much on the VNA vendor and exactly how that VNA is configured/implemented.  Some VNA solutions have poor failover/resynchronization processes.  Some look good on paper, but don’t work very well in practice.  With some VNA vendors, system failover and resynchronization in a mirrored environment is a real strong suit, as they support many options (VMWare, Load Balanced-automatic, Load Balanced-manual, and Clustering).  Some VNA vendors have limited options, which are costly and actually create down time.  The better approach is a Load Balanced configuration with automatic failover (which requires certain capabilities existing on the customers network-VLAN/Subnet/Addressing), with manual failover being the second option (and more common).  VMWare is becoming much more common among the True VNA vendors, but many of these vendors will still implement the VMWare clients in a load balanced configuration until customers are able to span VMWare across data centers and use VMotion technology to handle the automatic failover.  There is also the option of using DNS tricks.  For example, IT publishes a hostname for the VNA which translates to an IP in Data Center (DC) A, the DNS has a short Time to Live (TTL), such that if DC A fails, IT can flip the hostname in the DNS and the TTL expires in 1-5 seconds, then all sending devices automatically begin sending/accessing DC B.

There is also a somewhat unique model that implements the mirrored VNA configuration in an Active/Active mode across both Data Centers – whereby the VNA replication technology takes care of sync’ing both DC’s, the application is stateless so it doesn’t matter where the data arrives, because the VNA makes sure both sides get sync’d.

The point in all of this is simply that the better and obviously preferred approach to failover is a near fully automated approach, ONCE THE SYSTEM IS SET-UP.  Resynchronization of the data should be automated as well.  Only updates/changes to the user preferences might require manual synchronization after a recovery.

Q-2: What do the UniViewer (zero client, server-side rendering display application) users have to do to access the secondary instance of the UniViewer? Do the users have to know the separate URL to login to that second UniViewer?

A: If implemented correctly, the UniViewer should leverage the same technology as described above for the VNA.  The user’s URL call goes to a load balancer, which selects the Active UniViewer rendering server.   If the Primary UniViewer (Active) has a failure, another node, or another data center takes over transparent to the end user. The rendering server in turn points to a load balanced VNA such that the users need to do nothing differently if the UniViewer servers or the VNA servers switch.

Q-3: Where do modalities send new studies if the onsite PACS and/or the Primary VNA are down?

A: Once again, this is highly variable, and there are several options.  [1] If the designated workflow sends new data to the PACS first and that PACS goes down, then I’d argue that the new data should be sent to the onsite VNA.  That means changing the destination IP addresses in the modalities.  [2] Vice-versa if the designated workflow sends the new data to the VNA first.   Most of the better VNA solutions can configure a small instance of their VNA application in what I refer to as a Facility Image Cache (small server with direct-attached storage).  One of these FIC units is placed in each of the major imaging departments/facilities to act as a buffer between the Data Center instance of the VNA and the PACS.  [3] In this case, the FIC is the Business Continuity back-up to the PACS.

If both the PACS and the local instance of the VNA are down, the new study data should probably be held in the modality’s on-line storage, for as long as that is possible.  The modalities could also forward the data across the WAN to the Secondary VNA in the second data center, but the radiologists would probably find it easier to access and review the new study data from the modality workstations.

Of course all of these back-up scenarios are highly dependent on the UniViewer.  In the case of those PACS with thin client workstations, if the PACS system goes down, the workstations are useless.  In the case of fat client workstations, most are capable of only limited interactions with a foreign archive.  See the next question and answer for additional detail.

Q-4: Do the radiologists read new studies at the modalities and look at priors using the UniViewer whose rendering server is located in the offsite data center?

While that is possible, my recommendation would be to use the UniViewer for both new and relevant priors.  Some of the UniViewer technology is already pretty close to full diagnostic functionality, some of the very advanced 3D apps being absent. There are already examples of this use of the UniViewer at a number of VNA sites…not only for teleradiology applications, but also diagnostic review if the PACS system goes down.  My prediction is that the better zero client server-side rendering UniViewer solutions are going to be full function diagnostic within a year.   This is a critical tipping point in the VNA movement…a real game changer.  Once the UniViewer gets to that level of functionality, the only piece of the department PACS that is missing will be the work list manager.   As soon as it’s possible to replace a department PACS with a solid [1] VNA, [2] UniViewer, and [3] Work List Manager, the PACS vendors will have a very difficult time arguing that their PACS (less the Archive and Enterprise Viewer) is still worth 90 cents on the dollar, as they are doing today.

Q-5: Does the EMR, if linked to the onsite UniViewer, have a failover process to be redirected to the offsite UniViewer so that clinicians using the EMR still have access to images through the EMR, or do the users need to have the EMR open in one browser and another browser open that points at the offsite UniViewer which they login to separately?

A: Failover from Primary to Secondary UniViewer should be and can be automated (see 1 and 2 above), if implemented correctly and support by the UniViewer technology.

In conclusion, most healthcare organizations are highly vulnerable to the loss of their PACS, because most PACS cannot be configured with a Business Continuity solution.  That problem can be remedied with a dual-sited, mirrored Vendor Neutral Archive paired with a dual-sited UniViewer.  While most VNA vendors can talk about Business Continuity configurations, their failover and resynchronization processes leave something to be desired.  The reader is encouraged to build a set of real-world scenarios, such as those presented here, and use them to discover which VNA will meet their Business Continuity requirements.  The Request For Proposal (RFP) document that I have created for VNA evaluations has an entire section on Business Continuity and the underlying functionality.

What’s it going to take to achieve Meaningful Use of Images?

The other day a friend of mine forwarded to me a link to the Imaging Technology News eNews web site.  My friend encouraged me to look on the left bar of the web page and find the invitation to participate in their current survey.  The question was “Will PACS/RIS meet the meaningful use criteria to qualify for incentive dollars?”  If the survey is still running, you can check out the current results here.

Last time I checked, 33% thought that PACS/RIS would meet the criteria and another 30% thought that there’s a good chance it will.

I’d love to see the demographic of the survey participants, and I’d love to see a list of their assumptions.

I’m among the 30% that responded with a solid “no”, convinced that the PACS/RIS as we know it will not qualify for Meaningful Use, because it simply doesn’t have what it takes, and most likely never will.

If the survey participants gave serious thought to the question, they should have realized that the most critical component of what it takes to sustain Meaningful Use will be “ease of use”.  Most physicians are far too busy to learn and remember how to use more than one image viewer.  Most physicians are far too busy to switch back and forth between multiple viewers to assemble a montage of all the relevant clinical information in a single viewing window.  That’s exactly what will happen if we continue on the present path of developing individual URL links between the Physician Portal and the data elements being stored in each of the specialized departmental PACS, and using those department PACS viewers to view the data.  This approach shouldn’t make sense to IT, and it won’t make sense to the physician users.  So the participants must have been assuming that an all-encompassing Enterprise PACS will emerge, a single PACS that will embody all of the specialized department PACS requirements and thereby become the Uni-PACS.

In my opinion, it is highly unlikely that a current generation Radiology or Cardiology PACS or any other departmental PACS for that matter, will evolve in the next few years into an Enterprise Data Repository capable of managing the patient’s longitudinal record of all clinical information.  I seriously doubt that they will be able to manage all of the image information, much less all of the non-DICOM and non-image data objects.

Managing all of this clinical data is probably the easier part.  The harder part will be providing all of the expected display and processing applications that are specialized for each of the contributing imaging departments.  This is not to say that some of the larger vendors won’t try to become an all-encompassing enterprise PACS, or at least claim to be the Whopper of PACS, but I don’t see that happening.

In my opinion, the more likely scenario will be the Enterprise Neutral Archive fulfilling the role of the Enterprise Data Repository, and the (interfaced or embedded) UniViewer will provide the unified set of viewing tools that the physicians will use to access and view all of a patient’s clinical information, both the image and the non-image data being managed by that Neutral Archive.

Today, more and more Health Care organizations are “getting it”.  They see all of the advantages of separating the “archive” data management applications from the departmental PACS.  And it’s a natural to add a viewer to this new generation Archive.   Sooner or later, each of the PACS vendors will “get it”, and at that moment the push will be on in their R&D groups to further differentiate their department PACS products with the specialized applications unique to that department.  Their PACS will have to become an even better, specialized tool for each department, because the Neutral Archive will have already become the tool of choice for the Enterprise.  Meaningful Use will be much easier to achieve if the physicians know they only have to go to one repository and only have to use one viewing application to assemble all of the relevant clinical information in a single viewing session.  Get it?