Retrieving DICOM Studies via XDS(-I)

A comment from Gasper Andrejc on my "How to register a DICOM Study in an XDS network" blog inspired me to write this article to bust some myths that are related to retrieving DICOM studies through an XDS(-I) infrastructure. In his comment Gasper asked whether I have seen performance issues related to the use of RAD-69 or RAD-75 image retrieve transactions. Before addressing the performance part, first some "history lessons". 

In the early days of the Cross-enterprise Document Sharing for Imaging (XDS-I) profile it was intended that DICOM images remain at the source PACS until requested by someone else. Therefore, when "publishing" a DICOM study no image moves anywhere. Instead, in order to allow other XDS network participants to "discover" what images are available, a DICOM Key Object Selection (KOS) instance is created that is stored as an XDS document in a XDS repository, and registered in a XDS registry. The KOS contains two relevant elements:

1. A list of referenced SOP Instances, and
2. A "retrieve AE Title" (later replaced by the retrieveLocationUID attribute).

The combination of both allow someone who is interested in these images to submit a request to a source system identified by its "retrieve AE Title", and pick from the list of reference SOP Instance the images of interest to retrieve. 

XDS-I.a

In the initial version of XDS-I (referenced as XDS-I.a) two retrieve protocols were "allowed". One being the well-known DICOM C-MOVE/C-STORE transaction combination (a.k.a. DICOM Retrieve), the other being Web Access to DICOM Objects (WADO). 

1. it is a http(s)/restful based protocol, that allows a requestor to
2. add some parameters to the retrieve request (i.e. quality, region of interest, etc.). WADO turned out to be a very practical approach to request both low and high resolution images depending on the requestor's preferences. 

The downside of XDS-I.a was that both the DICOM and WADO protocols require the requesting consumer application to be able to "reach" the source PACS via a network connection. This poses some challenges in the real world, given that in an XDS-I based network the consumer and source applications are likely to live in different networks or network segments, making it difficult for the consumer to reach a source by its IP-address or hostname. Rather complex Network Address Translations (NAT), and other networking tricks are required to solve this. In addition the DICOM protocol is not a Wide-Area Network (WAN) "friendly" protocol since it lacks some of the security features that are more or less built in with the http protocol. Routing DICOM traffic through a series of firewalls can be pretty cumbersome. 

Another important handicap with both DICOM and WADO was that both lacked support for adding a SAML token to a transaction as is specified by the IHE XUA profile. The absence of a SAML token makes it difficult to implement a form of access control often required by an XDS Affinity Domain.

XDS-I.b

To solve some of these issues XDS-I.b introduced the famous RAD-69 transaction allowing DICOM images to be exchanged using a SOAP/Web Service protocol. I do not recall the reasons why, but RAD-69 was only designed to functionally mimic a DICOM Retrieve. This means that a consumer application is able to retrieve DICOM images by specifying the SOPInstanceUIDs, SeriesInstanceUIDs, and/or StudyInstanceUIDs of interest, and specify a transfer syntax (for example JPEG Lossless compression) to receive the image pixel data. RAD-69 includes an option to use DICOM JPIP (JPEG 2000 Interactive Protocol) to retrieve (partially) rendered images. However, little to no implementation of JPIP exists today, leaving no other option than to retrieve full fidelity DICOM images over RAD-69.

A perceived shortcoming of RAD-69 was the fact that it did not provide a functional equivalent to a DICOM C-FIND. RAD-69 was intentionally not designed to do so given the fact that, within an XDS network, consumers should use an ITI-18 (Registry Stored Query) transaction instead. Unfortunately many PACSs that rely on a DICOM C-FIND to discover studies in a foreign source did not, or could not, implement support for the ITI-18 transaction.

WADO-WS

DICOM introduced WADO-WS (DICOM over Web Services) as a modern DICOM Upper Layer stack. A few years later, as the frontier of technology moved towards the use of RESTful transactions, DICOM followed up with WADO-RS (the RESTful equivalent of WADO-WS). For reasons beyond my ability to understand, the DICOM committee deprecated WADO-WS before it had a chance to prove itself in the real-world. As a consequence IHE did not consider WADO-WS as a replacement or alternative for RAD-69, or its XCA-I equivalent RAD-75.

Only in the Netherlands, upon request by their customers, three vendors implemented WADO-WS as an alternative to RAD-75 as it allowed for adding similar parameters to the request as the initial WADO protocol allows. Most importantly, this enables fast(er) cross-community image transfers as a XCA-I initiating gateway can request images with reduced quality to achieve reasonable "first image display response times". 

As mentioned above WADO-WS did not become a widespread alternative as IHE profiles cannot be built on deprecated communication protocols. 

Back to performance

When using one of the transactions mentioned above to move images from a source PACS to a destination PACS (or consumer applications in general), there are a number of factors that have a significant impact on performance. 

The source system

The speed with which a source system is able to retrieve images from its local storage and "push" it into the network matters. Given that most image sources in an XDS network are still DICOM based systems, an often seen issue is that a source system "pushes" images out in sequential order where it waits for an acknowledgement of receipt of image N, before starting to push image N+1 to the network. This behavior is killing the image retrieval performance that theoretically can be achieved based on the available network bandwidth. It is not uncommon to see the effective bandwidth use drop to as little as 10% of its theoretical maximum. If the source system cannot be fine tuned to increase this effective bandwidth use, the second best option is to resort to "image retrieve caching" or a form of "prefetching".

The "consumer" system

Similar to the sending side, the speed at which received images are processed by the consuming side also matters. There is not a single cause in this case. Some consumer systems require a DICOM study to be retrieved in full before the first image is rendered. In other cases the transfer syntax the consumer requires is not supported by the source, forcing the source to revert to sending uncompressed pixel data. Furthermore some consumer systems only send out a receipt acknowledgement of image N after the image data has been processed (e.g. recompressed) and persisted on local storage. Hence causing an extra delay before the sending side will push image N+1. In a similar way to what was described above this kills your effective network bandwidth use. Except for prefetching there is little to nothing you can do to work around this behavior of consumer systems.

The size of the system data

A bit of an "open door" in this context is that size matters as well since all "image pixel data" must travel the network in order to reach its destination. As the size of radiology studies have increased over the years it is not uncommon to see DICOM Studies of many Megabytes, or even Gigabytes in size. In theory a multi-slice CT of 2.5 Gigabyte can travel a 100Mb/s connection in a little less than 3 and a half minutes. Because of the above mentioned causes such transmission times are hardly seen. Compressing pixel data may help to reduce the transmission time as lossless compression algorithms reduce the pixel data size. On average, 50-60% reduction is possible. However, if the pixel data is not available in compressed format at the source, the transmission time is negatively impacted by adding compression time for each image. 

The number of "network hops"

In the real-world the network between the "source" and the "consumer" system is never a straight line as it consists of many components as discussed above. Certainly in a cross community scenario where image retrieves are routed via XCA gateways this is the case. Each "hop" will result in a reduction of the effective bandwidth use as (a) the network bandwidth between the gateways may be different (mostly lower) than the network bandwidth between the consumer and its initiating gateway, or the source and its responding gateway. Furthermore (b), gateways may not have adopted "streaming" designs resulting in each "hop" caching incoming images before they are pushed out to the next "hop". Paying attention to your health information exchange network topology may help identify unnecessary "hops" in your network.

Summary

To come back to the question from Gasper "How do you see performance issues and what kind of optimizations or compromises do you see happening when DICOMs are exchanged using RAD-69, or perhaps even more performance-risky RAD-75?". 

My "summarized" response would be that it is neither the RAD-69, nor the RAD-75 transaction that cause performance issues with image transfers across an XDS(-I), or XCA(-I) based infrastructure. Before "blaming" XDS/XCA it pays off more to scrutinize the behavior of the sources and consumers that are being used. It also pays off to look for XCA-I gateways that implement a streaming based design. 

To that aspect, it does help to pay attention to the IHE conformance statements of systems that are connected to an XDS/XCA-based health information exchange network, and ask questions about these when procuring new systems. Even when they are primarily used for intra-enterprise purposes! 

In some way this is also a "forward warning" as often DICOMWeb and/or FHIR are positioned as (streaming) alternatives to existing IHE-based transactions. Unfortunately, a shift in protocol does not do anything to resolve the performance challenges discussed above. People that focus on the choice of protocol alone overlook the fact that the design of the system, and its individual components are much more important in determining overall performance. In similar ways as WADO-WS, RAD-69 and RAD-75 do, both DICOMWeb and FHIR allow for streaming implementations. Unfortunately neither gives any guarantees to that fact that their implementers also do.