3D Reconstruction from biplanar x-ray

Problem 

CT scans provide detailed 3D view of internal organs including bones. It plays an important role in diagnosing various diseases and when doing trauma surgery. However, CT scan is not accessible in most rural areas, and many people even in urban areas cannot access it because of high costs. Medical professionals therefore rely on X-ray scans which are much lower cost and are widely available even in rural areas. But X-ray images do not provide the full 3D picture of internal structures and can lead to sub-optimal clinical outcomes. Hence, many people in countries like Nepal are unable to get high quality trauma surgery or diagnosis of orthopedic diseases that require high quality 3D bone imaging. 

Research Aim 

Our research aims to develop AI system that can take two (or very few) X-ray scans from different angles and reconstruct accurately 3D bone structures to improve diagnosis and surgical planning. Building such a tool will make high quality 3D imaging from low-cost easily available X-ray scans accessible and affordable to a large fraction of people currently deprived of quality care due to expensive CT scans. 

Current Stage of our Research 

Some scientists from around the world have explored and built AI methods to reconstruct 3D images from two X-ray scans taken from the front and side views. However, these methods were not compared properly to each other. Moreover, it is not clear if these methods really help in clinical decision making, as they were assessed primarily using technical image-based metrics; not using the metrics relevant to clinical parameters. These methods were not easily available to use and test for other researchers. 

As a first step towards clinical adoption, we implemented major AI methods and compared their performance in terms of metrics that are directly relevant to clinicians including bone morphometry parameters.  We performed thorough evaluation on major bones including vertebra, pelvic bones, ribs and femur. This paves the path for meaningful intervention including reconstructing fractured bones and building robust systems that can work on challenging clinical environments such as low-powered X-rays. 

Outcomes so far 

We identified the best AI methods from among those currently described in the literature. We found various gaps in how these methods are evaluated and provided complimentary tools for better evaluation [1] that will help measure if the future methods will actually help better clinical outcomes.  

[1] Shakya, Mahesh and Khanal Bishesh. “Benchmarking Encoder-Decoder architectures for Biplanar X-ray to 3D Shape Reconstruction.” arXiv preprint arXiv:2309.13587(2023) (to appear at NeurIPS 2023) 

Note: top 10 scientific venue for research articles in any scientific disciplines by h5-index, and one of the top AI venues (h5-index ranking) 

Funding Agencies 

Self-funded by NAAMII 

Team 

Mahesh Shakya and Bishesh Khanal