Supervisor: Suresh Manandhar
Project Area: Natural Language Processing
Project ID: INT-2020-SM1
Project Title: Tools for processing languages of Nepal
Project Description:
This project will develop set up tools consisting of corpora, word embeddings, dependency parsing, morphological analysis with focus on languages of Nepal and more generally within the perspective of low resource languages.
It will demonstrate the effectiveness of the developed tools by applying it to various sequence modelling tasks such as sentiment analysis, named entity recognition and relation extraction. Particular focus will be on handling code switching with ability to handle multiple encodings (Latin and Devanagiri scripts). Embeddings such as LASER (from Facebook AI) that model embeddings from multiple languages in the same space will be relevant here. Aim will be to better handle word morphology for Nepali (and other languages) and build dependency parsing algorithms employing transfer learning.
References (optional):
—————————————————————————————————————————
Supervisor: Suresh Manandhar
Project Area: Natural Language Processing and 3D animation
Project ID: INT-2020-SM2
Project Title: Personalised Virtual Character Interfaces
Project Description:
This project will develop a set of tools for 3D character animation (half body) that is suitable for interfacing with a NLP system. It will use existing open source frameworks such as Blender for building a 3D character model.
Key components of the system will be mechanism for lip syncing and automated gesture generation based on the NL input. Existing text to speech conversion engines will be employed to generate timing information. Phoneme to viseme mapping, breathing, nodding etc will be available in a customizable setting. Deep learning methods can be explored for gesture generation.
References (optional):
—————————————————————————————————————————
Supervisor: Suresh Manandhar
Project Area: Natural language processing
Project ID: INT-2020-SM3
Project Title: Conversational Dialogue systems
Project Description:
Interfacing NLP dialogue systems with information systems (e.g. POIs, timetables and generic databases) is currently a hot topic. This research will aim to extend current research in dialogue systems that interface with a background knowledge graph. Such systems combine aspects of social chatbots together with task oriented dialogue systems. The research will focus on transfer learning (e.g. from large scale language models such as GPT2, LASER) together with better loss functions suitable for conversational dialogues. We expect the system to work in Nepali and other low resource languages.
References (optional):
—————————————————————————————————————————
Supervisor: Suresh Manandhar
Project Area: Natural language processing
Project ID: INT-2020-SM4
Project Title: Unsupervised and minimally supervised machine translation for Nepali and other low resource languages
Project Description:
There have been recent successes in unsupervised and minimally supervised machine translation. Key to this success has been the application of dimensionality alignment methods (e.g. PCA, Procrustes transformation) that align the latent spaces of word embeddings from multiple languages and the use of large scale language models. However, high quality language translation from such models is still a distant goal. The aim of the project will be to do a detailed analysis of key shortcomings of such models and address a subset of these with the aim of generating better quality translation. The project will focus on low resource MT with focus on Nepali.
References (optional):
—————————————————————————————————————————
Supervisor: Suresh Manandhar
Project Area: Natural language processing
Project ID: INT-2020-SM5
Project Title: Text to knowledge graph and vice versa
Project Description:
Knowledge graph extraction from text is a hard NLP problem that requires entity recognition, relation extraction and entity linking among other tasks as prerequisites. There has been only limited effort towards end-to-end systems for knowledge extraction.
Domain adaptation and transfer learning is key for achieving success in this task. Additionally, distant supervision from existing knowledge bases such as DBpedia and Wikidata alleviate the need for large scale annotated data.
The project will investigate encoder-decoder architectures for text to knowledge graph to text conversion. It will leverage existing work on text to knowledge graph generation using distant supervision and models for text generation from data. The project will aim to build an end-to-end model for this task.
References (optional):
—————————————————————————————————————————
Supervisor: Suresh Manandhar
Project Area: Natural language processing
Project ID: INT-2020-SM6
Project Title: Morphological analysis of languages of Nepal
Project Description:
Unsupervised morphological analysis is a notoriously hard problem and deep learning models so far have achieved limited if any success in this subfield. In this work, we aim to combine Bayesian sampling based methods together with deep learning to come up with an unsupervised morphological analysis system. Additionally, neural program learning together with RL (reinforcement learning) objectives will also be explored. The long term goal of this research will be to learn two-level morphology rules in a fully unsupervised setting together with embedding composition methods for handling both known and unknown words.
References (optional):
—————————————————————————————————————————
Supervisor: Suresh Manandhar
Project Area: Machine learning and Chemistry
Project ID: INT-2020-SM7
Project Title: Modelling chemical reactions using deep learning
Project Description:
Existing work on learning representations of molecules employ methods for transforming the molecular structure representation to a string-like representation (e.g. Weisfeiler-Lehman Network) and generating embeddings for the molecules or directly generating embeddings from the graph molecule structure. The project will focus on learning the reactions of organic molecules in a supervised setting. The aim will be to explore better representations and learning algorithms. The repeated nature of chemical reactions (where if a + b → c then it is possible for c + a → d resulting in a cascade effect) is currently a challenge for existing machine learning models. Additionally, the presence of catalysts and environmental conditions (e.g. temperature, pressure) can have a drastic effect on the reaction outcomes.
References (optional):
—————————————————————————————————————————
Supervisor: Suresh Manandhar
Project Area: Computer vision
Project ID: INT-2020-SM8
Project Title: Image reconstruction from fibre optic imaging
Project Description:
Image reconstruction from fibre optic has the potential for enabling low cost medical imaging. However, different fibres have potentially different optical properties making the reconstruction task challenging. The aim of the project will be to explore methods for easy calibration and image reconstruction. The project will involve building a small hardware for imaging although the bulk of the task will involve image reconstruction.
This project will be conducted in collaboration with Phutung Research Institute.
References (optional):
—————————————————————————————————————————
Supervisor: Suresh Manandhar
Project Area: Computer vision
Project ID: INT-2020-SM9
Project Title: Unified learning from visual scene graphs and Natural Language generated Knowledge graphs
Project Description:
Scene graph learning from visual input generates knowledge graph like representation that capture the semantics of an image. Similarly, converting from natural language input to knowledge graphs and vice versa is currently a key topic within NLP.
The aim of the project will be to build a unified machine learning platform that creates knowledge graphs based on both language and visual input. The ultimate goal of the project will be to deploy this within a robot that can understand multi-modal input.
References (optional):
—————————————————————————————————————————
Supervisors: Bishesh Khanal
Project Area: Machine Learning & Medical Imaging
Project ID: INT-2020-BK1
Project Title: Semi-supervised learning in medical Ultrasound (US) images
Project Description:
Deep learning usually needs a large amount of annotated data for training. In medical images, ground truth annotation usually requires domain expertise of radiologists. This makes getting a large number of annotations expensive or impractical in many cases. For example, segmenting a left ventricle in cardiac ultrasound (US) requires a radiologist or cardiologist to spend several minutes manually delineating the ventricle. Semi-supervised learning is an approach where a small number of labelled data together with a large number of unlabelled data are used in training for learning. In this project, we will first explore the state-of-the art semi-supervised learning algorithms in computer vision to segment relevant structures (e.g. Left ventricles) in cardiac US and infer clinical metrics (e.g. Ejection Fraction). Then, we will work towards developing novel semi-supervised learning algorithms improving the state-of-the-art.
References (optional):
—————————————————————————————————————————
Supervisors: Bishesh Khanal
Project Area: Machine Learning & Medical Imaging
Project ID: INT-2020-BK2
Project Title: 3D reconstruction from biplanar X-ray images
Project Description:
X-ray radiographs give a 2D image while CT scans give 3D images. CT scan is expensive, has high radiation and is not easily available in many towns or rural areas. In contrast, X-ray is relatively accessible, low cost and has less radiation. However, surgeons (e.g. spine surgery in scoliosis, knee/hip joint replacement etc.) benefit from 3D scans which gives clearer pictures of the anatomy for surgery planning. In this work, we will work towards developing a novel and state-of-the-art method for 3D reconstruction from two X-ray views: AP view and lateral view.
References (optional):
—————————————————————————————————————————
Supervisors: Bishesh Khanal and Basant Pant (Neurosurgeon, Annapurna Neurological Institute)
Project Area: Machine Learning & Medical Imaging
Project ID: INT-2020-BK3
Project Title: Automated measurement of Lacunar Infarction in brain MRI
Project Description:
Small cavities (around 3-15mm) filled with cerebrospinal fluid called lacune are known to be related to cognitive decline, particularly speed and motor control. Lacunes are also related to future stroke risk. Currently, neurologists have to manually annotate and measure the lacunes which is time consuming and also prone to high inter-observer variability. In this project we will develop automated methods for delineating and quantifying lacunar infarcts. We will further explore developing an index score for lacunar infarction based on the characteristics of the lacunes such as size, shape and distribution in the brain. This project will be in partnership with Annapurna Neurological Institute and Allied Sciences.
References (optional):
—————————————————————————————————————————
Supervisors: Bishesh Khanal
Project Area: Machine Learning & Medical Imaging
Project ID: INT-2020-BK4
Project Title: Lung X-ray Analysis
Project Description:
X-ray is the most common imaging modality used in various lung pathologies. Diseases such as Tuberculosis, Pneumonia and COPD are quite common in Nepal. However, there are not enough radiologists, particularly in smaller towns and rural areas for properly reporting X-ray images. In this project, we will develop a set of tools for predicting lung pathologies, and estimating important biometrics such as Total Lung Capacity (TLC) from X-ray images. This work will also include simulation of Digitally Reconstructed Radiographs (DRRs) from CT scans and make them realistic looking X-rays which will be used for training certain deep learning models.
References (optional):
—————————————————————————————————————————
Supervisors: Bishesh Khanal
Project Area: Machine Learning & Medical Imaging
Project ID: INT-2020-BK5
Project Title: Optic nerve sheath diameter estimation
Project Description:
Optic Nerve Sheath (ONS) diameter measured from Ultrasound (US) images are used to estimate intracranial pressure (ICP) which are important in emergency and trauma situations. However, experts are needed to be able to take ocular US images and measure ONS diameter. In this work we will develop automated methods to measure accurately ONS diameter from US images. Once this is achieved, we will further explore the guidance system for non-experts to be able to acquire right images for measuring the ONS diameter.
References (optional):
—————————————————————————————————————————
Supervisors: Bishesh Khanal and Raunak Shrestha
Project Area: Machine Learning & Histopathological Imaging
Project ID: INT-2020-BK6
Project Title: Histopathological image analysis for breast cancer prediction
Project Description:
Breast cancer is the second leading cause of cancer-related deaths among women in Nepal. Every year, more than 2000 new cases of breast cancer are diagnosed in the country (GLOBOCAN 2018 estimates). Majority of these women are under 50 years of age which severely impacts the economically active population as well as places a substantial burden on the Nepalese healthcare system.
Early diagnosis of breast cancer can increase the chance of successful treatment and survival. The diagnostic procedure often involves manual visual inspection of microscopic histopathology image section of suspected cancerous tissue biopsy obtained from the patient. This is a very tedious and time-consuming step, the accuracy of which largely depends on the expertise and experience of highly qualified pathologist. However, the diagnostic interpretation often varies between one pathologist to the other, especially in borderline or complicated cases. This in turn may have a great effect in the patient’s treatment decision. Therefore, an automated histopathology image-based breast cancer diagnostic system is required to aid the pathologist speed up the diagnosis and further improve the quality of diagnosis.
To address the challenges above, we propose (1) to develop a machine learning based histopathology image classification model to efficiently and accurately predict breast cancer. For this, we will train the machine learning models with Hematoxylin-eosin (H&E) histopathology images from biopsy confirmed breast cancer cases. (2) Furthermore, we will develop histopathology image classification model to classify different subtypes of breast cancer. For this, we will utilize Immunohistochemical (IHC) stained images generated using different diagnostic protein markers commonly used in breast cancer diagnosis.
References (optional):
—————————————————————————————————————————
Supervisors: Bishesh Khanal and Basant Giri (KIAS)
Project Area: Machine Learning & Medical Imaging
Project ID: INT-2020-BK7
Project Title: Smartphone based automated detection of (oo)cysts
Project Description:
Smartphone based platform for automatic detection and quantification of Giardia lamblia cysts and Cryptopsoridium parvum (oo)cysts using machine learning.
According to the World Health Organization (WHO), about 8% of total deaths reported in Southeast Asia are caused by diarrhea. Several instances of diarrheal outbreaks are reported in Nepal every year. One of the major causes of diarrhea is human exposure to giardia and cryptosporidium parasites contaminated food and water. Conventional methods using optical microscopes, immunofluorescence-based microscopy, and polymerase chain reaction techniques for determining endoparasite oocysts are expensive, therefore they are not suitable for field screening of samples in resource limited settings.
Researchers at Kathmandu Institute of Applied Sciences (KIAS) have developed smartphone-based bright field imaging as a cost-effective platform for detection and quantification of cysts of two important foodborne parasites, Giardia lamblia (elliptical, 7-20 um) and Cryptosporidium parvum (spherical, 3-7 um). Currently, the cysts on the images are identified and counted by human eyes. In this work we will develop deep learning based method to automatically and specifically recognize (oo)cysts from other micro-objects on the image which requires exploring very tiny object detection in noisy image settings. The ultimate goal is to have a ready to use smartphone platform that can take pictures of the sample count the (oo)cysts on the sample.
References (optional):
————————————————————————————————————————
Supervisors: Suman Raj Bista and Danda Pani Paudel
Project Area: Computer vision, Robotics
Project ID: INT-2020-SRB1
Project Title: Indoor Recognition/Navigation with Turtlebot3
Project Description:
This project will develop reliable and robust indoor visual navigation and object recognition for mobile robots like Turtlebot that has limited onboard computing power. The robot can take advantage of the remote GPU-power available in NAAMII.
The project will investigate mapping, localisation, path planning, collision avoidance, semantic scene understanding, and motion control for navigating in the indoor environment. The test platform for this project is Turtlebot3 Burger robot which is available at NAAMII.
References (optional):
—————————————————————————————————————————
Supervisors: Danda Pani Paudel
Project Area: Computer Vision
Project ID: INT-2020-DPP1
Project Title: Image Retrieval for Localization
Project Description:
Vision-based localization of an agent in a map is an important problem in robotics and computer vision. In that context, localization by learning matchable image features is gaining popularity due to recent advances in machine learning. Features that uniquely describe the visual contents of images have a wide range of applications, including image retrieval and understanding. In this work, we propose a method that learns image features targeted for image-retrieval-based localization. Retrieval-based localization has several benefits, such as easy maintenance and quick computation. This is achieved by guiding the learning process such that the feature and geometric distances between images are related.
References:
Thoma, J., Paudel, D. P., Chhatkuli, A., & Van Gool, L. (2020). Geometrically Mappable Image Features. IEEE Robotics and Automation Letters.
—————————————————————————————————————————
Supervisors: Sujaya Neupane, Suresh Manandhar
Project Area: Computational neuroscience
Project ID: INT-2020-SN1
Project Title: Dimensionality and linearity of representation in the brain
Project Description: Pool all the open access (monkey/rodent/human neurophysiology) datasets made available by various labs across the world. These dataset will be from cortex or subcortical structures (thalamus, hippocampus, cerebellum etc). We focus on cortex and hippocampus.
Question – is a general principle of cortex to linearize task variables [1] and represent them in low dimension (so that readout is simple)? Unlike cortex, does hippocampus represent task variables non-linearly in high dimension? E.g representation of a 2D arena as a 2D plane embedded in n-D neural space vs as a ‘swiss roll’ in n-D neural space.
Approach – apply linear and non linear dimensionality reduction techniques in all these datasets and quantify variance explained as a function of #of dim and type of method (linear vs nonlinear: MDS vs isomap for e g ).
Prediction – intrinsic dimensionality of the data is captured by linear methods for cortex but for subcortical structures linear methods overestimate the intrinsic dimensionality. For the same dimensionality, non-linear methods explain more variance for neural data from subcortical structures (hippocampus).
References: 1.http://dicarlolab.mit.edu/sites/dicarlolab.mit.edu/files/pubs/dicarlo%20and%20cox%202007.pdf
—————————————————————————————————————————
Supervisors: Binod Bhattarai, Sharib Ali
Project Area: Natural Language Processing
Project ID: INT-2020-BB1
Project Title: Nepali Dialogue Corpus
Project Description
This project studies the existing Nepali dialogue corpus (if any), identifies their limitations and creates a new Nepali dialogue benchmark. It also explores methods to collect dialogues in a weakly supervised manner eg. conversation on Twitter or similar other platforms. Finally, the scope of the project is to apply the machine learning baseline methods to validate the generalization of data set.
References:
Lowe, Ryan, et al. “The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems.” arXiv preprint arXiv:1506.08909 (2015).
https://docs.google.com/forms/d/e/1FAIpQLSfOfmG0r67TW8DcSZDJmY1iMQ6F3y3WxtBvrODS0uoTeuhy5Q/formResponse