Project title: Construction of VR Digital Twin using Machin Vision Approach
Team: Samarth Atulkumar Patel and Priyavrat Sharma
Project guide: Prof. Pradipta Biswas and Prof. Vishal Singh
About the project:
A digital twin of working space has a profound and transformative impact. It aids in space planning, visualization, and simulation of physical office environments. Additionally, it plays a crucial role in promoting energy efficiency and sustainability within the workspace. However, several challenges are associated with effectively placing assets when reconstructing a real[1]world space in virtual reality. This thesis presents a novel machine learning-based development of digital twins, an alternative to the conventional reconstruction process. It uses 2D frames from the real world to pass through the object detection model to detect objects of interest with an accuracy of mAP of 0.74. Then an image processing algorithm is presented to calculate the orientation of the movable objects in space. The analysis shows that the proposed pose detection algorithm can estimate orientation of the objects with error rate of 8.030 . Finally, this information is passed through a novel neural network structure that maps 2D coordinates to 3D locations in virtual space. A detailed comparison study is conducted between multiple machine learning models to decide neural network as the mapping algorithm. It showed quality performance with a correlation coefficient of R2 = 0.97. This proposed pipeline is tested in different test-bed to prove its efficacy.
