Real Time Image 3D Conversion using DCGAN wit...

Real Time Image 3D Conversion using DCGAN with Depth Map

Author Name : Lalita Panika, Rimjhim Manu, Sahil Raju, Rahul Verma

DOI: https://doi.org/10.56025/IJARESM.2024.121124144

ABSTRACT This project introduces a novel pipeline for real-time 3D image generation by augmenting Deep Convolutional Generative Adversarial Networks (DCGANs) with depth map information. The process begins with live image capture through a webcam, where each frame is processed to extract depth data using the MiDaS pre-trained Monocular Depth Estimation model. The depth map, after being normalized for clarity, is combined with the corresponding RGB image to form a four-channel input. This input is then fed into a DCGAN, which employs a high-resolution generator capable of producing detailed, 3D-like images. The generator is pre-trained on a custom dataset, allowing it to refine latent vectors and generate realistic 3D renderings. By incorporating both depth and color information, this method enhances spatial understanding, offering more accurate 3D representations. The pipeline's output is both displayed in real-time and saved for further analysis. This approach not only showcases the integration of real-time image capture, depth estimation, and generative modeling but also highlights its potential applications in fields such as 3D modeling, augmented and virtual reality (AR/VR), and computer vision research. The versatility of this method, capable of generalizing across diverse scenes and objects, ensures its broad applicability in various industries