Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving

Matúš Dopiriak, Jakub Gerec, Juraj Gazda

Reconstruction of Photorealistic 3D Urban Scenes Using Radiance Fields as Digital Twins for Autonomous Driving

Číslo: 4/2024
Periodikum: Acta Electrotechnica et Informatica
DOI: 10.2478/aei-2024-0015

Klíčová slova: autonomous driving, CARLA simulator, digital twin, radiance fields, vehicular metaverse

Pro získání musíte mít účet v Citace PRO.

Přečíst po přihlášení

Anotace: We explore the use of radiance fields (RFs) to reconstruct photorealistic 3D urban scenes, creating digital twins (DTs) for autonomous driving (AD) by leveraging Nerfacto and Splatfacto models integrated with the CARLA simulator. Our research demonstrates that publicly available RFs can be utilized through Nerfstudio library to create photorealistic urban scenes and extract arbitrary images based on the camera pose. These scenes can serve as simulations for AD or as DT repositories for static environments within the vehicular metaverse. Additionally, we quantitatively evaluate RF models and use masking to remove dynamic objects, successfully simulating real-world scenarios. Quantitative evaluation shows that the Splatfacto model achieves a peak signal-to-noise ratio (PSNR) of up to 26.40, a structural similarity index measure (SSIM) of 0.84, and a learned perceptual image patch similarity (LPIPS) score of 0.21, consistently outperforming the Nerfacto model.