AR-SLAM PROJECT

This project aims at introducing Augmented Reality into the park touring carts. The key question is simultaneous localization, including 3d coordinates and azimuth. Thus, the first solution goes for the ORB-SLAM, while the second goes for the integration of UWB and imu.

AR Cart AR Prototype Two Stereo Slam

V1  |  3 Seat AR Cart

  • Date:   2021 Aug-SEP
  • Type:   Grounded Project
  • Role:   Algorithm Test; Site Deployment and Test
  • SoftWare:   UE4 in win10; ORB-SLAM2 in WSL; ROS
  • HardWare:   GTX1650; i9-8950HK; Cart
  • Description:   To add AR to the tour carts running in the park, under the guidance of Dr. Li, I worked as an algorithm team member focusing on SLAM tests, joint debugging, and on-site tests. The goal is to find the SLAM parameters suitable for large-scale outdoor spaces, and the direction of subsequent iterations.

Camera Selection: The D435i camera has two Infrared cameras, one RGB camera, and one Infrared projector together with a built-in imu. Thus it can support the binocular SLAM, and use the imu to calibrate the position.
As we are using this camera outdoors and the distance of the IR projection is limited, this projector is disabled.

SoftWare Framework: The UE4 can only run smoothly in the Windows environment, however, the SLAM map saving and reloading can only be executed in the Linux environment. Therefore, we use the WSL (Windows Subsystem for Linux) to run the ORB-SLAM2, and ROS as the bridge to transmit data.

Problem

At the hardware level, the D435i stereo cam only achieves a 60° horizontal viewing angle, thus there is a risk of failure in map building when moving around places with many tourists nearby. Meanwhile, at the software level, instead of the dense point clouds, ORB SLAM uses keyframes and sparse point clouds to store the map. The content of each keyframe is strongly disturbed by the changing weather and sunlight.

V2  |   AR Prototype

  • Date:   2021 JUL-AUG
  • Type:   Prototype
  • HardWare:   LCD screen; Keyboard; Microcomputer; Lithium batery; Sheet metal; One little trolley.
  • Description:   One little fun cart that integrates various devices to be tested.

Tested message: On this cart, we put different numbers and types of cameras onto this cart, to test various kinds of SLAM, visual part and imu part, indoors and outdoors.

V3  |   Two Stereo Slam

  • Date:   2021 JUL- 2022 MAR
  • Type:   Research Project
  • Role:   Algorithm Developer
  • Github:   ORB_SLAM2_2_STEREOSGithub Link
  • SoftWare:   ORB-SLAM2; ROS; Ubuntu18
  • HardWare:   Two D435i cameras; Graphic workstation.
  • Description:   The original ORB SLAM2 does not contain map saving and reloading. Thus, based on the work of BoomFan Github Link, I added one more stereo camera to this SLAM project, making it possible to simultaneously model both front and rear environments.