基于二维图像集合的三维内容学习与生成

童欣（xtong@microsoft.com)

• Deep learning has demonstrated its great advantages in many fields
• Unified network architecture for different representations and tasks
• Compact non-linear approximator for arbitrary functions

Learning 3D representations from 2D image collections

• Exploiting huge amount of images/video datasets
• Learn high quality 3D representation from realistic images
• From image to 3D representation is a highly ill-posed problem
• No multiple views and no correspondence between images

Two Solutions

Learning by synthesis

• Learning a network to infer
• underlying 3D representation from each 2D image
• The rendering of the derived 3D representation should match the input 2D image
• How to solve the ill-posed inverse rendering problem?

Learning by discrimination

• Learning a generative model of the space of 3D representations from 2D images.
• The rendering of generated 3D representation from each view should be indistinguishable from 2D real.
• How to minimize the quality gap between synthesized images and real ones and preserve 3D consistency?

采用多幅图像对三维人脸进行生成

• 3D scene layout generation from RGB/RGBD image collections(Mingjia Yang, Yuxiao Guo, Bin Zhou, Xin Tong, Indoor Scene Generation from a Collection of Semantic-segmented Depthemages,ICCV 2021.)
• 3D object texture generation from real image collections(Rui Yu, Yue Dong, Pieter Peers, Xin Tong, Learning Texture Generators for 3D Shape Collections from internet Photo Sets, BMVC 2021.)

陈宝权教授的三位讲者报告：

Simulation and Optimization of Magneto elastic Thin Shells

JOINT NEURAL PHASE RETRIEVAL AND COMPRESSION FOR ENERGY-AND COMPUTATION-EFFICIENT HOLOGRAPHY ON THE EDGE

复杂三维形状的生成、分析与制造

轻质与可控弹性耦合的微几何结构仿真设计与优化

基于深度学习的无标签高泛化的快速均质化方法

面向制造的高保真几何结构优化：

提出了制造约束与复杂几何结构的共分析方法，解决增材制造中因打印精度、局部重力、分块拼接等造成的几何形状走样，克服了大体积或高度复杂结构如植物、毛发等在制造中的失真问题。

三维物体跟踪与增强现实（钟凡）

基本原理

PnP(Perspective-n-Points)问题

SIFT基于局部特征得到点对应

无纹理物体——利用轮廓形状信息（边缘信息与图像信息）

三维位姿优化

混合优化方法：只对面外旋转采用非局部搜索

可视化

BoxAR

tableau

Canis - A Language for Data-Driven Chart Animations

Deep Dive: Sampling for Progressive Visualization

https://kd-box.github.io

Deformation of 3D Printed Soft Robots: Sensing, Simulation and Control(Charlie C.L.Wang)

多模态信息感知的SLAM

特征点优化的相机与激光外参数标定

两阶段视角不变匹配法的激光SLAM

基于凸优化与IMU-KLT线段跟踪的点线SLAM

SLAM特征跟踪中存在问题:
在弱纹理、重复纹理、运动模糊、光照变化下，鲁棒性和精度较低
基于线的跟踪速度较慢

解决思路∶

• 提出了点线特征混合策略:Point line with shorter tracks are categorized into “MSCKF”features and with longer tracks into ““SLAM”features.Especially，“SLAM” lines are added into the state vector toimprove accuracy of the proposed system.
• 提出了基于L1-范数图优化的线段跟踪
• 实验效果优于VINS-Mono，PL-VINS，OpenVINS
• 在运动模糊和弱纹理场景下，实验效果优于ORB-SLAM3.

基于深度哈希学习的SLAM闭环检测与位姿误差纠正
基于随机森林学习与二值描述子学习相融合的并行搜索SLAM重定位
软硬协同SLAMI展示和展望

严肃化和泛化的游戏引擎技术

游戏化的xr工作——GritWorld

异构维度数据可视化应用的高效一体化构建方法（HUAWEI）

1.Aladdin定位:面向WEB应用的端到端智能无代码人机协同系统构建平台

1. 生成既有静态页面、交互、业务逻辑与数据表达的全场景WEB应用
2. 彻底打通从需求到设计到开发的全链路

2.Aladdin二维组件编辑方案:声明式语法、交互构建界面

3.Aladdin UVE Kernel三维编辑器

Taichi编程语言

1
2
3
4
5

#使用
pip install taichi
#在命令行中调用
ti gallery
#可以得到一个demo体验

作用：在图上优化代码

OpenGL中文文档