The Reality Labs (RL) Research Team brings together a world-class team of researchers, developers, and engineers to create the future of AR and VR. A key problem in processing input data of a human subject is finding dense correspondences (registration). Recently, a new class of methods has been opened up via 3D Gaussian Splatting: GitHub - JonathonLuiten/Dynamic3DGaussians. However, when applied in more challenging scenarios of longer-term correspondences, previous methods can easily get trapped in suboptimal solutions. Various heuristic methods have been utilized in the past, such as the usage of image pyramids or blurring, but these do not fully solve the problem and require ad-hoc tuning. The goal of this internship is to dive into this problem including its theoretical aspects, e.g., is there a useful class of quasi-convex loss functions that would yield more robust tracking? Research Topics: * Virtual humans, animatable avatar creation * Dense motion tracking (as opposed to sparse feature/keypoint methods) * Differentiable rendering via 3D Gaussian Splatting * Machine learning / AI Our internships are twelve (12) to twenty four (24) weeks long and we have various start dates throughout the year. Research Scientist Intern, Robust Tracking via 3D Gaussian Splatting (PhD) Responsibilities
- Design and execution of algorithms.
- Investigate, design and develop novel algorithms in areas such as body capture, tracking, reconstruction, generative models for clothed humans and deformable objects (soft tissue, clothes, hair).
- Prototyping, building and analysis of experimental systems.
- Collaboration with and support of other researchers across various disciplines.
- Communication of research agenda, progress and results.
- Currently has, or is in the process of obtaining a PhD in Computer Vision, Graphics, Machine Learning, or related field.
- Experience with computer vision, machine learning, 3D geometry and linear algebra.
- Experience or interest to learn 3D Gaussian Splatting.
- 2+ years of experience with Python and PyTorch.
- Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment.
- Intent to return to a degree-program after the completion of the internship.
- 2+ years of experience with one or more of the following areas: developing models for neural human reconstruction, human body learning, dynamic shape representation, numerical optimization, deep learning.
- Experience working and communicating cross functionally in a team environment.
- Proven track record of achieving results as demonstrated in accepted papers at top computer vision, graphics, machine learning or robotics related journals and conferences such as CVPR, ICCV, ECCV, NeurIPS, SIGGRAPH, SIGGRAPH Asia etc.
- Experience with C++.
- Interpersonal skills: cross-group and cross-culture collaboration.
Zurich / Switzerland
