3D Models

What is a scene?

Structured: Geometry + Materials

More structured: 3D models

* A Scene: a combination of 3D Models (+lights +cameras)

3D models can be complex

* One Firetruck: **13 sub meshes and 11 shaders** * Sometimes we have to work with less optimized 3D models * Sometimes 3D models are just really complex

Abstract over 3D models

* Simplifying scene graph by directly referring 3D models. * Content of 3D model is a ***blackbox***

And then there are dynamic 3D models

* Dynamic 3D models come with animations * Straight forward instancing => **identical animation** * Limited applications

Enter: Configurable Instances

* Make **current pose** configurable **per instance** * Individualized animations * Essential for VR

Configuration of Appearance

* Customize appearance * Varying clothes, hairstyle * Change colors, textures, replace geometry * Again: very relevant for VR

Finally: We want to be
fast and efficient

* Capable to instantiate a large number of 3D models * Minimize memory usage * Fast initialization of scene

A list of things we want

* Instancing of 3D Models * Abstraction over content * Configurable (animation pose, color and more) * Efficient instancing

State of Instancing

In Declarative 3D for the Web

Data Level Instancing

* Reuse data buffers attached to geometry * vertex positions, normals, texcoords etc. * Essential to save memory * Supported by XML3D/X3DOM

Problems with Data Level Instancing

* We still need to replicate the structure * Geometry, shader attachments etc. * No proper **abstraction** over content

Sub Graph Instancing

* Instantiate whole subtree of transformation hierarchy * Includes structure and higher level elements (animations) * Supported by X3DOM via DEF/USE * No support in XML3D * Problems * DOM Tree versus DAG * Event Targets? CSS Inheritance? * Similar: X3D Import

Problems with Sub Graph Instancing

* Not configurable * Can only be used for static 3D models

Facade Patterns

* Encapsulate functionality behind facade * Compact interface * Fully Configurable * Examples * X3D prototypes for arbitrary scene graph content * XBL, Web Components for DOM * Not available (yet) in Dec3D

Problems with Facade Patterns

* More than sufficient for configurable instances * But: too general, too complex * Most practical implementation: internal replication of content * Instantiation not quite as efficient

State of Instancing Summarized

* We still want configurable instances * But we want to have it more efficient

What about formats?

* COLLADA, glTF * Provided detailed content definition * No interface to access and configure content * Open Question: How to embed in te DOM? * A good instantiation mechanism could be **compatible** with these formats

Our Approach

Configurable Instances in XML3D

Very basic geometry in XML3D

<xml3d> <transform id="xfm" translation="5 6 0" /> <group shader="shaders.xml#wood" transform="#xfm" > <mesh src="geometry.xml#bench" /> </group> </xml3d> 

A static 3D model in XML3D

* Yes. It's confusing. * Let's simplify!

Step 1: Propagate Shaders

Step 2: Flat Transform Hierarchy

Static 3D Model simplified

* A list of meshes with 3 properties * shader * transformation * data

Introducing: asset and assetmesh

* mesh => assetmesh * Wrap everything into new element * Voila: An asset for static 3D models

Introducing: model

<xml3d> <group transform="#leftTransform" > <model src="asset.xml#firetruck" /> </group> <group transform="#rightTransform" > <model src="asset.xml#firetruck" /> </group> </xml3d> 

* Use the model element to instantiate an asset * Voila: Instancing of static 3D Models

Static 3D Models: Checked

* That was easy, now lets look at **dynamic** 3D models

Dynamic 3D models in XML3D

Simply wrapping in asset?

* Key value is the same for all instances * Only synchronized animation

Introducing: assetdata

<asset id="baseAsset" > <assetdata name="base" compute="dataflow['#skinning']" > <data src="#meshData" /> <float name="key">0.0</float> </assetdata> <!-- ... --> </asset> 

Linking assetdata to assetmesh

<asset id="baseAsset" > <assetdata name="base" ... > ... </assetdata> <assetmesh includes="base" shader="#shaderA" src="#indexDataA" /> <assetmesh includes="base" shader="#shaderB" src="#indexDataB" /> </asset> 

Extend asset

<asset id="customAsset" src="#baseAsset" > <assetdata name="base" > <float name="key">42.0</float> </assetdata> </asset> <model src="#customAsset" /> 

Extend within model

<model src="#baseAsset" > <assetdata name="base" > <float name="key">42.0</float> </assetdata> </model> 

Configuration via extension

* Override base data by extending asset within model * Can also override assetmesh properties * Shader, transformation etc.

Dynamic 3D Models: Checked

Implementation and Performance

Avoid DOM duplication per instance

* Asset definition very minimalistic * Straight forward mapping to list of meshes with shaders and dataflows * No need to replicate DOM structures on instantiation * Work on internal structures instead

Out testing models

* Fire Truck (static, 41 nodes) * Natalie (dynamic, 11 nodes) * Sniper (dynamic, 29 nodes)

DOM Size

Memory Peak

Page Load Time (Seconds)

How about updating instances?

* We need to continuously update instances * Example: animation key * We avoid reconstruction of asset data connections * Updates automatically handled by Xflow

Instance update performance

* Performance of skeletal animation * Computation in JavaScript

Use Cases

Collaborative Content Creation

* Use of static 3D models * With instancing: improved performance

Virtual Worlds

* Dynamic 3D models for avatars * Customizable appearance

Conclusion

Contribution

* New Mechanism for configurable instances of 3D Models * Small extension (4 DOM elements) * Abstraction for complex 3D Models * Configurable via extension * High flexibility with generic dataflows (Xflow) * Efficient!

Future Work

* Combination of multiple assets * Use of hardware instancing functionality * Instantiate 3D models by providing an array of transformations * Show certain parts of the 3D model * Filter assetmesh by class attribute (like CSS)