3D Model Guide: Difference between revisions

From XentaxWiki
Jump to navigation Jump to search
imported>Tsukihime
imported>Dinoguy1000
m (Reverted edits by LindaRogers (talk) to last revision by 72.43.201.2)
 
(99 intermediate revisions by 10 users not shown)
Line 1: Line 1:
This is a work-in-progress and open to any edits. You may add any information that you feel would be useful, and modify any existing information that may be inaccurate or could be improved. No permission is required. Of course, as this is an open page, "If you don't want your writing to be edited mercilessly, then don't submit it here."
==Preface==
==Preface==


Line 7: Line 9:
==Introduction==
==Introduction==


Please read the Definitive Guide to Exploring File Formats before starting this guide, which is aimed specifically for 3d file formats.
Please read the [[DGTEFF|Definitive Guide to Exploring File Formats]] before starting this guide, which is aimed specifically for 3d file formats. It is assumed that you know how to read files in hex viewers.
 
No knowledge in 3d is needed, however, any experience in 3d modelling or 3d programming will help. No advance math knowledge is used in 3d formats, though it is in 3d programming, but those are usually hidden inside 3d libraries you use, like your 3d engine or your 3d modeller's scripting language. You should be able to understand what vectors and matrices are though.
 
==Common data==
 
The following a list of things that you should look out for when you're exploring a 3D format. Note that in addition to data related to 3D models, the developers may choose to store other data with it that are only useful for to their engine or scripts. This miscellaneous data typically is not important and can be skipped by your parser.
 
Each item will also indicate how they are "typically" represented. Note that very often, 8-byte, 4-byte, or 2-byte floats may be used for precision or to optimize space. But 4-byte floats are more often used, so when you're working with float data, if one type doesn't work try another.
 
===Vertices===
 
Data that is usually stored with vertices.
 
*Vertex coordinates. Three floats
*Vertex normals. Three floats
*Texture coordinates UV(W). Two floats. Three if the W-value is specified.
*Vertex colors. Four floats
*Vertex weights
*Bone indices
 
===Faces===
 
Data that may be stored with faces.
 
*Vertex indices
*Normal indices
*UV indices
*Material indices
*Face Normals (in addition to each vertex having a normal, each face can also have a normal)
 
===Bones===
 
*Bone index
*Parent bone index
*Bone name
*Bone transformation.
 
===Animation===
 
===Morphs===
 
===Materials===
 
==Exploring a Model Format==
 
===Where do I begin?===


No knowledge in 3d is needed, however, any experience in 3d modelling or 3d programming will help. No advance math knowledge is used in 3d formats, though it is in 3d programming, but those are usually hidden inside 3d libraries you use, like your 3d engine or your 3d modeller's scripting language. You should be able to understand what vectors and matrixes are though.
Naturally, if you have no experience with format reversing, you will probably have no clue where to start. Even if you do, you probably still won't know where to start. Some things that you might want to consider before tackling your own format:


We should first understand the basics of how 3d data is represented.
*Get some decent tools. Some are better than others.
*Familiarize yourself with the API you're working with
*Look at existing formats and write a parser for it
*Get a feel for various patterns that arise in various formats. After awhile these things become second-nature.


===Representation of 3D objects===
===Common Patterns===


The building block of 3d data is the vertex. A vertex is a point in 3d space. So a vertex needs 3 values: x, y and z position. The position values are usually represented as 4 byte floats, though not always.
Design patterns often arise in file formats, due to various reasons. This is also true in model formats.


Two vertices (“vertices” is the plural for “vertex”) can connect and form a line.
====Chunk-based format====


Three vertices can connect and form a triangle. Although more vertices can connect and form more coplex polygons like quads (4 vertices), triangles are the most common. In fact, the GPU needs to break more complex polygons down to triangles before processing them, so 3d formats are likely to always use triangles.
This format essentially structures all of the data into data "chunks" or "blocks", where each chunk is identified by an ID and often has a size associated with it.
But how are triangles stored in files? Three vertices (3*3 position values) are enough to create a triangle. But this is not how triangles are always generated from the vertex data in the 3d files.
Triangles can also be represented as what is called a “tristrip”, or triangle strip. In a triangle strip, the first three vertices form a triangle, then every new vertex creates a new triangle by connecting with the previous two.
Tristrips are performance optimization for some GPUs.


There are other ways to connect vertices, like linestrips or trifans (short for “triangle fan”, each new vertex after the 1st one creates a new triangle, by connecting with the previous and the very 1st vertex). However these are less common.
"In this kind of file structure, each piece of data is embedded in a container that contains a signature identifying the data, as well the length of the data (for binary encoded files). This type of container is called a "chunk". The signature is usually called a chunk id, chunk identifier, or tag identifier." -- [http://en.wikipedia.org/wiki/File_format#Chunk-based_formats Wikipedia]


Some 3d formats will use what is called an “index buffer” (index list), together with the “vertex buffer” (vertex list). Index buffer is basically a list of (usually integer) numbers which tells which vertices connect with each other by their index in the vertex buffer.
It is usually easy to identify this format because strings are usually used as the chunk ID, but numbers can be used as well.


A collection of triangles or tristrips is called a “Mesh” or “Surface”. Each file you explore will likely have more than one of these.
If you suspect that the format uses a chunk-based format, consider examining a few of the values that you believe are the chunk sizes and see if you can parse the entire file without having to understand any of the data.


Now we have a basic understanding on how 3d graphics work.
====Offset tables====


===The role of the GPU===
====Material index ranges====


GPU's main job is to process vertices. They are very good at it. It is possible to store and render 3d graphics using other methods, like voxels. But 99.9% of modern GPUs are only good for rendering 3d graphics from the data we described above and most games need the GPU to run, as CPU is too slow for rendering 3d graphics in realtime, polygonal or voxel based.
Face material information may be stored in the form of index or face ranges. For example, if a particular mesh is composed of 3000 face indices and used 3 materials, the first material might be assigned to indices 0 to 799, the second material from 800 to 2045, and the third material from 2046 to 2999 (assuming zero-based indexing). If you find yourself with some unknown integers that add up to the total number of indices or faces, you might want to try this out.


We still don't know enough about how 3d data is stored though. We talked about vertices, but we didn't talk about vertex colors, vertex normals, vertex UV maps, materials and other common data which is used in just about every game. We didn't talk much about vertex and index buffers and we also don't know how 3d animations work, what are bones and vertex weights and how they are represented.
===Techniques===
 
Some techniques that you might use to help figure out what kind of data you might be looking at.
 
====Using a calculator====
 
====Comparing multiple files====


==A simple model format==
==A simple model format==


Here is a simple model format: http://forum.xentax.com/viewtopic.php?f=29&t=3739 (temporarily just linking to an existing tutorial)
Here is a simple model format: http://forum.xentax.com/viewtopic.php?f=29&t=3739 (temporarily just linking to an existing tutorial)
==Tools==
You will need two things
*A hex viewer
*A 3D model viewer/ 3D modeller
This is the bare minimum required to get started on your model reversing journey, but your choice of tools may make your experience easier or more challenging. Of course if you are working with a text format you won't need a hex viewer, but more often than not you will be working with binary formats.
Some common tools are listed below. It is by no means an exhaustive list. Try different tools to get a feel for what you like.
Links are available at the end of the page.
===Hex Viewers===
*'''010 Editor'''
**30-day trial, US$50 license
**Comes with many useful features.
*'''HxD'''
**'''Free'''
**Simple hex editor.
*'''Hex Workshop'''
**30-day trial, US$90 license
===3D Model Viewers/ 3D Modellers===
There are many packages for model editing. There are dozens of custom model viewers. The more common ones are as follows:
*'''3D Studio Max'''
**30-day trial, US$3500 license
**Comes with its own scripting language for importing models.
*'''Blender'''
**'''Free'''
**Uses Python scripts to import models.
*'''Noesis'''
**'''Free'''
**Provides C++ and python API's for importing models.
*'''Metasequoia'''
**'''Free*''', US$45 license
**Simple 3D editor. Does not support too many things, but it gets the geometry out. The MQO format is very simple as well.
:Note that the free version comes with limited features.
==Links==
*Blender - http://blender.org
*Python - http://python.org/
*Noesis - http://oasis.xentax.com/index.php?content=home
*Metasequoia - http://metaseq.net/english/
[[Category:3D Models]]
[[Category:Tutorials]]

Latest revision as of 18:32, 14 June 2014

This is a work-in-progress and open to any edits. You may add any information that you feel would be useful, and modify any existing information that may be inaccurate or could be improved. No permission is required. Of course, as this is an open page, "If you don't want your writing to be edited mercilessly, then don't submit it here."

Preface

This page will briefly describe how to reverse a 3D model format. This will not be a tutorial on how to read hex, but instead on various concepts used in 3D computing and rendering that will help make sense of what the data represents.

For a more detailed list of 3D concepts, check out the 3D model glossary

Introduction

Please read the Definitive Guide to Exploring File Formats before starting this guide, which is aimed specifically for 3d file formats. It is assumed that you know how to read files in hex viewers.

No knowledge in 3d is needed, however, any experience in 3d modelling or 3d programming will help. No advance math knowledge is used in 3d formats, though it is in 3d programming, but those are usually hidden inside 3d libraries you use, like your 3d engine or your 3d modeller's scripting language. You should be able to understand what vectors and matrices are though.

Common data

The following a list of things that you should look out for when you're exploring a 3D format. Note that in addition to data related to 3D models, the developers may choose to store other data with it that are only useful for to their engine or scripts. This miscellaneous data typically is not important and can be skipped by your parser.

Each item will also indicate how they are "typically" represented. Note that very often, 8-byte, 4-byte, or 2-byte floats may be used for precision or to optimize space. But 4-byte floats are more often used, so when you're working with float data, if one type doesn't work try another.

Vertices

Data that is usually stored with vertices.

  • Vertex coordinates. Three floats
  • Vertex normals. Three floats
  • Texture coordinates UV(W). Two floats. Three if the W-value is specified.
  • Vertex colors. Four floats
  • Vertex weights
  • Bone indices

Faces

Data that may be stored with faces.

  • Vertex indices
  • Normal indices
  • UV indices
  • Material indices
  • Face Normals (in addition to each vertex having a normal, each face can also have a normal)

Bones

  • Bone index
  • Parent bone index
  • Bone name
  • Bone transformation.

Animation

Morphs

Materials

Exploring a Model Format

Where do I begin?

Naturally, if you have no experience with format reversing, you will probably have no clue where to start. Even if you do, you probably still won't know where to start. Some things that you might want to consider before tackling your own format:

  • Get some decent tools. Some are better than others.
  • Familiarize yourself with the API you're working with
  • Look at existing formats and write a parser for it
  • Get a feel for various patterns that arise in various formats. After awhile these things become second-nature.

Common Patterns

Design patterns often arise in file formats, due to various reasons. This is also true in model formats.

Chunk-based format

This format essentially structures all of the data into data "chunks" or "blocks", where each chunk is identified by an ID and often has a size associated with it.

"In this kind of file structure, each piece of data is embedded in a container that contains a signature identifying the data, as well the length of the data (for binary encoded files). This type of container is called a "chunk". The signature is usually called a chunk id, chunk identifier, or tag identifier." -- Wikipedia

It is usually easy to identify this format because strings are usually used as the chunk ID, but numbers can be used as well.

If you suspect that the format uses a chunk-based format, consider examining a few of the values that you believe are the chunk sizes and see if you can parse the entire file without having to understand any of the data.

Offset tables

Material index ranges

Face material information may be stored in the form of index or face ranges. For example, if a particular mesh is composed of 3000 face indices and used 3 materials, the first material might be assigned to indices 0 to 799, the second material from 800 to 2045, and the third material from 2046 to 2999 (assuming zero-based indexing). If you find yourself with some unknown integers that add up to the total number of indices or faces, you might want to try this out.

Techniques

Some techniques that you might use to help figure out what kind of data you might be looking at.

Using a calculator

Comparing multiple files

A simple model format

Here is a simple model format: http://forum.xentax.com/viewtopic.php?f=29&t=3739 (temporarily just linking to an existing tutorial)

Tools

You will need two things

  • A hex viewer
  • A 3D model viewer/ 3D modeller

This is the bare minimum required to get started on your model reversing journey, but your choice of tools may make your experience easier or more challenging. Of course if you are working with a text format you won't need a hex viewer, but more often than not you will be working with binary formats.

Some common tools are listed below. It is by no means an exhaustive list. Try different tools to get a feel for what you like. Links are available at the end of the page.

Hex Viewers

  • 010 Editor
    • 30-day trial, US$50 license
    • Comes with many useful features.
  • HxD
    • Free
    • Simple hex editor.
  • Hex Workshop
    • 30-day trial, US$90 license

3D Model Viewers/ 3D Modellers

There are many packages for model editing. There are dozens of custom model viewers. The more common ones are as follows:

  • 3D Studio Max
    • 30-day trial, US$3500 license
    • Comes with its own scripting language for importing models.
  • Blender
    • Free
    • Uses Python scripts to import models.
  • Noesis
    • Free
    • Provides C++ and python API's for importing models.
  • Metasequoia
    • Free*, US$45 license
    • Simple 3D editor. Does not support too many things, but it gets the geometry out. The MQO format is very simple as well.
Note that the free version comes with limited features.

Links

Retrieved from "https://wiki.xentax.spektr.name:443/index.php?title=3D_Model_Guide&oldid=39091"