As mentioned previously, I re-used the frame handling code from Tom
Box's implementation of motion tracking. I re-coded the tracking algorithm
myself. To track a point, the co-ordinate of the starting point is given
to the program. The values of the pixels in a box surrounding the point
are then read. The next frame is then loaded, and the box is then moved
around the original co-ordinates, or "jittered". The values are compared
to the original box, and the co-ordinates for the box which is closest
in value are taken.
The images are output as another series of jpegs, but with the tracking
point highlighted in green so a program such as fcheck can easily be used
to see the results of the tracking. The system is not perfect, so a method
of manual keyframing needs to be introduced. I implemented this in the
form of a text file which is read in by the program. The user can type
in the frame number, and the x and y co-ordinates, and the program will
override its own values and use these in its place. Finally, the co-ordinates
for each of the frames are then output into a text file.
Program 1.
I wrote another program for the second part of the process. Once all
of the points have been tracked, we need to concatenate all the data into
one file. Also, we need to make the co-ordinate positions relative to the
nose point, as this will eliminate any movement of the face. The second
program does this, and again, outputs a text file.
Program 2.
We can now take this data into Maya, and displace the mouth with it.
To simplify this, an instance of the head is created, which we can work
on without it moving around, and any changes made to it will affect the
original. The first Mel script reads in the motion capture data, and applies
it to a polygon with 10 vertices. This is preferable to applying the data
directly to the model, as we need to change to scale and position of the
co-ordinates relative to the size of the mouth. This is far more easily
done interactively in the workspace by moving the polygon around rather
than by scaling the co-ordinates in the program.
Mel Script 1.
The second Mel script takes the vertex positions from the polygon and
applies them to the face. As the motion tracking points were affixed outside
the lips in the original video, and more than one vertex is controlled
by each tracking point, we need to be able to give an additional displacement
value to each vertex, to make sure they are in the right position.
The other part of the face which is deformed is the chin. As there are
many vertices which need to be moved by one tracking point, it is inefficient
to work out additional displacement values for each point on the chin.
Instead, we work out how much the chin control point has moved between
frames, and apply this displacement to every vertex on the chin. For all
the displacement on the face, only the x and y values are taken in to account,
the z value remains the same as the original model.
Mel Script 2.
The program worked quite well, and despite the fact that the source
material was produced using household equipment, relatively few keyframes
needed to be set. I found that some points tracked better using the original
images, and some tracked better if I applied a threshold filter to the
video.

Continue to Blendshapes.
Home. |