BEHIND THE SCENES

COME ON IN

                                     ... or download Legato 2020

WHERE I COME FROM WHERE I'M NOW WHERE I'M GOING
From a very young age I have been very interested in music, animation, photo- graphy, electronics, and computers.

One important event must have been early morning at Saturday december 6, in 1969.

I was three years old, and that summer we had just moved to the countryside, far far away from family and friends.
It was the day after Sinterklaas-eve (Saint Nicholas), when all Dutch children get presents ... or are deported to Spain (where he lives of course), if they had been naughty that year.

More


Fortunately that didn't happen to me so we were able to inspect our gifts that morning, and there it was: the L.Z.K. Magic Lantern with Disney slide films. These toy projectors were manufactured in Poland (Google "Lodzkie Zaklady Kinotechniczne" or "Diaskop Bajka"). Together with my brother and sister we watched slides from Disney's hand-drawn Snow White and the Seven Dwarfs and I couldn't believe my eyes. Instant fan!
In the years thereafter we eagerly read Donald Duck magazines on a weekly base. Donald is still one of my favourite characters.

My first two books at the age of six: one about a cute penguin, and one about computers. Linux didn't even exist (Unix v1 did).

At the age of nine I started learning to play organ and after a few years I could also start learning to play piano.

After keyboards, samplers and synthesizers I mainly use my digital Roland FP-5 piano and digital B3-style Hammond XK-3c organ in combination with Propellerhead Reason, a virtual music studio with all the instruments and effects equipment we could only dream of in the seventies.
The Pitch Edit mode in Reason, for monophonic sources, is ultra cool and allows me to finetune my voice and prevents birds from falling dead from the sky. It offers new possibilities for self development.

Learning to sing is much like learning to produce narrations. You think you can talk, but you only hear how bad it is when listening to a recording.
During recordings you need to suppress your accent(s), articulate well, control your breathing, and on top of that you pay attention to emphasis, accents, tune, sound, rhythm and speed.

And then the sound of your voice seems completely different each day and depends on mood and physical condition like fatigue and even the amount of milk or caffeine. Even after so many years, every production is another challenge, at least for me. The good news is that you do learn how to sound less bad. Besides that it's a lot of fun.

Photography started early with my own camera and a b/w doka. When I could afford it, that b/w doka was replaced for a color doka. It proved to be a forerunner for my video activities, much later.

Electronics was initially practically limited to simple DIY projects and lot's of electronics magazines and books, many were related to musical instruments, but that changed with avionics training in The Hague. Nowadays I hardly touch a scope or soldering iron.

On the Commodore-64 I learned to program in Assembler and I bought my first 286 PC in 1987 that I used to learn programming in C and C++.

To produce a hand-drawn movie, you need lot's of drawing skills and, in those days, expensive 8 mm film equipment so that was not really feasible. When DVD was introduced I bought every nice animation production on the market - I still do!

Then I bought the book Adventures in Ray Tracing and I was instantly in love with the 3D computer animation technique that let you render images on a computer, based on mathematical formulas. The included computer program POVCAD (it still exists) needed about one night on a 486 computer to render a marble ring. A few weeks after that, I read an article about LightWave 3D v4 in a magazine and I decided to buy the program on sight. It wasn't cheap, but not as expensive as others.

Then suddenly there was Pixar with Toy Story. I was very exited to see so many 3D features, that were also available in LightWave, in a fully 3D animated movie.

This was the start of my love affaire with LightWave 3D and it still continues after soo many years. Learn more about the history of LightWave 3D on YouTube or Wikipedia or in the Appendice of the LightWave 3D 2019 documentation. In those days ILM stood out with their first Jurassic Park movie and Pixar expanded their business. With every movie production, their results (and our tools) became better and better.
At work, over the years I moved from Avionics to IT to 3D to AV, Narrations and VFX to Interactive 3D. At home, there is also music and dance. Combined together, it is a vibrant cocktail of activities and possibilities that offer eternal inspiration.

I'm preparing for an independent 3D animation project that includes singing, story development, acting and voice acting, and of course animation.

To support this production I'm developing my own tools, LW-link for Vegas Pro and Legato for LightWave 3D.

More
View this introduction to Legato for a quick demo of one of its most important features: transferring keys from a compatible character in an external scene file that acts as a library.




Producing entirely in 3D would be overly ambitious. Instead, all backgrounds will be VFX compositions with footage from cameras. See also this early exercise.




Nowadays I'm experimenting to improve skills and help in hatching a number of ideas. There are still a few older exercises on my Vimeo page.

I can relax in our garden at Amstelglorie, far away from wall outlets and large computer screens, and we love to visit the theater at Het Concertgebouw and the Nationale Opera & Ballet in Het Muziektheater.

Mail   Vimeo   LinkedIn

I want to produce at least one animated short before I die. And I hope to pursue the necessary skills from the animation style of the 1940s which I love so much. Even if I fail, I really hope to have fun en route! Jump to project status

Adam Backwards is the 3D character that I've had in my head for a long time. Although he is sometimes underestimated by his small size, it often turns out to be his great strength. His somewhat unusual solutions are not always well received, but he always means well.

Welcome to the universe
of Adam Backwards!






IN THE BEGINNING


THERE WAS NOTHING


Sooner or later I would design a figure that would have the potential to captivate the public for a longer period of time. An original figure that was completely mine, protected by Dutch and EU copyright law. A figure that could grow with my skills and ambitions. With a 3D model that didn't take ages to render.




ONCE UPON A TIME


ON A COLD WINTER EVENING


ADAM WAS BORN




My first Adam character in 2005.
Copyrights © 2005- by V.D. Mesman, Amsterdam, The Netherlands, EU. All rights reserved.


Adam was born on a cold Tuesday in the winter of 2005, in LightWave 3D 8.2, installed in our home on the fourth floor of a monumental warehouse in the historical city center of Amsterdam. In the soft light of romantically shimmering canal lights of the Oude Schans, we named our little guy Adam_v000.lwo. He weighed a healty 57 kB and was saved on January 25, 2005 at 21:34 CET.

The most important characteristic of Adam that I wanted to give him is genuine kindness. And so it happened.
In the years that followed he was able to develop freely. He may occasionally be a bit hot-tempered (of course I reject any resemblance to myself), but that's just what balances his character. You have been warned: no jokes about his nose! It's a delicate subject.

In the years thereafter he grew into a cheerful boy and his model received many changes and improvements. These refined details give him a richer range of emotional expressions. Thanks to his rig, a skeleton that drives his skin, Adam magically comes to life.


My latest Adam character in 2020.
Copyrights © 2005- by V.D. Mesman, Amsterdam, The Netherlands, EU. All rights reserved.


Adam enjoys that life and developed many interests. He loves science and technology, and he likes to design (anything!) in order to solve problems or to make life easier.

Adam also enjoys all kinds of music but he is intensely passionate about opera singers, and he likes to scare his neighbours with his own music!

His original rig has been replaced since the introduction of Genoma in LightWave 3D. I've added custom parts to control the facial muscles. In time his rig will become more complex, depending on animation needs and technical challenges.

Experiments with fur made it clear to me that such techniques would distract too much from my initial goal: learn 3D character animation. Maybe I'll add some fur when he grows up! That would make sense, doesn't it?

The same applies to his wardrobe. For now I'm afraid that clothing is optional.
I promise to add some clothing in the future but first I'll be working on his moves!

WITH A REAL WORLD PURPOSE

BACKGROUND FOOTAGE

The world around us naturally produces rich images so modeling an entire virtual world for my purpose seems to be overly ambitious. Camera images provide a much more practical and flexible solution. In addition, identifiable locations can have a positive influence on a production.

STORYTELLING

The interaction between the virtual world and the real world can trigger many interesting plot turns, or motion details that strengthen credibility. It makes it easier to respond to current local issues.

FRIENDLY AS ALWAYS

character

In the future, Adam needs an antagonist with opposing interests, and a sidekick who supports him. For now I'll limit myself to the development of Adam as a protagonist. As stated above, Adam's dominant characteristic is his genuine kindness.


HIS NOBLE APPEARANCE

ADAM'S FACE




Adam's facial rig needs more attention but even then it's easy to spot how well
he comes to life, just by slightly changing the controls in the area of the eyes.


I wanted Adam to have a very expressive face that was not necessarily photo-realistic and could withstand explosive emotion changes. At the same time, his facial muscles should be easy to read so that the subtle movements do not lose their power of expression. See below for yourself how much influence the eyelids and eyebrows alone have.

Genoma is LightWave's fully extensible autorigging system that converts its rig definition (hidden in the model itself) into a useable rig, after loading the 3D model into a 3D scene.

Since Genoma doesn't have ready-to-use presets for facial rigs, I've developed one of my own. The controls of his facial muscles remain in neutral position while their rotation is driving the rotation of a bone that has been parented to a null inside the skull.

The result is that the bone tip seems to be driving a piece of skin that's sliding over the skull, without penetrating the skull. This is important to prevent the mass of the head from ever changing with every muscle movement.



Within limits, the controls work pretty good and the deformations are not too bad. Future improvements will be focused on assisting with extreme expressions, like pouting lips, improved flexibility of the cheeks, and better deformation near the hips. Also the mouth corners are too thin and could use more controls for precise skin manipulation.

A DEVINE BODY

ADAM'S BODY




Extreme poses easily identify weaknesses.

Adam's expressive head is working reasonably well up close, but for communication at a greater distance he needs a flexible body that does not distort with extreme poses or natural limitations, and has to distort when he needs it. For example, to make his poses more interesting and energetic with a clear Line of Action.

The Line of Action is the curve that defines the meta shape of a body. It accentuates motion, speed, gesture, and intent. Watch this video made by Matt Jones to get a better understanding about Line of Action.
Can you spot the strongest pose in these examples? Note that the weaker poses lack a clear Line of Action.

These are experiments with Adam's body rig, a Genoma preset enhanced with custom built joint type deformations that preserve body mass. The messy deformation issue with his lower body is on my to-do list. I'm still discovering the possibilities and limitations of this rig to figure out a best approach.

HIS BREATHTAKING


TENDER VOICE


lip-sync speech



While talking, lips and cheeks should only express a phoneme or a phoneme transition on the syllable level. Expressing every single vowel or consonant would result in a very hyper active style of talking that is not very realistic.
Also, the lips and surrounding muscles must express a constant certain mass and associated inertia. But even then the result is somewhat mechanical. Something seems to be missing.

It's the emotional expression that pulls you into the story!
At least if there is a story to tell.

Energy and vocal cues are coming from the voice actor on the voice track. For this reason it's much more productive to start with lip-sync (jaws first) to get the timing of the underlying emotions and goals and to drive the poses and the story. Then enhance the performance with physical acting, and finish with refining the lip-sync.

HIS NERVE WRACKING


STUNTS PROVE IT


   +++   WORK IN PROGRESS   +++   examples will be added   +++   

BASIC PRINCIPLES

The famous list:
squash & stretch
anticipation
staging
straight ahead action & pose to pose
follow through & overlapping action → successive breaking of joints
ease in & ease out
arcs
secondary action
timing
exaggeration
solid drawing
appeal

ADDITIONAL PRINCIPLES

characterization
drag
lead & follow
twinning
weight & balance
moving holds
double take
snap
gesture (posing) & line of action


   +++   WORK IN PROGRESS   +++   examples will be added   +++   

APPROACH

intention - move elements with a reason
isolation - prevent unintended movement
character evolution - does this action bring your character closer to its goal with this action?
simplest statement of the main idea? - is it overly comprehensive?
clear story points - does it add anything to your story? Is it functional? Will you come back to it later? Does everyone get it? Is it suitable for the target group?
going too far - are you trying to do something that you shouldn't do?
scene planning & research - video references, use a mirror
checks - silhouette, mirroring, reversed playback


COMMON MISTAKES

blocking - missing important breakdowns between extremes
ease in/out - mass related, starting from the root
arcs
spacing
one frame direction change
one axis movement
repetition
weight & balance
offset - twinning, not moving as one unit, moving things pose to pose
boring/cliche silhouettes - pointing, W-poses, twinning, ...
polishing - basics, deadline, ...



   +++   WORK IN PROGRESS   +++   examples will be added   +++   

WHILE DISCOVERING


THE RIGHT TOOLS


LightWave 3D 2019
lightwave3d.com

LightWave 3D is a 3D computer graphics program developed by NewTek. It has been used in films, television, motion graphics, digital matte painting, visual effects, video game development, product design, architectural visualizations, virtual production, music videos, pre-visualizations and advertising.

I've been a loyal LightWave 3D user since v4.5 of this software in 1995. From the start, LightWave was known for its affordability, excellent render quality and ease of use.

Many customers are very satisfied that NewTek has never succumbed to the much hated software subscriptions.

Simply add missing functionality through other software, or write your own plug-ins, and you have an excellent 3D package to earn your living.

For me, LightWave remains the center of my 3D modeling and animation activities.

In the future I'd like to see more improvements in character animation and GPU rendering.

I wrote Legato, a plug-in for LightWave. It contains a range of tools to support character animation processes. For details, visit my (latest) Legato page or (slightly outdated) Tools page and choose LW-link.

After a few restless days on the NewTek forum while the announcement of a new version was still missing, the release of LightWave 2020 has been mentioned (as in unofficially announced) in their announcement of LightWave 2019.1.5., which contains just a few bug fixes.

Reason Suite 11
reasonstudios.com

Reason is a digital audio workstation (DAW) for creating and editing music and audio. The software emulates a rack of hardware synthesizers, samplers, signal processors, sequencers, and mixers, all of which can be freely interconnected in an arbitrary manner. Reason can be uses either as a complete virtual music studio or as a set of virtual instruments to be used with other sequencing software in a fashion that mimics live performance.

I've been a loyal Reason user since v6 of this software in 2011, when they introduced the Neptune pitch corrector, with results similar to Auto-Tune. Not only suitable for (live) pitch correction. It can also use (polyphonic) midi notes to sing in harmony. In version 9 Propellerhead introduced the Pitch Editor, with results similar to Melodyne. Simply record your pathetic attempts to defy your favourite singer and use Pitch Editor to correct the note frequency, correct the volume, hit the correct beats, and stretch notes in time to change their duration. I love it!

SynthEyes 2013
ssontech.com

Matchmoving is a technique to retrieve the actual movements of a camera and properties of the lens by analysing the motion of stable reference points in a video fragment. SynthEyes does exactly this.

By applying the same motion to a virtual camera in 3D space it's possible to introduce new virtual 3D elements that exactly match the original camera movements of the video in the background. When lighting, sharpness and motion blur also match the background footage the new virtual 3D element (dinosaurs anyone?) appears to be in the same shot.

For best results, your camera should have a high resolution sensor with a global shutter (or use at least an ultra fast CMOS sensor with multiple readouts to minimizes rolling shutter effects), a low shutter angle, a high frame rate, the best lens you can afford (distortion, chroma aberration), shoot wide angle, using a small aperture (large depth of field), shoot in a RAW file format (or use a 10+ bit format) with a high dynamic range. Your camera should be mounted on a stabilization rig. Your shot should contain parallax for a stable Z-axis calculation. Always(!) disable auto focus, auto white balance, optical/electronic stabilization.

Fusion 9
blackmagicdesign.com

Blackmagic Fusion is post-production image compositing software. It is typically used to create visual effects and digital compositing for movies, TV-series and commercials and employs a node-based interface in which complex processes are built up by connecting a flowchart or schematic of many nodes, each of which represents a simpler process, such as a blur or color correction. This type of compositing interface allows great flexibility, including the ability to modify the parameters of an earlier image processing step "in context" (while viewing the final composite).

Fusion 16 is available both as a standalone program and integrated in their relatively young NLE and ingest tool: Resolve 16. Blackmagic Design is also known for their hardware products, like cameras and production switchers. Get Resolve 16 for free if you buy their latest 6K cinema camera body with EF mount, to ingest its sensor data in the latest Blackmagic RAW format, then edit, process and deliver the final result. All in a single program.

Their website is one big toy store!

Vegas Pro 17
vegascreativesoftware.com

I've used many NLE's but for me none of them were more intuitive and flexible than Vegas Pro.

These days video editing software should be compatible with high resolutions, high frame rates, high pixel depth, and many codecs for acquisition, intermediates and distribution. Hardware accelleration and proxy management is needed to keep productions realistic because computer specs can barely keep up with this trend.

Relatively new to Vegas Pro is the Storyboard feature. It's a feature that I've seen in NewTek SpeedEdit many years ago but I haven't seen it in other software.
The Storyboard doesn't feel flexible enough to my taste, but I hope they continue its development.

I'm sure it will be a good companion for LW-link, a script that I wrote myself. It enables me to setup and access LightWave 3D scenes from the timeline, using the parameters/markers/backgroundplates/audio from the Vegas Pro project, and it updates the video track to the latest preview file. The preview file has a 25 frame handle at the start and at the end of the animation, to allow for finetuning on the NLE timeline. For details, visit my (slightly outdated) Tools page and choose LW-link.

For me, good video editing software is about telling a story in an effective way, and exploring different options to convey that story to a public. The NLE is actually the heart of my production line. A central place where I develop concepts, and deliver the finals.

Unity
unity.com

A development platform for 3D games on multiple platforms, like Windows on desktop/notebook/SurfacePro, VR- and AR-glasses.

I use Unity at work to build interactive 3D worlds for instructional purposes in WebGL webbrowsers and standalone on SurfacePro tablets and Windows computers.

OBS Studio
obsproject.com

Free screen videocapture.
Recording webcam and screen activity for instructional videos.

KRITA
krita.org

Free drawing software.
A very good Photoshop replacement.

DJV
darbyjohnston.github.io/DJV

Free image sequence player.
View video and add in-frame annotations, frame by frame.
No audio support.

Blender
blender.org

Free 3D animation software.

THERE IS ONLY


ONE ADAM BACKWARDS


PROJECT STATUS

I'm still running tests with Adam's current rig. A few things are on my wish list: ribbon style facial muscle controls, foot roll on the sides, squash & stretch controls, toe spreading, chest breathing, mouth shape U, fist control, hand palm cave-in, swallow controls, object layer switching, model resolution switching.

Right now I'm doing tests with Adam's current rig using a video reference in two ways: full copy vs interpretation (and exaggeration).




1 / 9
Adam the entertainer
2 / 9
Adam in a good mood
3 / 9
Adam a bit skeptical
4 / 9
Adam still awake
5 / 9
Adam still young
6 / 9
Adam doesn't mind a classic pose
7 / 9
Adam in the kitchen - does it blend?
8 / 9
Adam slightly uncomfortable
9 / 9
Adam in a nostalgic mood




Please, do come back for the latest updates!

TO BE CONTINUED

Copyrights © 2005- by V.D. Mesman, Amsterdam, The Netherlands, EU. All rights reserved.