Class10 - Manual set reconstruction from an HDR panoramic image

It would be great to have more automatic set reconstruction by using, LIDAR, 123D Catch, Photosynth, or other point-cloud based method.  But that is not always possible due to access to the location, time, money, and patience.  Also many times you really only need basic layout of the area, such as floor, wall, and one major light source.  For these cases we can manually rough out a 3D model of the scene in our panoramic image and map the geo with that HDR.

All credit for this scene and lesson to Christian Bloch and The HDRI Handbook 2.0

http://www.hdrlabs.com/book/index.html

Make sure you Maya gird is set to meters so we are working in real world scale.

Start with a nurbs sphere, increase spans from 4 -> 8, so we can flatten top and bottom.

We will shape a big disk/cylinder type shape and project our pano inside there.

When flattening the top and bottom, use move tool options, turn off “Retain component spacing”

In your panel go to shading -> interactive shading = off, to prevent wireframe while moving cam.

In the attribute spreadsheet, filp normals in render tab so we can see the inside of the disk.

Apply a surface shader with spherical projection of our panoramic image.

Check for backward text, you may have to horizontal flip the image, in shader Uscale -1.

Here is a great slide presentation in PDF format from Ben Snow at ILM presented at Siggraph 2010 which describes using their software Ethereal for extracting lights from a pano and how they would use basic set reconstruction with HDR textured on for lighting Iron Man 2.

http://tojek.com/jon/?attachment_id=584

Must place spherical projection center at same height the panorama was shot from, 1.75m.

Next estimate the ceiling height to be 4.7m, and make the thing big, about 40m diameter.

1) bake the spherical projection into a UV map, so we can map with surface shader

When you project and create the new UV maps for geo, make it spherical from exact same point as the original spherical projection is from, so you can switch back and forth.

2) the columns and boxes could be planar projected and adjusted in the UV window.  

The back walls will need to have UV’s normalized to fit, may also have to quad the triangles around the ugly nadir, this stuff could be easier in Modo or other modeling program.

2) separate the projected pano into foreground and background, also two shaders

3) use photoshop or other paint program to fill in or re paint occluded areas

4) select a column you don’t want in bg, delete, fill with content aware fill, photoshop

If you get this error using exr files as textures, just convert them to hdr format with Nuke.

// Warning: Failed to open texture file d:/class_fxphd/project_fxphd01/sourceimages/class10_panorecontruct/subway-lights_aligned.exr //

The EXR format will actually work in the render, but it gives this error anyways.

Once you have the environment geometry textured, you will now be lighting with primarily Final Gather.  To get better shadows you could place an area light at the locations of light in the scene.  Another option to get direct lighting and shadows is to have one render layer that swaps out your geo environment with an IBL same map, then turn on Environment Lighting Mode.  This is slow to render, but would only be used for shadows landing on the ground caught with mip_matteshadow.

Class09 - Texture map 3D set geo with HDR pano - continued

This class will continue the concept started last week with a different background in the light tent.  These things come with 4 different colored pads to get a variety of looks in your photography, so we will work with the blue floor and back wall.  

http://www.amazon.com/gp/product/B001B8G4R8/ref=oh_details_o01_s00_i00?ie=UTF8&psc=1

What are string options?

String Options are essential a tool for you to declare a variable to the mental ray renderer that is not directly built into the Maya/mentalray translator.  You can use string options to tell mental ray to turn on/off a feature and what settings to use.

http://docs.autodesk.com/MENTALRAY/2013/ENU/mental-ray-help/

http://elementalray.wordpress.com/2011/10/31/string-options/

Once you start using the user_ibl_rect for your area lights, you must add this string.

select miDefaultOptions

hit “add new item”

scroll down to the last new empty string in the attribute editor

name = light relative scale

value = 0.318

type = scalar

Spelling is important, and the value 0.318 is the recipricol of pi.

It has to do with how shaders work with light (and the amount, hence, scale) and that newer shaders handle light differently. By adding this string option you ensure scenes with old, new, or mixed shader types react the same way.

Lastly, and very important: you must use the string option for Light Relative Scale for this shader to correctly scale the light for non-BSDF or older (legacy) shaders. This value is 1/pi. This is added in the miDefaultOptions.

Light scaling. If using traditional materials, rather than bsdfs, etc., use the string option "light relative scale" with a scalar value of 0.318, which is roughly 1/pi

And make sure disable the spec from the mia

b) Rectangle type area light shape. For user_ibl_rect, ensure that the type of area light shape is rectangle.

There are 3 IBL systems in maya 2013.

(1) There is standard IBL that was in maya since decades.

(2) There is Native IBL that work with standard IBL just accelerate it better and importance sample the HDRI, also gives you direct lighting with shadows, can turn off or reduce FG.

(3) There is User_IBL_env that is new with MR 3.10, you connect HDR image to an area light and camera env.

Lastly there is the other IBL shader calle,  User_IBL_rect - which is a lights shader that we can use with HDR textures. So if we plug texture into this shader it will be  used as light source it will use importance sampling to sample it and render quite fast, good with Unified Sampling.

Now we take a closer look at the user_ibl_rect light shader.

When rendering command line, I get this message because I have visibility OFF for my lights.

“IBL shaders should be applied to a VISIBLE user area light to work optimally”

First we will do some simple comparison tests between visibility ON/OFF.

Visibility OFF creates a specular response, this highlight is not good because it doubles the light.

The renderer messages and documentation suggest keeping visibility ON.

Lighting - Create an animatable control switch to handle lighting, texture switch, and two light intensities.  On the light tent geo (1) create attribute, float, between 0 and 1  (2) write an expression on the blend node that will dissolve between the two textures (3) use set driven key to control the two light intensities between 0 and 2

Kit-bashing - Look at the animated spider file.  Do a clean export of only parts you need, no render layers, hidden geo, old shaders, all optimized.  Replace legacy shaders with current mia light conservation shaders with textures plugged in.  Group, import, then scale, rotate and translate into place.

Modelling -  Using polygon extrude, dig out a hole where the spider will emerge.  Be sure to go into UV texture window to un stretch around the mouth of the spider hole.  

Camera animation - make a two node camera so you can keyframe the aim node and attach to motion path for the camera body.

Render - break into as few render layers as possible.  Use PCM pass contribution map to render foreground character with background light tent, then have matte passes with holdout to separate later.  Because the light tent is using a surface shader and not actually being affected by lights, you will need to create diffuse pass for the hole only.  Also will need a shadow pass with mip_matteshadow.

Comp - put layers together in Nuke, compare to basic photographs of the light tent.

Voxelizer Script

http://www.creativecrash.com/script/voxelizer

I had to try one experiment with voxelizer after seeing some cool examples online.  The idea of breaking any complex model into little discreet parts is always interesting to me, so I took the Infinite Realities head and ran this script on it.  After a two minute wait it pops out 4000+ cubes, will a little bit of spacing between them, and they each have their own Lambert shader with a color resulting from a sample of the texture that was on there.  The only problem was I wanted SSS skin shader on each one, but this script would only put out lambert, and I did not find a way to convert them all.  So I rendered that in the blue light tent and posted here, very little effort, but a fun exercise.  For sure I want to make them look like skin, and I want them to drop to the ground or explode with Rigid Body Dynamics.  Next time.

Class08 - Shoot and Stitch HDR pano then texture map 3D set geo

For this class I will experiment with a 30 inch cube light tent and two big lights.  The idea is to shoot an HDR panoramic photo inside the tent, then rebuild the tent geometry in Maya, and do a spherical projection of the HDR in order to verify we are getting true inverse falloff from our HDR lighting image.  In addition I will shoot the tent under different lighting conditions so we can flip textures to turn lights on and off.

This is a great presentation of HDR panos projected on 3D scanned and rebuilt set geometry for the film Beautiful Creatures.

http://www.fxguide.com/fxguidetv/fxguidetv-165-scott-metzger-on-mari-and-hdr/

Show the photos and basic workflow for shooting a panoramic using fisheye lens, Nodal Ninja, table tripod, ProMote for exposure brackets, and Nikon D40.  In this example I shoot 6 times around, 2 up zenith, 2 down nadir, and 7 exposure brackets from 1/2500 to 1/2.5 or 2EV’s per step.  We will shoot with a variety of lighting conditions, but identical everything else (basically turning lights on and off), giving us 70 RAW photos per pano.

http://www.promotesystems.com/products/Promote-Control.html

http://shop.nodalninja.com/nodal-ninja-3/

Very fast description of viewing the photo sets in Adobe Bridge and the Hybrid stitching workflow as described in “The HDRI Handbook 2.0 by Christian Bloch.  Merge each exposure bracket into HDR image and tonemapped jpeg using batch Photomatix.  Then stitch only the 10 jpegs in PTgui for faster software response. Once you have a great stitch, you swap in the 10 HDR images, and PTgui will process them together much faster. This workflow breaks up the work and makes it faster for interactive parts and more automatic for the mindless batch parts. Helps when you have lots of panos.

http://www.hdrlabs.com/news/index.php

In Nuke show how to use the Spherical transform node to paint out the tripod from you pano.  Also some white balance and possible boosting of the gain to increase light values.  To have  later with user_ibl_rect, crop cut out each light and export out as HDR texture.  Light extraction.

In Maya do a comparison of a model rendered with this HDR on a typical spherical IBL and the same pano mapped onto the 3D model of the light tent which should allow us to have accurate inverse falloff of light.

Set up two area lights with user_ibl_rect that will generate direct lighting and shadows, allowing us to depend less on Final Gather.  You may need to add light relative scale = 0.318 (the reciprocal of Pi) in the string options for this feature.

In addition we will try to use direct lighting (we want good shadows) with these HDR textures by using either user_ibl_rect (maya 2013) or builtin_object_light (maya 2014)

If you are using Maya 2013, this is the method we will cover here.

Introduced with mental ray 3.10 are new shaders called the user_ibl shaders.

Inside this shader package are two new shaders with different usage scenarios.

• user_ibl_env
• A more simplified usage than the Native IBL (mental ray), the user_ibl is a scene entity used for lighting a scene globally from an environment.
• user_ibl_rect
• A shader used to generate light cards or “billboards” to replace otherwise complex geometry and lights in a scene.

Try user_ibl_rect, just attach as light shader to your MR area light, it can illuminate and cast shadows.  Also use the connection editor to connect user_ibl_rect Samples -> areaLight areaHiSamples and areaLoSamples.  Must have “use light shape” ON, and visible.

If you are using Maya 2014, we will not cover this now, perhaps next week when I upgrade.

mental ray 3.11 includes a new shader called the builtin_object_light that tells mental ray that the object is a light.

Animate lights turning on and off by using the blend mode to dissolve through our identical but different light condition HDR textures.

Drop in some animated creature and hope for the best.