Troubleshooting mental ray renders of Maya XGen

Got a nice XGen groom? Everything looks decent in VP2, but in your mental ray renders parts of the hair are missing or, even worse, the hair is not rendered at all?
Don’t panic, these issues can be addressed.

Hair geometry shader setup

In mental ray, XGen hair and other procedurals like cards or randomly instanced archives are handled by a geometry shader. This geometry shader needs to be set up correctly. Usually, this is done automatically by Maya when a new XGen description is being created. In some cases it can happen though, that the geometry shader gets lost or has not been applied. As a result, the XGen primitives are not rendered. To reapply the shader, open the preview/output tab of the XGen window and click the “Setup” button to the right of  “Setup Geo Shaders” in the output settings section:

xgen-setup-geo
After that, your XGen description should render.

Adjusting render output settings

In case your hair looks clipped or thinned-out in a rendering, pay attention to the render output settings. One important setting that usually needs to be adjusted manually is the area, that is supposed to contain all generated XGen primitives. By default, this area is computed from the the bounding box of the XGen base mesh,  plus a padding value of 1 for the XGen primitives.  This padding value is suitable for fitting a default XGen description with hair of length 1. Real-world grooms rarely meet this condition, that’s why renders come out with clipped hair.

To change the padding, open the preview/output tab of the XGen window and click the “Auto Set” button beneath “Primitive Bound” in the output settings section to compute a proper padding-value:

xgen-primitive-bound

This ensures that all hair (or whatever primitives you are working with) is being rendered correctly.
As you groom along, the padding value might need to be re-adjusted, again.

In case the hair appears less dense in the rendering than in the viewport, make sure the “Percent” slider has been set to 100:

xgen-percentage

A lower value will cause the number of generated primitives to be reduced accordingly. This slider can be a nice tool to get fast preview renders with fewer primitives.

Batch rendering XGen primitives

For batch rendering, one additional step is necessary. The XGen collections used in your scene need to be exported to disk to ensure that mental ray can find them. To do so, click on File -> Export Patches for Batch Render in the XGen window:

xgen-batch

Happy hair rendering,
Sandra

5 thoughts on “Troubleshooting mental ray renders of Maya XGen

  1. Sean says:

    Hi, I’m using xgen to make a furry creature in maya. This is my first time using it and it simply won’t show up on my batch render. I’ve gone through all the steps and i even started from scratch to ensure that everything is in order, but it still won’t show up on the batch.

    Is there anything else i can do to fix this?

    PS. I’m using maya 2016

    Thanks in advance!

    1. nvsandra says:

      Hi Sean,

      without further information, it is hard to say what is going wrong. Did you do the batch export of the xgen patches?

      Cheers, Sandra

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s