CS283 Assignment 3: Reflective Shadow Maps, SSAO and Environment Maps

My work in this assignment can be divided into three parts as the title suggested.

Reflective Shadow Maps (RSM) is an extension of shadow maps to estimate one bounce indirect illumination using per pixel lighting. Screen Space Ambient Occlusion is a technique originally developed for the video game Crysis to estimate ambient occlusion in real time. The focus of my work is to produce plausible looking images in real time by combining the output of RSM and SSAO. And I also implemented mirror reflections from environment maps to fulfill the requirement of this assignment.

This writeup is accompanied with 2 vidoes, and explaination of the parts comes after the video.

*Note: The slight jerkiness of the demonstrations comes from the screen capturing software. In fact, all components described on this page run at 60fps, the fps cap imposed on my machine by some API/drivers.*

**Reflective Shadow Maps and SSAO**

**Mirror Reflections from Environment Maps**

Developed by Dachsbacher and Stamminger[2], Reflective Shadow Maps is an extension of standard shadow maps to also estimate one bounce indirect illumination using per pixel lighting. In a standard shadow map, depth value is gathered from scene using a perspective or orthographic projection from the light source, depending on whether the light is spot light or directional light.

Figure 1 - 4: World space coordinate, normal, flux and depth RSM textures from camera perspective

*Ordered from left to right, top to bottom.*

In RSM, world space coordinates, world space normals and flux are also gathered in addition to the depth value. These 4 sets of information can reasonably estimate one bounce indirect illumination of diffuse surface by sampling pixels nearby in screen space. The indirect illumination due to a pixel light as suggested by the paper is

where n denotes world space normal, x denotes world space coordinates, phi denotes flux, p denotes the pixel light, and bracket denotes dot product.

However, naive implementation of the algorithm requires hundreds of pixel light samples per fragment to produce quality results. That would make smooth animation in high resolution impossible. A way is needed to reduce the number of samples taken while maintaining reasonable quality. The original paper suggested to first run the naive implementation once in low resolution, and bilinearly interpolate the result in full resolution in the second pass given that the normal between the 4 samples are close enough.

The sampling method used in my code is explained in the "Rotating Kernel Sampling" section below.

Figure 5, 6: Scene without and with indirect illumination

SSAO estimates ambient occlusion by taking samples in screen space. Different people have different ideas on how the occlusion term should be calculated. One implementation of SSAO only compares the depth value of the samples; a fragment is occluded if the sample is closer to the viewer than the fragment.

Nathaniel suggested to include cosine term and attenuation factor in calculation[3]. This gives a better result than only comparing the z-values. To calculate the cosine term between the screen space sample and the fragment in OpenGL shader, one needs to store the world space coordinates in the first pass. Therefore, SSAO is a form of deferred shading.

Figure 7, 8: The scene and its ambient occlusion

Similar to RSM, it is not possible to acquire hundreds of samples per fragment in real time. The next section will discuss the sampling strategy used by both RSM and SSAO.

Picking random points in a circle gives good result given that enough samples are taken. However, taking hundreds of samples for each fragment in fragment shader is simply not possible. Taking less samples than needed will result in banding. However, enough samples can be taken if samples are only taken in one direction; that is, turning the sampling kernel from 2D to 1D.

That being said, the output is meaningless if each fragment takes samples in a totally random direction. However, the output is useful if a repetitive sampling pattern is introduced. This would give an output with no banding, but a patterned noise. The patterned noise can be removed by taking average of nearby pixels. The filtering introduces bluriness to the final image. Yet, indirect illumination and ambient occlusion do not suffer much from bluriness because they are not directly rendered. It is difficult to notice as long as the direct illumination is sharp.

In rotating kernel sampling, 20 to 30 samples per fragment is enough to give quality output. It enables both indirect illumination and ambient occlusion at 60 fps at resolution of 1280x768. Therefore, rotating kernel sampling gives robust performance at the cost of blurred images, which is a perfect fit in this situation.

Here is the outline of the sampling method I used for both RSM and SSAO. I learned this idea from Chapman[1].

- Create an 1D sampling kernel with a distribution of x^2. That is, more samples are closer to 0, the pixel.
- Create a 4 x 4 texture with random theta values from 0 to 2 PI. This 4 x 4 map is tiled over the whole screen space.
- For each fragment, rotate the 1D sampling kernel centered at the fragment by its corresponding theta, and take samples according to the kernel.
- Filter the output by averaging the pixels in a 4x4 square to remove the noise.

Figure 7, 8, 9, 10: Indirect illumination and SSAO using Rotating Kernel Sampling, without and with filtering

*Notice the patterened high frequency noise without filtering, and how they disappeared after filtering.*

Basic mirror reflections from environment maps with one modification. Instead of conventionally converting a panoramic environment map into a cube map, my code uses a panoramic environment map directly. By doing so, the environment map can be indexed directly by theta and phi. Yet, a plane needs to be placed behind the scene for rendering the environment map as background.

Figure 11, 12: Standford bunny with reflections from seaside and Grace Cathedral.

- Chapman, John. 2013. SSAO Tutorial. John-Chapman-Graphics. http://john-chapman-graphics.blogspot.co.uk/2013/01/ssao-tutorial.html
- Dachsbacher, Carsten and Stamminger. 2005. Reflective shadow maps. In Proceedings of the 2005 symposium on Interactive 3D graphics and games (I3D '05). ACM, New York, NY, USA, 203-231. DOI=10.1145/1053427.1053460 http://doi.acm.org/10.1145/1053427.1053460
- Meyer, Nathaniel. 2012. Shader Effects: Screen Space Ambient Occlusion. Devmaster. http://devmaster.net/posts/3095/shader-effects-screen-space-ambient-occlusion