Unity: How to playback fullscreen videos using the GL class
Posted by Dimitri | Dec 22nd, 2011 | Filed under Programming
This Unity programming tutorial explains how to use the immediate mode rendering available at the GL class for the playback of video files. There is already another post here on 41 Post that shows how to do the same thing using GUITexture component. However, in this tutorial, a quad is going to be rendered and the video will be played at its texture.
Not only the steps required for achieving fullscreen video playback are going to be explained, but how to properly scale the video based on the screen dimensions is also featured in this post. Warning: this tutorial works only with Unity Pro because the free version doesn’t support video decoding. This post has been created and tested in Unity version 3.4. At the end of the post, a Unity project featuring all the code explained here is available for download both in C# and JavaScript.
Before setting the scene or programming the script, you have to create or find a video file from the internet to use with Unity. Here’s a list of Unity’s supported video formats. It’s basically what the QuickTime Player supports (without additional codecs).
After obtaining the video file, add a folder named Resources in your Project tab. Just right-click anywhere inside and select New->Folder. This is where the video files should go, just drag and drop them inside this folder.
Create the 'Resources' folder if it isn't there yet. Drag and drop the video files in it.
With the videos inside Unity, create a Unlit Texture material, by right-clicking inside the Project tab and selecting New->Material. Give it any name, such as White Unlit. Select this recently created material and set it to use the Unlit Texture shader, by selecting it from the Shader drop down menu:
Select the 'Unlit/Texture' shader from the dropdown menu. Don't assing any texture to it.
Now, take a look at the C# script responsible for setting up and starting the video playback. Here’s the code:
using UnityEngine;
using System.Collections;
[RequireComponent (typeof (AudioSource))]
public class PlayVideo : MonoBehaviour
{
//a Material where the movie texture will be displayed
public Material mat;
//the name of the movie file (without the extension)
public string movieName;
//the texture wich holds the movie
private MovieTexture mTex;
//an AudioSource to play the audio from the movie
private AudioSource movieAS;
//Use this for early initialization
void Awake()
{
//if the material hasn't been found
if (!mat)
{
Debug.LogError("Please assign a material on the Inspector.");
return;
}
//load the movie texture from the 'Resources' folder
mTex = (MovieTexture)Resources.Load(movieName);
//get the atttached AudioSource component
movieAS = this.GetComponent<AudioSource>();
//set the AudioSource clip to be the same as the movie texture audio clip
movieAS.clip = mTex.audioClip;
}
//Use this for initialization
void Start()
{
//assign the video texture to the material texture
mat.SetTexture("_MainTex", mTex);
//Play the video
mTex.Play();
//Play the audio from the movie
movieAS.Play();
}
void Update()
{
//Check if the video has finished playing
if(!mTex.isPlaying)
{
//hide the video by disabling the script
this.enabled = false;
}
}
//Called after camera has finished rendering the scene
void OnPostRender()
{
//push current matrix into the matrix stack
GL.PushMatrix();
//set the material pass
mat.SetPass(0);
//load orthogonal projection matrix
GL.LoadOrtho();
//render a quad where the video will be displayed
GL.Begin(GL.QUADS);
//set the color
GL.Color(Color.white);
//Define the quad texture coordinates and vetices
GL.TexCoord2(0, 0);
GL.Vertex3(0.0f, 0.0f, 0);
GL.TexCoord2(0, 1);
GL.Vertex3(0.0f, 1.0f, 0);
GL.TexCoord2(1, 1);
GL.Vertex3(1.0f, 1.0f, 0);
GL.TexCoord2(1, 0);
GL.Vertex3(1.0f, 0.0f, 0);
GL.End();
//pop the orthogonal matrix from the stack
GL.PopMatrix();
}
}
In order to work properly, this script must be attached to a camera, since it relies on execution of the OnPostRender() method, that only gets called by the Camera component of the game object it’s attached to. In this code, four different member variables are being declared. The first one is a public Material object, which will display the MovieTexture (line 9). The second one, also public is a string that will hold the movie’s name (line 11). Both of then need to be initialized at the Inspector. The Material is going to be the Unlit Texture material that have just been created, and the string is going to be the name of the video file placed at the Resources folder we want to play, like this:
Select the Material and type the name of the video at the Inspector.
Back to the script, the other two declared variables are the MovieTexture and an AudioSource object. The MovieTexture is where the video will be decoded into and the AudioSource will play the movie’s audio (lines 13 and 15). The Awake() method is where all the other variables are initialized. Inside it, there is a if statement that checks if the material has been assigned (lines 21 through 25). After that, mTex is initialized by loading the MovieTexture by its name from the Resources folder (line 28). Then, the AudioSource is initialized and assigned to play the audio clip that comes with the video file (line 31 and 34).
At the Start() method, the MovieTexture is assigned as the texture of the material we have defined on the Inspector (line 41). So, each time this material is used, it will display the movie. But that’s not enough, since it would display only the first frame. To actually play the movie, the Play() method from the MovieTexture object must be called (line 44). The AudioSource object Play() method is also called, to start the audio playback (line 47).
The only thing that the Update() method does is to check if the movie has finished playing. Case it does, this script is disabled, hiding the quad that was used to render the movie texture (lines 50 through 58). As the reader might have noticed, no geometry has been assigned at the Scene to display the material mat which is rendering the MovieTexture. This geometry is being set at the OnPostRender() method, that, as previously explained, only gets called if the game object that contains it has also a Camera component.
There, a quad is being defined in a manner very similar to the calls made to the fixed function OpenGL pipeline. This means that a material is being set, in this case, the material that is displaying the movie texture (line 66); an orthogonal projection matrix is being loaded (line 68) and the texture coordinates and vertices positions are being defined (lines 74 trough 81). All that creates a quad that takes the whole screen displaying the video. When loading the orthogonal projection matrix, in Unity, the position X:0, Y:0 is the bottom left corner of the screen and the position X:1,Y:1 is the top right corner of the screen.
If this script is now executed, the movie will play taking the whole screen, but it won’t take the video’s aspect ratio in consideration and will be stretched into the four corners of the screen, because the quad vertices are set to match the edges of the screen. To successfully maintain the aspect ratio of the videos, read the following sections.
Stretching the movie’s height and how to find the new width
This is one of the cases where you want to have a 16:9 video into a 4:3 screen by cropping its sides. For this case, the height of the video is know: it’s the same as the screen’s height, in pixels. The width of the video must be calculated based on the new height in such way that it maintains the video’s original size ratio. By knowing this new width, it’s now possible to subtract the screen’s width from it and divide the obtained value by two, to find out how much each side of the video is going to be outside the screen.
With that in mind, only the following would need to be added to the above script:
//*** Omitted Code ***//
//the quad half width needed to fit the video on the screen
private float quadHalfWidth;
//Use this for early initialization
void Awake()
{
//*** Omitted Code ***//
//find out the quad width based on the screen height and original movie aspect ratio
quadHalfWidth = (((mTex.width/(float)mTex.height) * Screen.height) - Screen.width)/2;
}
//*** Omitted Code ***//
//Called after camera has finished rendering the scene
void OnPostRender()
{
//push current matrix into the matrix stack
GL.PushMatrix();
//set the material pass
mat.SetPass(0);
//load orthogonal pixel projection matrix
GL.LoadPixelMatrix();
//render a quad where the video will be displayed
GL.Begin(GL.QUADS);
//set the color
GL.Color(Color.white);
//Define the quad texture coordinates and vetices
GL.TexCoord2(0, 0);
GL.Vertex3(-quadHalfWidth, 0.0f, 0);
GL.TexCoord2(0, 1);
GL.Vertex3(-quadHalfWidth, Screen.height, 0);
GL.TexCoord2(1, 1);
GL.Vertex3(Screen.width + quadHalfWidth, Screen.height, 0);
GL.TexCoord2(1, 0);
GL.Vertex3(Screen.width + quadHalfWidth, 0.0f, 0);
GL.End();
//pop the orthogonal matrix from the stack
GL.PopMatrix();
}
Basically, there is going to be one more float variable to store how much the video is going to be placed on the X axis of the screen. Moreover, instead of calling GL.LoadOrtho() to load an orthogonal projection matrix, the GL.LoadPixelMatrix() has been called. It’s the same as the orthogonal projection matrix, the only difference is that the screen coordinates are not normalized. Therefore, the screen coordinates range from 0 to Screen.width on the X axis and from 0 to Screen.height at the Y axis, instead of going from 0 to 1. By using this matrix, it isn’t necessary to normalize the quadHalfWidth and Screen.width: they can be directly used as vertex coordinates.
Stretching the movie’s width and how to find the new height (letterbox)
This case is exactly the opposite from the previous one: the video’s width must match the screen width, and what needs to be calculated is the new video height. The video must also be centered vertically on the screen, with two black bars: one of the top, and one of the bottom of the video. The height of the video must be calculated based on the new width in such way that it maintains the video’s original size ratio. Width the new height, the bottom vertices’ Y position must be calculated by subtracting the video’s height from the screen’s height and dividing it by two.
With that in mind, only the following would need to be added to the PlayVideo.cs script:
//*** Omitted Code ***//
//the quad height needed to be display the video in the letterbox format
private float quadHeight;
//Y position of the bottom vertices
private float quadBottomVert;
//Use this for early initialization
void Awake()
{
//*** Omitted Code ***//
//find out the quad height based on the screen width and the inverse of the movie aspect ratio
quadHeight = (mTex.height/(float)mTex.width) * Screen.width;
//find the Y position of the bottom vertices
quadBottomVert = (Screen.height - quadHeight)/2;
}
//*** Omitted Code ***//
//Called after camera has finished rendering the scene
void OnPostRender()
{
//push current matrix into Che matrix stack
GL.PushMatrix();
//set the material pass
blackMat.SetPass(0);
//load orthogonal pixel projection matrix
GL.LoadPixelMatrix();
//render a black quad that takes the whole screen
GL.Begin(GL.QUADS);
//set the color
GL.Color(Color.white);
//Define the quad vertices
GL.Vertex3(0.0f, 0.0f, -1.0f);
GL.Vertex3(0.0f, Screen.height+1, -1.0f);
GL.Vertex3(Screen.width, Screen.height+1, -1.0f);
GL.Vertex3(Screen.width, 0.0f, -1.0f);
GL.End();
//set the material pass
mat.SetPass(0);
//render a quad where the video will be displayed
GL.Begin(GL.QUADS);
//set the color
GL.Color(Color.white);
//Define the quad texture coordinates and vertices
GL.TexCoord2(0, 0);
GL.Vertex3(0.0f, quadBottomVert, 0);
GL.TexCoord2(0, 1);
GL.Vertex3(0.0f, quadBottomVert + quadHeight, 0);
GL.TexCoord2(1, 1);
GL.Vertex3(Screen.width, quadBottomVert + quadHeight, 0);
GL.TexCoord2(1, 0);
GL.Vertex3(Screen.width, quadBottomVert, 0);
GL.End();
//pop the orthogonal matrix from the stack
GL.PopMatrix();
}
The above code features two more float member variables that will store the video’s new height and the position of the bottom vertices. Additionally, there is another Material member variable, that should be assigned at the Inspector. This material should be a black diffuse material. and is used for rendering the black bars that frames the video.
As the previous case instead of calling GL.LoadOrtho() to load an orthogonal projection matrix, the GL.LoadPixelMatrix() has been called. It’s the same as the orthogonal projection matrix, the only difference is that the screen coordinates are not normalized. Therefore, the screen coordinates range from 0 to Screen.width on the X axis and from 0 to Screen.height at the Y axis, instead of going from 0 to 1.
Not only the video but the black bars must be rendered. So, a black quad that takes the whole screen is being rendered behind the video.
That’s it!
Downloads
- fullscreenvideo2.zip 8.01M
Want to see more posts like this more often? Please donate by clicking at the button below:
Don't feel like donating? Alternatively, you can support the site by purchasing the FULL version of the Arrows Live Wallpaper app at the Google Play Store™:
Thanks a lot!
your post is really helpfull
but one little detail..would it be possible to read
transparent(alpha) movies with one or two additive lines of codes..?
I don’t know if it is possible.
when I tried for iPhone it got compile time error.