360 Videos in Unity with VR Panorama 360 PRO Renderer – Review & Tutorial
VR Panorama 360 PRO Renderer is an Unity Asset that will help you render 360 videos in Unity. You can use those 360 videos on YouTube, Facebook and more importantly any VR device. It’s an amazing tool especially if you want to do some Architectural Visualizations and you want to provide a 360 video for the client. Yea, having an actual application that runs your scene can be amazing, but let’s say that you want to fast demo your work for a possible client. You prepare your 360 video by putting it on your phone, you bring your GearVR with you, then you give that GearVR + Phone to your possible client and bam! Now your client can actually see your work in 360. That way your amazing scene can “run” on your Android device.
Just to be more clear. Using VR Panorama 360 PRO Renderer you can render your scene in 360 images. Then those 360 images get transformed into a 360 video. With that video you can do various things. You can put it on YouTube, Facebook or use it on your phone or a 360 video player. I actually developed an Android app that plays 360 videos.
GAIA Terrain 360 Video Demo. (Not ideal for YouTube since YT compression introduces a lot of artifacts because of the grass foliage)
VR Panorama 360 PRO Renderer Demo - Arena 3840x2160
VR Panorama 360 PRO Renderer Demo - Arena 7680x4320
Now let’s see how can we use VR Panorama to render 360 videos.
Read More Below…
One thing to keep in mind is that my demo scene is not 100% ideal for YouTube. It has grass and lot’s of green stuff and YouTube doesn’t like that for it’s compression. So the demo video will be of a lower quality than what you could do. I created this terrain in about 10 Minutes using Gaia from the Asset Store.
In order to use the plugin you have to create a new VR Panorama Camera. So right click in your hierarchy and click on the VR Panorama Camera. This will create a new Game Object that has a Camera attached but also the VR Capture Script which has all those juicy settings.
After this, you need to add your special effects to the camera in order to enhance your visuals. You have to be a bit careful with what effects you add. Keep in mind that you will have something like 6 cameras rendering this so some visual effects might not work because artifacts will appear. I remember some time ago a SSAO plugin I was using had no support for 360 cameras so everything looked pretty weird. Since then it was updated.
Another thing you want to do now is to find a way to move your camera. In my demo scene I’m moving my camera using an Asset called Curvy (I will cover this plugin soon). It’s a spline tool creator. I created a Spline and I’m animating my camera to move on that spline. However! With Curvy in order to move the camera on a spline path I had to add a script that was doing just that. And it worked, however my 360 video had lot of artifacts and looked weird. Then I remembered something. The VRPanorama script, takes your camera and clones it for each angle. That’s how it manages to do the stereoscopic effect. So when it cloned my camera, it also cloned my Spline Movement script which created the artifact. So in order to fix this I created a GameObject that has the Spline Movement script, and as a child it has the Camera.
VR Panorama 360 PRO Renderer currently has 3 capture modes. Animation Capture, Still Image and VR Panorama RT.
Those 3 capture modes have similar settings so I’m not going to go over all of them … but.
Still Image will capture a still image whenever you press “Space” (you can change the key to something else).
VR Panorama RT is a Real Time camera that will show in the editor how your capture will look. This is very good in order to debug all your settings to see if you don’t have any artifacts.
Animation Capture is your main thing. In this mode we will capture a screenshot aka a Frame for a specified resolution for a number of total frames. So let me go over all the settings.
The Capture Type is one of the most important settings. We have:
1. Equidistant Mono
This is your normal mode. It’s good for YouTube & Facebook. Basically the places where you only want a 360 video and not a “VR Experience”. This will capture an image like this:
As you can see it’s just a normal, mono 360 stereoscopic image.
2. Equidistant Stereo
This creates Stereo images. Those videos are good if you want to showcase them on a VR device. Oculus/VIVE/GearVR/etc. Equidistant Stereo really make the video seem like 3D. Equidistant Mono is pretty flat if you show it on VR, but with Stereo… it’s amazing. Hard to explain if you don’t see it for yourself, but it really ads depth.
3. Equidistant Stereo SBS
SBS stands for Side-By-Side. Same thing like above only that the images are being place Left/Right.
4. Video Capture
This is a new mode that I havn’t really used. It’s a hi-quality standard camera with configurable Anti-Aliasing.
So, if you want your video to be placed on YouTube, you go for Mono. If you want your video to go on a VR device and have that feeling of depth, you go Stereo.
Your next choice for a setting is the format of the image. JPG or PNG? PNG is lossless so you will have the highest quality. However your images will also be larger in size. Keep in mind that a 7680×4320 JPG image was around 11 mb. Depends of course on the content… BUT. 1200 frames x 11 mb = 13 gb. This was for a video that was around 39 seconds! When I was doing the final renders on a 5 minute video with 60 frames per second, in PNG … I had a folder that was 300 GigaBytes BIG!!
The next important aspect is the resolution of the image (video). YOu have some buttons there with Resolution Presets. Those are some good presets there. One advice that I have is that when you do your tests, go for a smaller resolution like 2048×2048. That way everything will render faster so you can test things faster. Only when you are 100% sure that everything is ok, you go for the higher resolution.
The 2nd important setting that you need to keep track of is the FPS. Frames-per-Second. I will not go into details about what FPS is exactly but basically… a video is composed of still images. FPS = how many images we have for 1 second. In Europe we go for around 24, in US around 30. Lately we have been using 60 on some YouTube videos. It makes everything more smooth. So if you set it up to 60 FPS you will get 60 images for 1 second. You want to record 10 seconds that will equal a total of 600 images. So after you decided on how many FPS you want, you have to decide how many frames you want to render. If your experience is 2 minutes long. That is 120 seconds. so 120 x 60 = 7200 frames. VR Panorama is kind enough to show us our Sequence Length so you can always toy around with the numbers so you will be sure you render exactly how much you want.
The next important thing is the Quality Settings. Again, you have a silder that goes from Speed vs. Quality. As you might have figured it out… you want to max out the Quality when you are doing your final render.
The settings that we talked so far are strictly related to the quality of the image. When VR Panorama finishes rendering all the frames, if you checked the “Encode H.264 Video After” checkbox, it will start transforming those images into an video using the H.264 codec using the Bitrate that was set up. The higher the bitrate the higher the quality, however you need to keep in mind where are you going to use that video. For example a 1920×1080 video on YouTube will be re-encoded by YouTube at 8000 Bitrate. So there is no point to render it at a higher quality anyway. But again, it’s a good idea to have the final video at a higher quality just in case you need it for some other things.
The next step you need to do in case you want your video ready for YouTube is to use the “Spatial Media Metadata Injector“. When you click that button a new window will open. In this window you have to click the Open button, then you have to select your video.
You will be able to check the “My video is spherical (360)” checkbox and this little tool will inject some metadata into the video.
That way youtube will know to convert your video into a 360 video.
But, what if you have audio? Normally, you will get a video that has no audio. You can take this video and re-edit it into a software editing tool like Adobe Premiere, and you will be able to add there whatever sounds you want. BUT.
Let’s say that you have a scenario build-in into Unity. I did something like this in this project. If you have a project like this, first you need to press the red button called “Capture Audio”. This will start your scenario at the normal speed and it will record all the audio from your scene and it will save it into an WAV format. Once you have that, the renderer will know to include the audio when it will encode the video.
How to render a 360 Video in Unity for YouTube?
- Right click on your Hierarchy and select VR Panorama Camera. This will add a new camera that has the VR Capture script attached.
- Have a spline in your scene and make your camera move along that spline.
- Set Capture Mode to: Animation Capture
- Set Capture Type to: Equidistant Mono
- Set Capture Angle to: 360
- Set Sequence format to: PNG (if you want the highest quality, however the image file size will be larger)
- Set the path where your sequence images will be saved. Make sure you have enough space on that HDD.
- If you want the highest quality set the Preset for “YouTube 8k”.
- Decide on the FPS. In our case let’s go for 60 FPS.
- We know that our scenario is 2 minutes long so we tell VR Panorama to render 7201 frames.
- On Optimizations we set the Speed vs. Quality to 32. This will use A LOT of Vram so make sure you have a decent video card.
- If you have Audio in your scene, press the Capture Audio button and wait for it to finish.
- Check the “Encode H.264 video after” checkbox.
- Set the H.264 Bitrate to at least 50000
- Press the Render Panorama button.
You will see something like this:
Wait for everything to finish. When this will end you will see this window pop up:
When this will be done you will find a mp4 file in your folder that has all your other still images. You can do whatever you want with this video, but if you want for it to be 360 and on YouTube you need to inject the metadata (see above).
So this are the basics of VR Panorama 360 PRO Rederer!
If you have any questions ask below or go on the Official Thread on the Unity Forums.