Currently I’m working on a project at work that requires the information (lets call it unity content) displayed on twelve screens in another room/building (a long way away from the computer generating it), it’s a little bit of an unrealistic-solution/challenge to connect a dvi/hdmi/<or you name it> cable through buildings… as quality degrades the longer the cable gets… and not to mention all other obstacles along the way.
Enter streaming over ip… now this is not the easiest thing to do there are many ways to go about it.
The first option that came to mind was lets get a hardware adapter that can convert the DVI output to an H.264 IP stream.. that is a pretty expensive solution if you have twelve outputs that need to go to twelve screens, average cost of a decent hardware encoder runs somewhere around and above 4000 euros that times 12 and you have yourself an expensive streaming setup..
So Ideally thinking about costs the best solution would be do it in software, the solution first thought of was to do screen capture and encoding on the fly, all possible with vlc and many other software screen capture programs. After a few small tests this proved to be really CPU hungry and not really the best solution, because we need as much CPU & GPU time as possible for generating the Unity Content.
So now almost at the end of all options a colleague of mine mentioned AMD’s Radeon Sky Series which is basically cloud gaming tech not yet available for the masses. This got me thinking and looking more closer at cloud gaming which is basically a server on the web streaming games to specialized clients. Looking further into this I found Nvidia and the new range of GPU’s with Kepler Architecture *hallelujah* each Kepler based card is fitted with a beautiful h.264 encoding chip. Which turns out to do encoding without even touching the GPU or CPU time, and mighty fast too… 76 frames at 600ms which boils down to about 8ms per frame  which is what i’m looking for. with Low Latency settings I even got a 1.08ms encoding time… *jaw drop*
So I dove head first into the reasonably clear Nvidia API and expanded on my previous experience in creating a decent C++ native Unity Plugin. After bumping into alot of “unresolved external symbol” errors (which by the way usually mean your missing a lib file) and a few unicode/ascii hurdles (Yes i didn’t think this would be an issue, but it turns out C++ is picky about string encoding) I eventually got some output from the chip even though it was just a green screen.. it was still output and the encoder had been initialized to create it. Which meant progress!
Realizing that the green screen wasn’t my actual unity content I figured I’d better dig a little deeper and found that the green screen was actually blank YUV pixels, eventually I managed to change that to red using different YUV values and filling the texture with them. Still not so useful because Unity doesn’t output YUV pixels now does it. This led me to search the internet for a way to convert RGB to YUV (actually BGRA to YUV444) pixel for pixel conversion was an option with readily available formulas online.. only not the fastest way.
Browsing through the built-in shaders I found a reference to a YUV conversion, using that as a reference I created a new shader for myself to do the YUV conversion with some modifications to values and the order they are stored in.
Now a little under half a year later i’m using my plugin to encode 12 H.264 streams to send to VLC which passes it on to a VMS.
Leave a Reply