Upscaling Video: What Is It
Upscaling is the technique to increase the image resolution without recourse to an actual source of higher definition. Any video sequence will be upscaled for display on a screen with higher resolution than the video itself. Upscaling a video to be encoded will give control over the automated upscaling done by the output devices (television sets or computer monitors). Unless a video has the exact resolution defined as standard for a given output device, the video will always be rescaled. Thus (proper) upscaling will increase the image quality on a high-definition screen while not reasonably change its quality on a standard-definition screen. Yet there is a downside: a higher resolution means more pixels per image that may or may not increase the bitrate required for optimal compression.In short, the increase in quality generally (as in: for most people) does not justify the increase in file size and the additional time spent on preparing and doing the encoding. But you may think it interesting if you are watching low resolution video on a high definition screen.
Proposed Web Solutions
There are few solution proposed on the internet.Robert Martens has written a whole guide, providing the necessary tools and a his own autoloaded AviSynth script SimpleSlugUpscale but it does little more than deinterlacing and resizing the image. But that it does well!
gyroshot.com - SimpleSlugUpscale
Meanwhile Jean Brünn has made a comparison of AviSynth resizers that are commonly used for upscaling videos. Winner in his little test was tritical's NNEDI filter. NNEDI apparently stands for something like neural network edge directed interpolation.
Upscaling in Avisynth – Comparison of Resizers
tritical's AviSynth Filters
Upscaling a Progressive Video
Here is my proposed solution for a video with progressive scan. Progressive scan is today often used on Blu-ray or PAL DVD video. Obviously not deinterlacing, that is reverting interlaced video to progressive scan, is required on this sort of material. Robert Martens' SimpleSlugUpscale and tritical's series of edge directed interpolation filters were created for deinterlacing and upscaling. But NNEDI3 has a special function that does just the upscaling of the image and that's what I'm going to use.To give you the basic idea, here's an outline of my approach. AviSynth is a frameserver program acting as a non-linear video editor. It does not have a GUI but is controlled by scripts written in the AviSynth scripting language. It is inasmuch a frameserver that any program able to open AviSynth scripts will actually see an uncompressed AVI file. But reduce the workload on the computer VirtualDub can be used to actually create a real AVI file before continuing processing the video. To reduce the filesize a bit, I will use Lagarith for lossless video compression.
The AviSynth script will load the source video, do some pre-processing before converting it into the YV12 color space needed by most AviSynth filters like NNEDI or WarpSharp. NNEDI will do the major upscaling before cropping, sharpening, and resizing it. VirtualDub then will encode the script.
List of the Tools
AviSynth 2.6.0 (still in alpha release)AviSynth MT 2.6
VirtualDub 1.10.1 (experimental build)
If either version does not work or crash, use the stable releases 2.5.8 and 1.9.11, respectively. To install the MT package (adding multithreading capabilities to AviSynth) follow the instruction that come with the archive (in case you use the stable version of AviSynth you'll need the proper MT package).
Lagarith Lossless Video Codec 1.3.26
Next are the necessary filters that are not included in AviSynth:
DeDot and Checkmate are optional for the pre-processing in case you have older video transfers that may reduce rainbows, dots and dotcrawl.
nnedi3 and WarpSharp are doing most of the work, though the sharpening filter is optional, both in its use and in its choice.
Since sharpening the image may also increase some unwanted grain, I'll apply a denoiser, the VagueDenoiser.
The AviSynth Script
Turning on multithreading for the script must come as the first line:SetMTmode(5,0)
After loading your video you may do the pre-processing (and you'll speed up multithreading):
SetMTmode(2)
Checkmate()
DeDot()
ConvertToYV12()
Next is NNEDI3. I have included most options even if I use on some the default so that I could have left them out.
nnedi3_rpow2(rfactor=4, nsize=3, nns=4, qual=2, etype=1, pscrn=4, threads=0, opt=0, fapprox=0)
Next I do the cropping. Why now and not before? It basically depends on the source material. Older transfers have often black bars all around that are relics of the scanning. They are not necessarily straight vertical and horizontal bars so it is better to see where they actually are. Modern digital video usually does not have that kind of bars unless it is letter- or pillar-boxed. In this case the cropping should be done before the upscaling.
Crop(1, 2, -3, -4)
The values will depend on your source video and is in the order 1=left, 2=top, 3=right and 4=bottom. They are always integers.
After that comes the sharpening and denoising. I have used the default setting on WarpSharp but actually you can pump it up quite high.
VagueDenoiser(threshold=2, method=1, nsteps=6, chromaT=2.0)
WarpSharp(depth=300, blur=100, bump=128)
VagueDenoiser(threshold=2, method=1, nsteps=6, chromaT=2.0)
Yes, I use the denoiser twice. Once to denoise the source material and then to smooth out side effects of the sharpener.
And finally the resizing to the target screen resolution.
Spline64Resize(960, 720)
But why use Spline64Resize, you might ask. Well, NNEDI only upscales to an enlargement factor that has a value between 2 and 1024 and has to be a power of 2. The target resizing is done by the internal resize filters of AviSynth and of those Spline64Resize appears to be the best. But so that it works best, I will enlarge the original image to something quite larger than my target resolution so that the resizing is actually a downscaling.
The script will be excruciatingly slow (about one frame each two-and-a-half seconds on my Windows XP with 3GB RAM on an Intel Duo at 3 GHz) but you'll have upscaled your video and improved sharpness a bit. Most usual sharpeners will fail, while sharpeners using warp technology can easily distort your image. You may increase speed by reducing NNEDI's enlargement factor rfactor to 2 and reduce WarpSharp's depth and blurring but you will most likely increase speed at the cost of increasing warping the image. Multithreading has increased the frame rate by maybe 60 per-cent.
Finalizing
After creating the script you will load it with VirtualDub. To limit any further conversion you will set in the Video menu the Color Depth to "Autoselect" for decompression and "Same as decompression format" for compression. And in Compression you chose Lagarith as your codec and configure it to "Use Multithreading" and "Prevent Upsampling When Decoding". The resulting video file will not play on most video players, including VLC or Media Player Classic, but still can be loaded into AviSynth using AviSource. If playback is an issue, try unchecking "Prevent Upsampling When Decoding" or use another codec (or no compression at all, though that will increase the filesize).Conclusion
This script is definitely not for your everyday needs. Its speed is so slow that you have to think twice. In many situation you may want to leave out the sharpening effect or even just use a Spline resize (there exists nowadays a plugin for AviSynth that use 10 or even 12 sample points instead of the 8 used by Spline64Resize).Above all this script is a play to see how far upscaling can be taken. If you know of a way to achieve an even better upscaling with improved sharpness while limiting or even removing any warp effect, let me know!
