Vibration Reduction compared with handholding (April 2, 2008)
Handheld photography with Leica and with VR technology
It is well known that the best results will be obtained when you take pictures with the camera on a sturdy tripod. When you want to capture the finest possible details on a pixel surface or in a silver halide emulsion, a tripod is a must: in the studio and on location.
Who wants or needs to photograph with an unsupported camera had a few options: use a monopod, look for additional support (a wall, a tree, a fence) or train yourself to steady the camera: use the elbows and forehead to create a triangle, hold your breath and so on.
There is some cult around the perceived ability for taking handheld shots with extremely slow shutter speeds. Everywhere you can find claims that handheld shots taken with 1/4 of a second are tack-sharp (whatever that means).
Leica users claim that using a CRF is one of the best options for high quality imagery when slow shutter speeds are required. One of the main advantages of using a CRF camera is the ability to use slow speeds and still get good results. The absence of the mirror shock and a smooth pressure movement to unlock the quiet shutter help in stabilizing the camera body. At least that is the theory. Now with many slr-camera’s offering VR-technology in several versions (in camera with a moving sensor or in-lens with moving lens elements) and claiming a three to four stop advantage, one might be tempted to study those claims.
Companies like Canon and Nikon claim that their system of optical stabilization is best, because the lens designer can incorporate into the design the additional lens groups for the OS. And knowing the lens properties, that stabilization can be optimized for every lens.
Optically speaking unsharpness by movement looks like unsharpness caused by image defects and therefore lens stabilization can take into account the characteristics of the lens.
Mechanical stabilization (as used by Olympus or Sony or Pentax) employs a movement detection sensor in the camera, that senses the direction and magnitude of movement and uses a program that moves the sensor in the opposite direction.
Every company claims that their system helps to improve the image quality by three to five stops, based on proprietary tests, never telling what these tests are. Basically we assume that you can take a picture at 1/15 with VR and getting equal performance as with a 1/250 without VR.
Another point of dispute may be the definition of sharpness: are you looking at the reproduction of very fine detail at magnifications of 10x or 20x? Are you looking at sharp subject outlines in scenes where there is hardly any detail. We know from psychology that a fuzzy picture with one or two eye-catching sharp elements is interpreted as sharp.
What then is the benchmark for sharpness?
I used the Siemens star as a test object and The Imatest program to analyze the results.
Te Siemens star is divided into eight segments that define different orientations of the lines in order to see image defects: the stars with number 1 and 5 are the horizontal direction, 3 and 7 the vertical direction and the others represent the skew direction in a diagonal sense.
Test charts nowadays represent linepairs per picture height and not linepairs per mm as in the usual MTF graphs. Both have merits and uses, but you cannot mix them. It is common sense to divide the bandwidth of possible resolution in three broad bands: the main subject outlines are represented by the 5 and 10 lp/mm, the finest textural details are represented by the 40 to 50 lp/mm and the medium frequencies (around 20 lp/mm) represent the texture and plasticity of the image. When you are using lp/mm, you are not interested in the size of the capture area, but only in the absolute performance of the lens. But we know that the size is an important parameter when enlarging the negative. For a given print size, we need a certain amount of linepairs to define the image. We know that the eye can discern at best a few linepairs per mm on the print at normal viewing distances. The norm here is the A4 print for a 35mm negative and this implies an enlargement factor of ten times. The 10, 20 and 40 lp/mm are derived form this magnification factor.
With the introduction of the digital sensor a new definition has been introduced, derived form the TV research. Now the number of lines per picture height is important as the TV screen has a finite number of display lines. Having a picture that can handle more lines that the screen can display is a waste of money and effort. Photographic sensors then use the linepairs per picture height as reference. The fact that a sensor will reproduce 1000 lp/ph is not a good measure for image quality, as there is no reference to the sensor diagonal. An E-3 sensor is much smaller than a D3 sensor and when both have a resolution of 1200 lp/ph the optical performance is quite different. The D3 sensor has a picture height that is twice the size of the E-3 and when both have the same performance in lp/ph, the actual optical performance of the E-3 is twice that of the D3. This makes sense because of the fact that the E-3 sensor captures as much detail as the D3 sensor on a much smaller sensor area. The smaller E-3 sensor must fill the A4 print with the same amount of detail that the D3 captures on a sensor that is much larger. Recall the classical discussion between the Minox format and the 135 format or the 135 format versus the 120 format. Minox lenses were invariably better than lenses for a Mamiya 6x6 as example, but in the final print always the larger format was always the winner.
The translation of the classical 10, 20, 40 lp/mm into the lp/ph domain is as follows: subject outlines are captured with 200 lp/ph, fine detail with 400 lp/ph and the finest textural details with 800 lp/ph. You could go one step higher and use 250, 500 and 1000 lp/ph. But one should beware for number fetishism! And when operating in the digital domain, one can always improve the image with post processing.
Test design
I have analyzed the Olympus E-3 camera (with in-camera VR) and the Nikon D3 (with in-lens VR) and the Leica M8 (without VR). All three were equipped with a lens that matched the sensor size to get the same image size: the 2/50 Macro for the E-3, the Apo-Summicron 2/75 for the M8 and the Nikkor 2.8/105 Micro VR for the D3.
At first all cameras were used on tripod to establish a benchmark. This test also allowed me to compare the image quality of the three cameras: the small sensor of the Olympus, the bigger sensor of the Leica and the biggest sensor of the Nikon. All sensors offer above 10 Mb pixels, with the Nikon having 12 Mb, not the maximum number of pixels for a sensor size that is equal to the size of a 135 film negative. A Siemens test chart was used and a 2 meter distance was chosen.
The benchmark test can also be used to compare the image quality of the lenses and the three cameras. As the shutter speed is not important when using a tripod, one would expect a gradual increase in performance. With handheld shooting, the reverse is true. Slower shutter speeds invariably degrade performance, but there is an extensive debate about the limiting speed where the quality drop will be visible. The classical rule (and a myth!) states that the lowest usable shutter speed is the inverse of the focal length: the lenses used in this test could then be safely operated at 1/125. I used speeds from 1/250 and lower to find out where the limit is.
It has always been my view that shutter speeds above 1/500 and even 1/1000 are required to approach tripod quality. This sounds overly conservative and contradicts the claim of many users who state that they can get ‘tack sharp’ pictures when using speeds as low as 1/8. I am not sure how to define this concept of ‘tack sharp-ness’. The results of the tests conducted in this report, may point in the correct direction.
Tripod quality
It is not my primary intention to compare the three camera models as such, but nevertheless the opportunity to compare the sensors with three excellent lenses, all of them highly regarded in the public space, is too good to let it pass.
First the Nikon D3 with the Nikkor VR 2.8/105. At f/2.8 overall contrast is excellent with very good rendition of very fine detail. The maximum frequency is 0.48 cycles/pixel and a contrast of 0.2 is available at 1200 lp/ph. At f4 the overall contrast at the medium frequencies improves visibly and detail definition crispens the fine lines. At f/5.6 the optimum is reached and a contrast of 0.2 is now available for 1300 lp/ph and the critical mid frequencies are of very high contrast indeed. At f/8 the definition of fine detail suffers and the homogeneity of the image breaks up visibly
Second the Olympus 2/50 Macro lens attached to the E3. The E3 sensor has 10 Mp on a small area and can be seen as the equivalent of a very fine grained film emulsion, where the D3 has a coarser structure and a higher speed, equivalent to a high speed film emulsion. Tri-X compared to Pan-F if you wish. Wide open the Olympus lens resolves 0.6 c/p in accordance with its fine grain structure. The lens/sensor combo offers a contrast of 0.2 at 600 lp/ph, but overall contrast is on the low side. At f/2.8 contrast improves as does the definition of very fine detail, but one needs f/4 to get the results that can match the Nikon system. The critical medium frequencies are still of lower contrast in the Olympus image and in a direct comparison the Nikon image has more punch. Post processing may improve the Olympus performance, but at its basic settings, the Olympus lacks the bite. At f5.6 the Olympus is quite close to the Nikon at f5.6, an excellent performance, given its much smaller sensor size.
Third the Leica 2/75mm lens. This is not a lens specifically designed for macro work, but its floating design should compensate for this at least partly.
At f/2 the Summicron has a maximum frequency of 0.54 c/p, again close to the theoretical maximum. A contrast of 0.2 is available at 1200 lp/ph, and overall contrast is significantly better than what we get with the Olympus lens at the same aperture. Contrast at the critical medium frequencies is much better with the Leica lens. At f/2.8 the Leica lens is still visibly ahead of the Olympus and even close to the Nikon system: the Leica is better at medium frequencies, but loses a bit in the definition of the very fine frequencies: here the Nikon has a tighter image. At f/4 the Leica has the edge over the Olympus and the Nikon in overall contrast, but the Nikon wins in homogeneity of the image, as can be seen from the graphs which are close together. At f/5.6 the quality gaps do narrow to a point where it is more a question of philosophy what one will prefer as the best lens. A contrast of 0.5 is reached at 900 to 100 lp/ph for the Olympus, 1000 to 1200 for the Leica and the same numbers for the Nikon. Definition of fine detail is best with the Nikon, then very close the Leica and then with a wider margin the Olympus.
The results for the Leica and the Nikon are close, which is a bit surprising considering the sensor size and amount of pixels of the D3. Where the Nikon wins easily and with superiority is the high ISO speed quality. The performance of the Olympus was a bit below expectations given the good reputation of the Zuiko lens. But reputations in the public domain may be based on different aspects of the lens/camera system and on different benchmarks. I will return to this topic later. I would propose that the Olympus lens might be the victim of the camera designers’ wish for a very fast AF system that may be detrimental to accuracy of focus. I did repeat the test for several days under identical situations and got every time comparable results.
Handheld results
What can we expect from these lens/camera systems when taking pictures handheld in normal situations? The Nikon feels rock solid and has a very ergonomic grip, the Leica has much less weight, but feels secure in the hands, the Olympus feels a bit uneasy, despite the fact that the body is solidly built. The 50mm lens used here is a light weight design, which might constitute a problem area as the system is a bit unbalanced in the hands.
The Nikon system at 1/80 presents an image with good definition of subject outlines and a rapid loss of fine detail and contrast at medium frequencies. Really fine detail is lost completely. The good results are a function of the excellent shape and weight distribution of the camera system. It is well known that the human visual system is very easily fooled by the fact that one single sharp detail in a picture that is overall quite fuzzy gives a good sharpness impression. This fact may help explain why many persons may se a sharp picture when there is none. At 1/25 of a second the Nikon is operating at its limits.
The Olympus system at 1/100 presents an image where the lower frequencies are reproduced with good contrast: the 20% threshold is reached at 500 lp/ph, but after that the definition of fine detail is suppressed completely. The impact of the picture is therefore quite good, even with the absence of fine detail definition. At 1/50 subject outlines are reproduced with good contrast, but overall contrast is low. At slower speeds the results are unpredictable: sometimes sharper, sometimes quite fuzzy.
The Leica can compete with the Nikon till 1/60. From 1/125 till 1/60 the image stays at the same level and is quite usable. At 1/15 the end of the line is reached. The Leica has manual focus and moving the focus ring while trying to steady the camera is not easy.
While most of these images are usable at not too high a magnification, representation of fine detail is absent and not even in the vicinity of what the tripod pictures do record.
Overall one might say that a shutter speed below 1/200 does not do justice to the potential image quality in all three cases. Everything below 1/50 is a lottery option: sometimes the picture is OK (for subject outlines, not for fine detail!), but most often it is not. At 1/60 the Leica offers best performance, with the Nikon close and the Olympus somewhat lagging.
Handheld photography at slower shutter speeds may be necessary to get that picture you want, but under these circumstances really fine detail is not recorded at all.
The introduction of VR technology.
The Nikon offers in-lens vibration reduction and at 1/100 the results are very close to what you get on tripod!! Only the very fine details get lost in the process, but a contrast of 20% at 1200 lp/ph is an excellent performance. This result holds for shutter speeds as low as 1/15. One may say that VR at 1/15 is much better than no-VR at 1/180. For number aficionados this implies an improvement of 3.5 to 4 stops.
The Olympus at the stronger VR setting presents a picture at 1/200 that is quite close to the tripod quality. Only the definition of very fine detail suffers visibly. This quality holds at slower speeds till 1/60. At 1/30 and 1/15 overall quality drops visibly., but is still better than what you can get with unaided handheld shooting. Generally one can say that the VR at 1/30 produces better results that 1/200 at simple handholding situations: a gain of 2.5 to 3 stops.
VR works! That is the basic conclusion and it works very effectively: a tripod is only needed when the full potential of the lens/camera system is required. The Nikon in-lens performance is definitely better than the Olympus in-camera performance, but the Olympus is still ahead of the Leica handholding option.
Handholding with some support.
In addition I made pictures with the cameras and some additional support: a wall or a table. The Nikon at 1/80 with support delivers almost the same results as the VR option at 1/100. There is a slight overall drop in contrast at medium frequencies, but not that much! At 1/25 the support technique is no longer a viable option compared to the VR technology. At 1/15 VR is vastly superior to the handholding with support situation.
The Olympus with support at 1/250 is much better than the simple handholding situation and not fare removed from the VR and tripod conditions. At 1/15 the handholding option is not good at all, the support condition delivers an acceptable picture, and the VR offers good quality.
The Leica has no VR, so the best you can do is finding some support. At 1/180 the support is quite effective and not far removed from the tripod condition. At 1/15 the supported Leica camera produces still quite acceptable results.
Upshot.
Best quality asks for a tripod. Period. Every camera sees a significant drop in performance when being used in handholding conditions. You will not see this at low magnifications, but when you look for fine detail, you will only see fuzzyness. Any support you can find is definitely helpful. The VR of Nikon is surprisingly effective and a true alternative to he use of a tripod. The Olympus VR is less effective at least with the lens chosen in this comparison. The Leica can be used with good effect when handholding is the only possible choice, but compared to VR there is no contest. At slow speeds the Leica might be at an advantage, but the chance factor cannot be disregarded.