« The Digital Bits | Home | New Feature in Windows 7 Revealed: XP Mode »

Individual Entry With Comments


April 10, 2009

David vs. David: Blu-Ray/DVD Taste Test

Filed under Gadgets, HDTV, Movies, Work

image

First, some background.

I showed David Bullock, a buddy of mine from work, a blog post I did about HD-DVD vs. DVD and how the difference for a typical TV size and viewing distance is not as obvious as you would think.

We got in several “discussions” about the topic. DavidB thought it would be easy to tell the difference between Blu-ray and DVD. I claimed it would be difficult.

Who would be right?

Arguing got us no where. We decided to settle this like they did in the Pepsi Challenge: a “Blu-Ray/DVD Taste Test.”

The loser of the test has to take the winner out for dinner at a restaurant of their choosing.

The taste test had the following rules:

  • 10 popular movies of David Lenihan’s choosing
  • Play a scene of David Lenihan’s choosing
  • May be all DVD, all Blu-ray, or a mix
  • Played in random order
  • No replaying
  • No pausing
  • Played on a PlayStation 3 connected to a Sony XBR3 46” LCD via HDMI
  • TV/PS3 running at 1080p
  • Tuned with Blu-ray Digital Video Essentials
  • 1:1 Pixel Mapping for Blu-ray content
  • Test subjects cannot watch video setup or when video stops
  • Viewing distance of 6ft or more
  • Scoring is as follows:
    • 1 point for each movie identified correctly as Blu-ray or DVD
    • -1 point for each movie incorrectly identified
    • 0 points for blank answer

I invited people from work to join in the fun.

image

From left to right: CharlieK, Ed, DavidB, Shelly, Greg, An, and Ted.

Here are the movies I showed:

  1. Death Proof - DVD
  2. Matrix – DVD
  3. Dark City – DVD
  4. Pixar’s Lifted – Blu-Ray
  5. Reservoir Dogs – Blu-Ray
  6. Close Encounters of the Third Kind – Blu-Ray
  7. Sleeping Beauty – Blu-Ray
  8. Terminator 2 – Blu-Ray
  9. Blade Runner – DVD
  10. Sin City – DVD

I had everybody predict what their score would be on the scorecard.

Here are the results (click on the names to see the actual scorecard)…

Place Name Predicted Score Actual Score
1. Greg 5 6
2. (tie) An (left blank) 0
2. (tie) Shelly 5 0
4. (tie) DavidB 6 -1
4. (tie) Ed 3 -1
6. CharlieK 2 -2
7. Ted 6 -3

 

Some interesting results:

  • Everyone picked Lifted as Blu-ray, and it was
  • Everyone picked Sin City as Blu-ray, but it was DVD
  • A score of zero is what you would expect if someone guessed on every movie. Only Greg beat the guessing score.
  • Ted did 9 points worse than he predicted, the biggest disparity
  • Greg was the only person that did better than they predicted, by 1 point
  • The average score was –0.14, which is worse than the guessing average

So where should I have David take me to dinner? Post your suggestions in the comments.

This was a lot of fun…thanks to my buds from work for coming out and being great sports!

Comments (26)

Interesting results. Since the Pixar film was digital start to finish, that result doesn't surprise me. Sin City is so stylized that you can't tell what's an artifact from an artistic choice (not a criticism, still a cool looking movie).

So with this in mind, how did the testing score based on how the films were MASTERED? Is it a case that film grain, optical processing fuzzies, etc., in the older films are masked by the lower definition DVD, or some other process going on?

Glad to see you're using your genius for good and not for eeevil.

How can you find out how a disc was mastered?

Sooo...on my 37" 1080i LCD a PS3 would be best utilized for games? Whoa!

Why those particular DVDs?

What scenes?

We've had our LCD tvs (32", 37", both 1080i) for over a year, and at 12-14' viewing distance I can see huge difference between SD and HD sources (Time Warner cable, HD DVRs), even though Time Warner compresses more. The frustrating thing? My wife has NEVER mentioned noticing a difference!! I have to basically goad her into sitting there with me while I flip between SD and HD versions of the same channel. When forced, she admits there is a difference, but it's so not a big deal to her.

That, plus your Pepsi Challenge on my setup being even harder to see the difference tells me I won't get much mileage out of using the Blu-Ray capabilities of the PS3 as a selling point! Hello, Little Big Planet!

I picked 7 movies that I had on both Blu-ray and DVD. I flipped a coin to decide which one I would use.

The other 3 were movies I had on DVD, but not Blu-ray. I checked to make sure Blu-ray editions existed.

I usually picked memorable scenes from a movie:


  1. This-car-is-100%-death-proof scene in Death Proof

  2. Neo-first-uses-his-eyes scene in The Matrix

  3. Jennifer-Connelly-sings-"Sway" scene in Dark City

  4. First part of Lifted

  5. "Stuck in the Middle with You" scene in Reservoir Dogs

  6. Richard-Dreyfus-first-encounter-at-the-train-tracks scene in Close Encounters

  7. "Once Upon A Dream" scene in Sleeping Beauty

  8. John-Connor-meets-the-T1000 mall scene in T2

  9. "He say you Blade Runner" scene in Blade Runner

  10. Jessica-Alba-dancing scene in Sin City

If I did this test with HD/SD cable movies, it would have been obvious the difference. The main reason is because SD cable movies aren't displayed "anamorphic" like DVD's are.

I don't think compression artifacts ever came up as a reason why people chose DVD/Blu-ray.

I mentioned compression just to throw out a dis at shitty Time Warner. Their HD feed isn't really HD in what I think of it, and when I'm watching a concert in "HD" and the camera pans, turning the screen into compression artifacts, my eyes roll. It may have nothing to do with picking Blu-Ray over DVD.

An:

Thanks for an educational, entertaining evening! But readers should note the grains of salt discussed during the test: numerous scenes were darkish and the [older movie] source/master material often was of limited fidelity closer to dvd than blu ray. Bright or sunny scenes from hi fi masters would make blu ray much more readily distinguishable.

ps i don't own blu ray or even hd tv =)

OK, let me chime in on how things really went last night.

First, the question I thought I was going to be asked was: “can you tell the difference between blu-ray and DVD at 6 feet from a 46 inch HD TV”. That was no problem. But the questions we ended up answering was “are there case where blu-ray quality can drop to the point where you can not tell the difference between DVD and Blu-ray?”. The answer to that is yes. Most of the blu-ray scenes that confused people were dark, foggy or otherwise a poor condition for quality. It wasn’t the case that the DVDs looked so good but finding a place where blu-rays look bad. I’m sorry but saying you randomly selected 5 out of the 7 blu-rays you own is not random selection. The selection happen when David first picked the blu-ray movies in the first place. Then picking a particular scene that is very low quality is also is not random. A better way would have been for a third party person to randomly select a blu-ray from a larger list, and then randomly select a point in which to view it. Lifted and Sleeping Beauty blu-rays pretty much everyone got. They were good examples of Blu-ray quality. The other blu-ray examples were taken from scenes where quality was low. I mean come on, the scene from Close encounters was a dark foggy night. No amount of resolution could make it look sharp. It’s hard for me to believe that this was a random selection.

The only DVD that looked like Blu-ray quality was Sin City. This is a black and white film. The high contrast and David having his TV set to a high sharpening setting made it look very nice.

The “Shelly” test which we ran after David’s test was more telling. We played a disc from the BBS Planet Earth series in HD and in DVD. These are both good examples of transfers and we could compare the same scene in both DVD and HD quality. People didn’t have any problems seeing the differences between the examples even at distances greater than 6 feet. Viewing distance is important but with the setup last night it would probably be more around 8-10 feet before you could not tell the difference between the two.


Shelly

@ David
Not the disc master, the film. The older ones are optically processed start to finish, while some of the newer ones were shot on film and transferred to digital for FX, compositing, etc. Then of course Pixar has everything digital start to finish.

My guess is based on my experience with the Star Wars laserdiscs-- you could actually see the garbage mattes on a lot of the scenes, due to a change in contrast when they did the transfer. So I'm guessing the source material affects the way the movie shows up after its transferred.

BR might allow a lot of defects to show through that DVD covered up, thus giving the impression that the DVD was of higher quality.
But then I'm just guessing.

Greg:

One definite conclusion is that if you bought a great TV and blu-ray player, you ought to trade it for my setup, since I can see the difference =)

Bubba:

This site ranks Blu-Ray movies based on “quality” of movie based on different criteria and then placed into one of six different categories. Tier Zero - Blu is reference quality while Tier 5 - Coal is unacceptable quality.

The ranking is subjective but it must meet certain criteria to make a specific ranking and rankings can be challenged. Also, “For the purposes of this tier system we do not take director's intent into consideration when evaluating the visual quality of each Blu-ray. This list represents an absolute ranking system, where every available Blu-ray's picture quality is directly compared against every other release. Those who are interested may want to use the thread search feature to peruse the individual reviews that contributed to these placements for further clarification. Tier placements can change over time as feedback warrants.”

That being said, here are the rankings of all the movies you tested with. I went ahead and ranked movies that you tested with as DVD with the Blu-Ray ranking. Two of you movies have not been ranked yet.

Tier Movie
NA   Death Proof - DVD 
1    Matrix – DVD 
2    Dark City – DVD 
NA   Pixar’s Lifted – Blu-Ray 
3    Reservoir Dogs – Blu-Ray 
2    Close Encounters of the Third Kind – Blu-Ray 
2    Sleeping Beauty – Blu-Ray 
4    Terminator 2 – Blu-Ray 
2    Blade Runner – DVD 
2    Sin City – DVD

I would be interested to see you test using movies only from Tier 0 Blu. Those movies are considered reference movies and have the best transfer to Blu-Ray.

Great list! Is there one for DVD?

Some notes...

Death Proof is similar to Planet Terror, which is Tier 4 (on Blu-ray...this was DVD).

"Lifted" is Tier 3 from "Pixar Short Films Collection, Vol. 1". However, that includes a bunch of really old shorts of varying quality. "Lifted" was also in "Ratatouille", which is Tier 0. I would put "Lifted" in Tier 0...it looks flawless.

Sleeping Beauty is Tier 0 (not Tier 2), and some people thought this was DVD.

The Blade Runner I showed is from 1996, and it is *terrible*. It doesn't even fill the screen horizontally. It was the worst looking movie I showed, but some people still picked Blu-ray for it.

People complained about how poor Close Encounters looked. After the test, I re-ran Close Encounters to show there was a lot of detail they could not see from where they were sitting (i.e. it was obvious if you were sitting close).

Sin City Blu-Ray is tier 1, not 2 (I showed DVD, though).

For your Tier 0 test to be fair, I need a list of *DVD* titles that are Tier 0. Also, my TV is tuned for Blu-ray. I should have shown DVD with settings that are best for DVD.

In a battle of Tier 0 DVD and Tier 0 Blu-ray both shown in their best settings, I would expect similar results.

Russ Urquhart:

As there are a lot of variables at play here, i think i would have selected those films that have had a new master done for them, that was THEN used as the basis for both the dvd and blu-ray version of the disc. (The new version of the Godfather is a good example of this.) As some dvd's and blu-rays are only as good at the master they are coming from.

Also, you may have mentioned this, but did you turn your PS3's upsampling of dvd's option off? (Or was that part of the test as well?)

And finally, as an owner of a HD-DVD player as well, do you find that your HD-DVD's look better than your blu-ray? I don't know if its just that i started with HD-DVD, but they just seem that way to me!

fwiw,


Russ

The PS3 (or is it my TV) is upsampling the DVD.

I don't consider that an issue...I'm showing DVD's at their best, just as I am with Blu-ray.

In my setup, Blu-ray has the advantage that I did all my tuning with the Blu-ray Digital Video Essentials...which may actually hurt the quality of DVD's with the same settings.

I don't see a difference between HD-DVD and Blu-ray. I'm just glad we have a winner so I can start my HD library.

Bubba:

Russ probably has a Mac. Oh snap... Apple does not included any Blu-ray hardware in stock Macs. You can't view a Blu-ray movie on a Mac unless you are running Windows XP or Vista under Boot Camp.


Sources:
http://www.engadget.com/2008/10/14/steve-jobs-calls-blu-ray-a-bag-of-hurt/
http://www.tuaw.com/2009/04/09/second-generation-blu-ray-ships-for-mac/

How far along is the technology relative to the point where the screen resolution would exceed our capability to detect any differences? Is it live, or Memorex? or Blu-Ray? :) Is there a way to put a number on this? i.e. it's at 70%, 80%, 90%? What are the limiting factors? Where is the weak link in the system? Or is the technology already there in R&D labs? and just not commercially viable? Cheers, John

This chart answers the question of resolution/screen size/viewing distance and the ability for the eye to detect a difference.

Net result: we all need ginourmous TV's to take advantage of Blu-ray....or sit *really* close to your small TV.

The only problem is that chart is wrong.

My viewing distance in my living room is roughly 13', and my TV is 37". That table says I should basically see no difference between 480p and 1080p.

Nope!

Remember seeing the difference between the non-HD and HD channels when you were here? There is clearly a difference - just check the fine print on pharmaceutical ads. This "chart" says I will need a 50" screen to start seeing the benefits of 720p, and just over 70" to see a benefit of 1080p!!

Who made this chart - a consortium of television vendors? What's the original source?

Obviously I'm using an HD cable source instead of DVD vs Blu-Ray, and obviously I know a larger TV is necessary the farther one's viewing distance, but this chart doesn't work.

.trey

Broadcast television is an exceptionally poor example of 480p. Use a good DVD (Sin City or any Pixar movie, for example) for comparison.

I got the chart from here.

Read the article and tell me if you still disagree with the chart.

Bubba:

From Crutchfield

Q: What size screen should I get for my room, and how far away should I sit for the best picture?

A: It depends on how big your room is. For a bedroom, kitchen, or office, where you'll probably be pretty close to the TV, you can go with a relatively small screen: 32" or even 26". But for the main TV in your living room or home theater, we recommend getting the largest screen that fits your budget and room. That's based on our own experiences and on feedback from our customers.

Sitting too far away from a smallish screen will reduce the impact and immediacy of the viewing experience. On the other hand, if you're too close to a large screen, you may be distracted by the screen's "pixel structure" — the grid of tiny picture elements that form the TV image. Compared to conventional tube TVs, digital HDTVs have higher-resolution screens (and more space-efficient cabinets) that let you put a larger set in your room and/or sit much closer.

High-quality video material like DVDs and HDTV programs look amazing on these new TVs. But some people also find that noise and distortion in lower-quality analog signals (like standard broadcast and cable) are exposed and magnified. That's why our viewing distance chart (below) offers a range for each screen size. If most of your viewing is DVD-quality or better, you'll see more details by sitting closer. If you watch more regular (non-HD) video, sit farther back for a smoother picture.

We calculated the viewing ranges in the chart below by multiplying the screen size (measured diagonally) by 1-1/2 and 2-1/2.

Screen size Viewing distance range
26" 3.25 - 5.5 feet
32" 4.0 - 6.66 feet
37" 4.63 - 7.71 feet
40" 5.0 - 8.33 feet
42" 5.25 - 8.75 feet
46" 5.75 - 9.5 feet
50" 6.25 - 10.5 feet
52" 6.5 - 10.8 feet
55" 6.9 - 11.5 feet
58" 7.25 - 12 feet
65" 8.13 - 13.5 feet
70" 8.75 - 14.75 feet

Recommended viewing distances for high-definition TVs. We suggest a range because the ideal distance will depend on several factors, including signal quality.

http://www.crutchfield.com/S-iQBBqi21uiJ/learn/learningcenter/home/tv_faq.html#1

I still think the chart is misleading.

Since the article never mentions DVD/Blu-Ray, we can apply it to my non-DVD/Blu-Ray comparison.

Poor example or not, let's call whatever comes out of my TV on standard definition cable channels "Resolution A." Let's call whatever comes out of my TV on whatever Time Warner calls a "High Definition" channel "Resolution B."

Only a fool would argue that Resolution B is not better than Resolution A. For the sake of arguing in blog comments, let's just approximate Resolution A as 480p. Resolution B would have to be higher than that - we can just leave it as that approximation for now. I don't even care if it is truly in full HD...the resolutions are somewhere around 480p and somewhere above that but below 1440p.

The chart says that at my viewing distance for my TV size, Resolution A and Resolution B "appear to be equivalent."

I'm not sure what to say besides this is incorrect.

I'm positive CB knows more about resolutions, HD, home theater, etc, than I do. If this data and chart only apply to viewing DVD/Blu-Ray, then that needs to be disclosed somewhere in the article.

.trey

Carlton never mentions what the source material is because...there isn't any. This is a discussion about the limits of what the eye can see in terms of HDTV resolutions.

I used his chart to pick my TV size. I wanted a TV that could show HD content, but SD content would still look good. I'm *very* happy with my decision.

Mike Schriever:

I think you're size chart is jacked. I watch 154" from 12 feet and it's great. If it were economical, I'd cover every square inch of wall in my house with TV.

37 inches? I expect more from somebody "in the biz" and a gadget geek. Time to step up you're game.

I think you're reading comprehension (and grammar) is jacked.

My set is 46 inches and I sit 5-6 feet away.

MFC:

This whole thread is jacked. I prefer my 250 lb. CRT 27" color TV, and I sit wherever the hell I want. I am not a nerd.

yes you are.

Mike Schriever:

46"... my bad. It's good to see you stepped up to a bathroom or kitchen TV.

Carry on.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


About

This page contains a single entry from the blog posted on April 10, 2009 1:25 AM.

The previous post in this blog was The Digital Bits.

The next post in this blog is New Feature in Windows 7 Revealed: XP Mode.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.34