Forum

Pages: [1] 2   Go Down
  Print  
Author Topic: ISO in Olympus E-M5 is way off  (Read 25279 times)
seta666
Newbie
*
Offline Offline

Posts: 5


« on: September 24, 2012, 06:35:22 PM »

<div id="linkdxomark">This a comment for <a href="http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Olympus/OM-D-E-M5">this page on the website</a></div>

For me is incredible the way ISO is off in the E-M5, almost 1EV off

Manufacturer ISO: 200   Measured iso: 107
Manufacturer ISO: 400   Measured iso: 214
Manufacturer ISO: 800   Measured iso: 394
Manufacturer ISO:1600   Measured iso: 782

In isos over 400 it is over 1EV off, worst I have ever seen. That is why iso 1600 looks clean, because it is not even iso 800

Anyway, seems a good 4/3 sensor but I find this iso thing very anoying

If you compare this to the GF3 or GF5 which are very well calibrated it would show very easy
http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/(appareil1)/816%7C0/(brand)/Panasonic/(appareil2)/793%7C0/(brand2)/Olympus/(appareil3)/763%7C0/(brand3)/Panasonic

Same shot in same lighting conditions with same exposure time, same aperture and same iso the E-M5 will always be 1EV underexpossed

For me this lying to the customers; I guess is better to say you have an amaizing iso 1600 than to tell the truth and say it is just iso 800
« Last Edit: September 24, 2012, 06:38:11 PM by seta666 » Logged
AaronMC
Newbie
*
Offline Offline

Posts: 38



« Reply #1 on: September 24, 2012, 06:47:02 PM »

I agree. This is lying.

It seems like Olympus would have done this to game tests, but one has to assume that they couldn't be that stupid. Dozens of publications do ISO tests to find out how accurate the measurements actually are, so the deception would have been immediately apparent to anyone who cared to look.

Unfortunately, the alternative is equally bad: Olympus is incompetent. In this hypothesis, Olympus simply has no idea how to design a camera correctly. This seems equally unlikely, since it's obvious that Olympus can, in fact, design cameras.

I guess, regardless of the explanation, this is very bad. But at least even with the numbers corrected, the E-M5 comes out looking good.
Logged

If you've got pants on, you're doing alright.
seta666
Newbie
*
Offline Offline

Posts: 5


« Reply #2 on: September 24, 2012, 07:00:49 PM »

Dozens of publications do ISO tests to find out how accurate the measurements actually are, so the deception would have been immediately apparent to anyone who cared to look.

I only know of DXOMark testing real ISOS

Fixing this is as simple as changing the menu isos to the real ones via firmware, something they will not do for sure.

 This practice is good to fool people that follow reviews like those of Dpreview, were no one care about the conditions of the shot.

But in real world people will have to use ISO 3200 in conditions where they would normally would use ISO 1600; maybe they do not complain because ISO 3200 on the E-M5 is as good as iso 1600 in other cameras. It is ISO 1600 indeed!!
  
 To be fair most companies (canon, nikon included) lie about real iso but to the date worst example was Fuji X100; the E-M5 has made a new benchmark about what lying is.

 This practices are unfair for companies like Leica or Panasonic which have very well calibrated sensors

Javier
« Last Edit: September 24, 2012, 07:03:45 PM by seta666 » Logged
hoodlum
Newbie
*
Offline Offline

Posts: 28


« Reply #3 on: September 24, 2012, 08:08:22 PM »

As mentioned in another thread DPReview only tests exposure variance with JPEG while DXO tests the exposure variance with RAW.  That is the difference you are seeing here.

Some E-M5 users have already reported that RAW is underexposed by 1EV vs JPEG when shooting RAW+JPEG.  Oly likely does this to help preserve the highlights.  So they likely push the underexposed RAW output of shadows and mid-tones by 1EV, leaving highlights underexposed therefore preserving them in JPEG.  You noticed from Dpreview the E-M5 had very high JPEG DR.

Fuji does something similar with the X series above ISO1000.
Logged
Ctein
Newbie
*
Offline Offline

Posts: 7


« Reply #4 on: September 24, 2012, 08:51:50 PM »

Dear seta666,

I think you have a misunderstanding of what the sensor ISO means. There are two fundamentally different ways of defining ISO.

1) The industry-standard definition of sensor ISO is based on the amount of light it takes to saturate the sensor (produce a value equal to pure white).  That's important to know, because any exposures greater than that just get clipped.  It can produce counterintuitive values. If you double the efficiency with which the sensor  captures photons and converts them to photoelectrons, leaving all its other characteristics unchanged,  then it only takes half as much light to saturate the sensor and you've doubled the  ISO. That make sense.

But… Suppose you make a sensor that just has larger pixels or deeper wells, so that it can hold more photoelectrons. Not changing anything else about it, just giving it a longer dynamic range. What happens then? Well, it takes more light to saturate the sensor. Which means its sensor ISO **DROPS**!  But it still has the same sensitivity to light at any given level. WTF?

As I said, counterintuitive.

This is one reason why many large format digital backs, with their huge pixels and huge dynamic range  (which means a huge exposure range) come in with such low ISO values. Their pixels aren't insensitive, but they are designed to be huge light buckets, so it takes a lot of light to fill them.

2) The industry-standard definition for camera ISO, and the one that makes the most intuitive sense to photographers, is based on evaluating real photographs to determine which exposure produces the best looking results. It doesn't directly connect to what exposure saturates the sensor.  That is just one factor.   For example, less exposure (higher ISO)  leaves you with more highlight headroom before you start getting clipping, but the picture will look noisier.  Conversely, more exposure (lower ISO) produces a cleaner photograph but your highlights will block up sooner.

One result is that you can get different camera makers deciding on different camera ISOs for the very same sensor, depending on what they think is the optimum balance for image quality. But almost always, they will choose  a camera ISO that is somewhat less than the sensor ISO (unless it's a VERY long-dynamic-range sensor) because blown-out highlights are unpopular.

Another consequence is that the camera ISO can change with the camera design, even with the same sensor.  The noise in the picture isn't just the result of the sensor characteristics, it's the result of the entire electronic chain, the amplifiers and all the rest.  Different electronics designs will produce different amounts of picture noise for the same exposure with the same sensor. That affects the balance between what are acceptable levels of noise and an acceptable range of highlight detail. Lower noise means you can increase the camera  ISO and still get good image quality.  But the sensor ISO hasn't changed!

That's why DxOMark gives you a plot of camera ISO vs. sensor ISO. It doesn't reveal that a camera manufacturer is “fudging” or “lying.”  It tells you how the two different kinds of ISO, which are legitimately determined in very different ways, compare.

That's all. It's not an exposé. It's just technical information.  So far as real photography practices go, that particular plot  isn't important.

      pax  \  Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
Logged
dosdan
Jr. Member
**
Offline Offline

Posts: 50



« Reply #5 on: September 24, 2012, 10:22:16 PM »

<div id="linkdxomark">This a comment for <a href="http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Olympus/OM-D-E-M5">this page on the website</a></div>

For me is incredible the way ISO is off in the E-M5, almost 1EV off

Manufacturer ISO: 200   Measured iso: 107
Manufacturer ISO: 400   Measured iso: 214
Manufacturer ISO: 800   Measured iso: 394
Manufacturer ISO:1600   Measured iso: 782

In isos over 400 it is over 1EV off, worst I have ever seen. That is why iso 1600 looks clean, because it is not even iso 800

Anyway, seems a good 4/3 sensor but I find this iso thing very anoying


You do understand that, with the advent of matrix/evaluative metering, manufacturers come up with their own idea of what is a "good" exposure. This means one manufacturer's ISO Sensitivity setting may be different from the other.

The Olympus ISO reference level is fairly consistent within its own brand:

http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/%28appareil1%29/793|0/%28brand%29/Olympus/%28appareil2%29/724|0/%28brand2%29/Olympus/%28appareil3%29/682|0/%28brand3%29/Olympus

Or, compare across manufacturers:

http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/%28appareil1%29/740|0/%28brand%29/Fujifilm/%28appareil2%29/689|0/%28brand2%29/Samsung/%28appareil3%29/623|0/%28brand3%29/Nikon


The manufacturers typically use either the REI or SOS method, but they don't apply to raw files, which is what DxOMark uses. See:

http://en.wikipedia.org/wiki/ISO_sensitivity#Digital

Quoting:
[color=brown]Because the output level is measured in the sRGB output from the camera, it is only applicable to sRGB images—typically TIFF—and not to output files in raw image format. It is not applicable when multi-zone metering is used.
[/color]

DxOMark has covered this:

http://www.dxomark.com/index.php/Publications/DxOMark-Insights/Pushed-ISO-Let-s-make-it-clear/RAW-ISO-measures-are-inferior-to-manufacturer-ISOs-is-this-a-problem

Dan.
Logged
Hakeem
Newbie
*
Offline Offline

Posts: 39



« Reply #6 on: September 25, 2012, 01:54:16 AM »

Ctein and Dan!

Thanks very much for the detailed explanation, intuitive ans some counterintuitive but nevertheless very useful and makes very good sense.

I still, however, feel as a RAW only shooter, to get the camera in RAW ISO settings instead of Manuf JPEG optimised one! Manuf can still point out, or we can find from our own testing, or through review sites, what is the best ISO for quality and can mark it ISO100 in our mind, for that purpose Smiley

This also explains why DPR High ISO RAW comparison shows better results for OMD than any other APSC Smiley I must agree here, I prefer DXOMark, now! However still one confusion on test results, TechRadar also uses DXOMark tools why they getting DR higher than any other in that test? Numbers clearly different from DXOMark! They are not calling it RAW but TIFF:

http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/olympus-om-d-1075717/review/page:5#articleContent

Logged

Hakeem - A Soft Coder
@ Flickr
Pany GH1, GF3, Oly Pen-EP1, XZ-1, Nikon D5100
Interested in Sony A580, Nikon J1
Waiting for Oly XZ-2, Pany GH3, Nikon D800/ Canon 5D-III
Ctein
Newbie
*
Offline Offline

Posts: 7


« Reply #7 on: September 25, 2012, 04:37:54 AM »

Dear Hakeem,

The methods the camera makers use to determine camera ISO with cameras that can provide both JPEG and RAW files out will produce the same ISO for both kinds of files. If they didn't, it would look very confusing to photographers, since what they see is a JPEG preview in camera. If JPEGs and RAW  files had different effective ISOs, the converted RAW photograph would look much lighter or darker than what they saw,  and they would be unhappy.

That said, just as back in the film days, you're always best off determining your own personal Exposure Index. Me, I tend to bias exposures towards the low side, even with the OMD, because the slight increase in noise doesn't bother me and I really hate blown out highlights. Your mileage will differ. If you decide you consistently want something different, you can program in an exposure offset, but meanwhile the conveniently-located exposure bias dial is your best friend.

I can't explain TechRadar's  exposure range results, which do seem anomalously high to me, but I don't think there's much point in trying. Measuring RAW exposure range requires you to make some assumptions about what your noise floor is going to be, and it is also sensitive to experimental design (dpreview has a poor  design for measuring the exposure range of very long-range cameras, in my opinion).

Basically, you shouldn't try to compare camera reviews across different sites (or magazines). So long as the testing procedures don't change, you can compare different cameras within one site, but you cannot expect different testers to produce the same results.  Sometimes it happens, but there's no guarantee.


      pax  \  Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
--     Ctein's Online Gallery    http://ctein.com 
--     Digital Restorations      http://photo-repair.com 
======================================
Logged
jyhfeei
Newbie
*
Offline Offline

Posts: 13


« Reply #8 on: September 25, 2012, 07:03:22 AM »


Basically, you shouldn't try to compare camera reviews across different sites (or magazines). So long as the testing procedures don't change, you can compare different cameras within one site, but you cannot expect different testers to produce the same results.  Sometimes it happens, but there's no guarantee.


Very true.  However, the comparative DR and color depth measured by DXO does not seem to agree with comparative RAW data from other testing sites.  For example, compare the RX100 sensor to the OMD sensor (with a 1 stop hit in ISO).  Competing sites I've looked at have the OMD with at least 1EV advantage in DR and ~1 bit in color depth even with the 1 stop ISO hit.  DXO has the sensors at parody.  The same lack of comparative agreement for DR and CD applies to NEX sensors as well.  Seems a bit fishy to me as the sensors are in the same family.

What I don't understand is why the test data took so long and what the "anomalies" were in the data to cause the months of delay.  Added "processing"?  Who knows.
« Last Edit: September 25, 2012, 07:27:16 AM by jyhfeei » Logged
Ctein
Newbie
*
Offline Offline

Posts: 7


« Reply #9 on: September 25, 2012, 07:50:21 AM »

Dear jyhfeei,

Huh?!?!

First off, there is no "1 stop ISO hit." Go read my earlier posts-- you're misunderstanding what the ISO chart is telling you.

Second, how does amount of time you had to wait for the test have any bearing on its validity?

Third, I can tell you, with 40 years of product testing and review writing under my belt, that there is absolutely nothing "fishy" about different folks getting different experimental results. It would be fishy if they DIDN'T!

Fourth, "sensors ... in the same family" doesn't mean damn thing. Nobody is testing bare sensors, they're testing sensors embedded in cameras, and the camera design (both hardware and software) will always make for differences in the resulting images, sometimes substantial ones.

pax / Ctein
Logged
seta666
Newbie
*
Offline Offline

Posts: 5


« Reply #10 on: September 25, 2012, 10:23:54 AM »

Well, for the ISO measurement that matters is the RAW iso; if they pushing an image 1EV via software they are not making good use of the sensor information. Almost half the information the sensor gathers is on the last EV to the right, this is why we expose to the right in digital. There is no need to loose information on the highlight if you expose well.

 Underexposing every shot 1EV to preserve highlights in JPGs it is one of the silliest things I have heard of. I do not really care about JPGs, I always use RAW

 DR goes both ways, to the shadows and to the highlight so if you are underexposing 1EV you are preserving nothing, you are loosing 1EV in the shadows. When you expose to the right you are the one to chose where the whites will clip; I rather be the one to choose, not some stupid algorithm. For scenes with only 7-9 EV  DR you can expose to the right safe and have all the DR needed and low noise, for high DR scenes you have to choose what to sacrifice.

In my opinion as user this is cheating, because if I buy a camera for its very good high iso (1600, 3200)just to find out that the RAWs are deliberately being underexposed and I have to bump the exposure time to get the right exposure (to the right)I will feel like a fool.

Medium format cameras may need more light to feed the sensor, I do not of anyone using digital backs for low light work. They do have a very clean base ISO which is what most MF users need.

 The E-M5 is being sold as a 4/3 low light beast with APS-C like performance and if you compare JPGs may look like it, at the end is only a software trick. I am possitive an E-M5 will need twice the exposure time over a Panasonic G5, even in JPG. I can not try it as I do not own any of this cameras.

I have nothing against 4/3, Olympus etc but this cheating pisses me off; I find it unfair for other companies like panasonic which gives their customers well calibrated sensors.

Regards
Javier
Logged
Ctein
Newbie
*
Offline Offline

Posts: 7


« Reply #11 on: September 25, 2012, 06:11:41 PM »

Dear Javier,

Well, I'm bowing out. Every paragraph in your last post contains some factual error or fundamental misunderstanding about how exposure works. I just can't address all that bad information.

If you want to go on thinking that this is somehow a huge cheat and a functional failure, nothing I can do to fix that.

I will leave you with this. Along with having the technical background and long experience to understand this subject that you don't have, I also have an OMD and have run my own tests and comparisons. Whereas you are theorizing based on a profoundly incorrect understanding of exposure determination and ISOs.

pax / Ctein
Logged
jyhfeei
Newbie
*
Offline Offline

Posts: 13


« Reply #12 on: September 25, 2012, 09:26:51 PM »

First off, there is no "1 stop ISO hit." Go read my earlier posts-- you're misunderstanding what the ISO chart is telling you.

I did read your informative comments of the ISO/DR/CD relationship.  Thank you.  I was not referring to the DXO ISO measurements in my comment.  I was attempting to equalize the DR & CD data from other sites.  It was my assertion that DXO uses a different ISO reference than other testing sites.  Perhaps this is wrong to make this correction.  If it is, the relative difference between what DXO states for DR & CD (RX-100 vs OMD) and what other sites state is even greater.

Second, how does amount of time you had to wait for the test have any bearing on its validity?

The testing was done on the OM-D at least 2.5 months ago noted by their own people.  The reason given not to release the info was "surprising" results.  I see nothing surprising about these results other than they do not align with the relative findings of other testing sites.  It would be good if DXO would explain what the surprising issue was.

Third, I can tell you, with 40 years of product testing and review writing under my belt, that there is absolutely nothing "fishy" about different folks getting different experimental results. It would be fishy if they DIDN'T!

I understand that there are differences between measurements between sites.  I was commenting to the relative difference between DXO's conclusions as compared to as compared to the conclusions of other sites.  In this manner, they do not track.  As stated, you should be able to compare cameras within one site and get similar results or rankings.

Fourth, "sensors ... in the same family" doesn't mean damn thing. Nobody is testing bare sensors, they're testing sensors embedded in cameras, and the camera design (both hardware and software) will always make for differences in the resulting images, sometimes substantial ones.

Perhaps, yet Panasonic/Olympus (Panasonic) sensors of the same size and vintage measure similarly.  Nikon/Sony/Pentax (Sony) sensors measure similarly as well.  From other sites, it appeared that the OM-D had higher performance than Sony NEX sensors.  Perhaps from added processing of the RAW data.  This higher performance does not show up in DXO's results.  

In the end, Olympus has done their own compromise between low noise performance, dynamic range, and color depth with the processing of the Sony sensor.  I'm just noting that the DXO relative test results are poorer (although still good) as compared to other sites relataive test results for the OM-D.
« Last Edit: September 26, 2012, 04:59:48 AM by jyhfeei » Logged
seta666
Newbie
*
Offline Offline

Posts: 5


« Reply #13 on: September 26, 2012, 12:16:54 AM »

Dear Javier,

Well, I'm bowing out. Every paragraph in your last post contains some factual error or fundamental misunderstanding about how exposure works. I just can't address all that bad information.

If you want to go on thinking that this is somehow a huge cheat and a functional failure, nothing I can do to fix that.

I will leave you with this. Along with having the technical background and long experience to understand this subject that you don't have, I also have an OMD and have run my own tests and comparisons. Whereas you are theorizing based on a profoundly incorrect understanding of exposure determination and ISOs.

pax / Ctein

Well, you may be right then. I do not own an E-M5 and I do not intend to own one. It is just that this manufacturer/measured iso thing bothers me somehow. Maybe it should not bother me but it certanly does, I can not help it.
 
So, what does your test show? better performance than simillar APS-C sensors? I don't think so....

Javier


« Last Edit: September 26, 2012, 08:41:44 AM by seta666 » Logged
Hakeem
Newbie
*
Offline Offline

Posts: 39



« Reply #14 on: September 26, 2012, 10:42:45 PM »

Hi everybody!

I can understand now the 1EV difference, quite clearly. Since its capturing the image at ISO100 roughly, but matching the exposure time to ISO200 by underexpsoing -1EV and then brightning up the shadows to match what you could possibly get at ISO200 with the equal brightness in shadows!

To be fair, I like this approach more than Nikon D-Lightning where the impact is only on JPEGS and I had to push up the shadow details in RAW myself in post processing tone-curve adjustment. And if its right out of the camera, its not only time saving, but also it also gives me a degree of confidence that Oly have done something more sensibly within their Processing pipeline, that I dont have to bother now.

The only negative impact it could possibly have is more noise in the shadows, but if am not seeing it much, Oly has all the right to call it whatever ISO they want, provided its not giving me a penalty of taking a longer expsosure (slower shutter speed) comparative to lets say Pany G5 at equally rated ISO; so am agreeing with @Ctier here. I would like to request if you can share your OMD results with us.

But I can also completely understand the frustration @Javier is having here, but on the fair side, if there are still cleaner images at equal ISOs, ISOs-scores should be relative. And as Oly sacrifices ISOs for DR then we should have clearly seen a massive difference than what is measured. The only score I could accept here blindly is maybe a lower Color Depth, but I guess Oly scored pretty well on it. But Javier I am bit more concerned with DXO results than you as I am still quite keen in upgrading my m43 body and depending on GH3 resultsheet, I may decide to go for OMD, will see. That is why I really like to clear my doubts and confusions around DXOMark testing process.

@jyhfeei
same sensors behaves slightly differently in different bodies, depending on the underlying electronics and assembly designed around it. Oly with an IBIS for the same reason was giving some bad results compard to Pany with OIS in lenses. Sony SLT using the same sensor with their aXXX DSLRS and NEX range have visible differences too. Even DSLRs from Nikon, Sony and Pentax got some marginal differences, because of slight differences of electronics, processors and even lenses, on all these outputs higher ISO, CD and DR. But the trouble is that in this test result the massive difference is huge and very visible and I agree results are posted very late.

I don't wanna blame DXOMark for anything immoral here, as they must be doing quite hardword here producing these results but I think they should bring more agility and clarity to their testing process and should also mention what lenses they are using while testing these cameras.

Also if these test results inconsistency is due to their normalisation on Image size to lower resolution, probably they should re-consider it.

Regards!
Logged

Hakeem - A Soft Coder
@ Flickr
Pany GH1, GF3, Oly Pen-EP1, XZ-1, Nikon D5100
Interested in Sony A580, Nikon J1
Waiting for Oly XZ-2, Pany GH3, Nikon D800/ Canon 5D-III
Pages: [1] 2   Go Up
  Print  
 
Jump to: