First Mir benchmarks

June 28, 2013 – 2:14 pm

The first neutral (i.e. not published by us) benchmark of Mir is out. Michael over at Phoronix has a good write up of the current state of things and also mentions that the install was smoother than anticipated.

The results (~about 10% median, 15% average penalty in FPS, see below) are totally within the expected range for where Mir is at right now and doesn’t have us sweat.

The first chart shows the absolute comparison of Saucy on X and Saucy on XMir on the Phoronix Test Suite. There are some obvious issues (e.g. Nexuiz 800×600) which need to be investigated, but overall judging from the second chart, most benchmarks are around the statistical important median (9.68%). It is admittedly a 9.68% penalty, but overall not too bad, considering we haven’t done any major optimizations yet. Feel free to hit this spreadsheet to get the exact numbers (courtesy of Phoronix) & interactive charts.

One of the reasons for this result set is missing composite bypassing support, which we are aware of since January. Composite bypass helps when apps/benchmarks run fullscreen because… well, because they don’t need to be composited. Gamers out there… there is hope and a plan in place to get you your precious FPS back. This feature/bug is currently scheduled once other key functionality landed. Also, in order to make FPS based benchmarks really count, we need eglSwapInterval(0) implemented, which is currently in progress. Another important but hard to measure benchmark is how the desktop feels, responds and reacts to your interactions. This can’t be expressed in FPS and only user testing can really tell. We are eager to get your reports through launchpad.

I’d like to do a dedicated spike later this summer and squash as many performance bugs as needed for 13.10 – right now there are still a lot of other critical bugs and features that need our attention.

Performance is important for Mir to succeed – starting next week, we should have the test results from our performance lab available on the Ubuntu QA Dashboard. This will add another layer of transparency to our development process and should help people understand that Mir is more than just talks – Canonical is putting its money to where its mouth is. Sorry if others have a hard time following our pace, but we are on a mission and have no time to spare.

  • Alan Bell

    what are the odds of multi-monitor with unmatched resolutions working fine in 13.10?

  • morgan cox

    so it’s confirmed mir is slower…?

    Yes I know its not final yet… The main issue I can see is a Nvidia user is going to lose about 60 – 80 % GL speed using Mir to Xorg at present. (same for wayland).

    Until there is ‘proper’ Nvidia (and AMD) support Steam (Nvidia) users should stay away from Xorg replacements.

    • Andrei Zisu

      Some people would say “I can’t believe you’re choosing a binary blob over ditching X”

      • https://explodie.org/ Aranjedeath

        Some would be ignoring the priorities of most “users” and people who could be “users”.

        • Andrei Zisu

          I think ditching X as soon as possible is a priority.

          • Aditya Raj Bhatt

            Are you crazy? For a gamer, is supporting “free software” more important, or is it utilizing your hardware (bought with hard earned money), to the fullest? Or for that matter any user.

      • morgan cox

        Why spend good money on a card they run it at half its potential speed? That’s insane.

        Especially as other OS’s also would be twice as fast (as well as Linux+xorg)

        The thing is for Linux gaming really Nvidia is the only option at present.

        • Andrei Zisu

          Intel…

          • morgan cox

            Intel GPU’s cannot play AAA steam titles as well as Nvidia at present (no where near)

            Which is why i say for gamers only Nvidia will do – AMD are o.k in terms of speed but their support policy stinks,

            p.s – I completely believe in opensource – most of my various desktops are completely opensource with the exception of nvidia + flash + the odd steam game..

            I just can’t bring myself to use a driver that slows down my gaming by more than half.

          • Andrei Zisu

            I know, I wasn’t implying you don’t believe in open source. Meh… we can only wait and see.

          • Sabun

            Very true. From my own experience with a 4850, 6850 and 6870 the drivers are really not worthwhile. Went with a GTX680 and the driver difference is leaps and bounds [no screen tearing, FPS in games are stable, backwards support for REALLY old cards]. Nvidia really is the only proper choice for any gamer on Linux right now.

        • Aditya Raj Bhatt

          Do not be so hard on it. It is still in development, and the writer did mention that performance bugs would be quashed later on.

  • Bryan

    Are you going to run the Phoronix tests every day on Mir.. because that would be a great way to catch regressions, and see how good the improvements are..

  • http://www.mockel.se Falkowich

    I think that this is a nice gesture of showing what you guys are working with, and not afraid to put it to test.. jmho..

  • ienuz12

    I love the way Canonical is working (weekly reports of the job done for example). It looks really professional. All the best for your job and try to surprise us Canonical! ;)