xUnit.net vs NUnit, a quick pragmatic comparison

At the beginning of our last project, which was a greenfield project on .NET Core, I was responsible to choose a testing framework, isolation framework and all tools and frameworks, related to unit and integration testing. So I started searching for a decent testing framework for .NET Core and came up with 2 major opensource candidates: NUnit and xUnit.net. In this article, I’m going to briefly go through the features of each framework and share my final verdict!

I’ve been using NUnit for many years with no problem whatsoever. So I tend to have a bias towards NUnit. My task was choosing the best testing framework though. So I put my experience and comfort a side and tried not to have any familiarity bias, as learning curve of a new testing framework wouldn’t be that steep.

NUnit
nunit-logo

NUnit is actually ported from a Java testing framework named “JUnit”. It’s a mature framework with a long history started from JUnit since 1998. Widely used in dotnet community, highly documented and at the time of writing this article 19225 questions are tagged as NUnit in stackoverflow.

xUnit.net
xunit

xUnit.net is a relatively new testing framework, written by original inventor of NUnit v2. At the moment, xUnit.net is the latest unit testing tool, and well-accepted by .net foundation. Designed to be extensible, flexible, fast and clean. So far, there are 4008 questions tagged as xunit in stackoverflow.

NUnit xUnit.net
Supported Platforms  UWP, Desktop, Windows Phone, Xamarin Android, Xamarin iOS, ASP.net  UWP, Desktop, Windows Phone, Xamarin Android, Xamarin iOS, ASP.net
.Net Core Support  YES YES
IDE Tools Support  Visual Studio, Visual Studio Code, Resharper  Visual Studio, Visual Studio Code, Resharper
CI Tools Support  TeamCity, VSTS, MSBuild, CuiseControl.net, Bamboo, Jenkins  TeamCity, VSTS, MSBuild, CuiseControl.net, Bamboo, Jenkins
Parallel Execution  YES  YES
Execution Isolation Level Per test class Per test method
Extensible Test Attributes  NO (Test and TestCase attributes are sealed)  YES (Theory and Fact attributes are extensible)

As you see, both NUnit and xUnit are strong, mature and widely adopted by .net community. So it’s not easy to say either of them is not good enough.

To me, the way that xUnit.net runs the tests is admirable. By default, it creates a new instance of the test class for each test method. This means that test methods are completely isolated and do not interfere each other. So it actually mitigates the risk of dependent test methods which is a bad practice. However, if in some rare cases you need to share the context among several test methods, there is a way to do that.

Another pro of xUnit.net over NUnit is more extensibility and flexibility. We can inherit from Theory, Fact and other attributes. Personally I don’t care about this feature though. As it never happened to be useful in any of the gigs I’ve done. But anyway, xUnit.net designers have had extensibilty in mind.

NUnit on the other hand, has a longer history. There are many more projects already done via NUnit, you can find many more documents, samples, and discussions about NUnit in developers’ communities. i.e, you can find 5 times as much as xUnit.net questions for NUnit in stackoverflow website.

The bottom line is: I’d rather xUnit.net over NUnit, mainly because of the way that it runs the tests, whereby test methods run in separate test class instances in isolation, it can also increases the level of parallelism. I like It’s nice style of coding as well.(using constructor/Dispose instead of ugly TestFixtureSetup and TestFixtureTearDown attributes)

N.B: I have also made a Powerpoint presentation to talk about the chosen tools and frameworks plus a quick introduction to TDD. You can see the slides in here.

Advertisements