Using nUnit AND Mstest (Visual Studio Unit Testing) together in perfect harmony

I like mstest. I like the integration. I like right clicking on a method and adding a unit test SNAP done.
It works great for me. nUnit is 'ok' to me. Integration is poor without third party tools. You have to change things around to get it to work. Sometimes I get strange errors I just dont get using mstest.
You have to set config files up to be named the same as the project. Not intuitive at first since nothing actually tells you this - you expect to be able to tell it your assembly and voila. Now.. you can use a different sequence of events to prevent this, such as add assembly before creating a project in nUnit, but thats not obvious to anyone first using it.

however.. to use nUnit on a build server I dont need to install Visual Studio. In order to use Microsoft's unit testing, you need to either install Visual Studio on the build server or go through a hack to get the components on there. Its not pretty. So nUnit shines here. But.. why not use mstest in the IDE and nUnit on your build server? They have different namespaces - but you can change the aliases around to do this successfully.

At the top of your classes in your unit test projects define the following to default to microsoft testing unless a build constant is defined called NUNIT

#if !NUNIT
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Category = Microsoft.VisualStudio.TestTools.UnitTesting.DescriptionAttribute;
#else
using NUnit.Framework;
using TestInitialize = NUnit.Framework.SetUpAttribute;
using TestContext = System.Object;
using TestProperty = NUnit.Framework.PropertyAttribute;
using TestClass = NUnit.Framework.TestFixtureAttribute;
using TestMethod = NUnit.Framework.TestAttribute;
using TestCleanup = NUnit.Framework.TearDownAttribute;
#endif

Go into Visual Studio, choose the Build menu and select "Configuration Manager"
Select "new" and name this build Nunit (or mstest or whatever you want to NOT be your debug/release build). Personally.. I use a build name called MsTest since my debug builds are what get automatically tested every night with cruisecontrol.net/nant/nUnit so I want my default builds to compile with nUnit, but when I want to add/debug unit tests in the IDE I simply change my build to MsTest.

Once you create a new build name, go into each of your projects once you select the build, and go to prokect properties. In the Build tab under "Conditional Compilation Symbols" type in MSTEST or NUNIT.
Use NUNIT if you call your build nUnit and use the code above. This will assume you want your default debug/release build to default to Microsoft. Case DOES matter for the conditional compilation constants.. make sure you use all uppercase.. its easier.

If you want to create a separate build configuration for testing Microsoft unit tets, but want to keep the default debug/release for nUnit then your headers would look like this:

#if MSTEST
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Category = Microsoft.VisualStudio.TestTools.UnitTesting.DescriptionAttribute;
#else
using NUnit.Framework;
using TestInitialize = NUnit.Framework.SetUpAttribute;
using TestContext = System.Object;
using TestProperty = NUnit.Framework.PropertyAttribute;
using TestClass = NUnit.Framework.TestFixtureAttribute;
using TestMethod = NUnit.Framework.TestAttribute;
using TestCleanup = NUnit.Framework.TearDownAttribute;
#endif

hope this helps : )

An easier way to track parameters in unit testing for Visual Studio (mstext.exe)

One downside in unit testing is having an easy way view and use the test parameters. Sure you can put them in the code, but its hard to see how it varies from method to method. Ok.. maybe 'hard' isn't it, but 'not as clean as having all test parameters in one spot' I'll say. So I wrote a quick routine to extract the parameters. note:

I created a method called SetPropsToAttributes() and also GetAttributeValue().

For SetPropsToAttributes, Pass in an object and based on the TestProperty() attributes, it will find the matching property (not case sensitive) and set its value. So you can pass in an object and any attributes that are named the same as the object's properties, will have the values set to whatever are in the attributes. So all test information is kept then at the top of the method.

For GetAttributeValue, the idea here is an easier way to get a single value:
If I have this method:

[TestMethod]
[TestProperty("UserName","Adam Tuliper")]
[TestProperty("Password","nobodyknows")]
public void TestLogin()
{
   LoginHandler handler = new LoginHandler();
   handler.Login((string)("UserName"), (string)GetAttributeValue("Password"));
}

You can see here I easily get the parameters. I don't have to look through code to get test information. Every method has it at the top. This also makes it easy to copy and paste information into a test script for documentation purposes.



On the helper methods below I use [MethodImpl(MethodImplOptions.NoInlining)] because this is meant for debug code only in your _test program_ (not in the actual bytes being unit tested) which you would have during a unit test. The stack frame tracking and debug information is different (debug information is missing unless you do a release build with the pdbonly advanced option selected - this should work in that scenario but I haven't tried it) depending on options selected) in a release build. When you compile a release build, there are optimizations performed and the compiler may decide to inline the method calls, thus breaking the way this routine finds the method that called it. Read again: this will break in release builds - it is for debug code only, but then again thats the whole purpose of this posting : )

One more note - Do not call SetPropsToAttributes and GetAttribute from more than one level up.. Ie dont do this: Method1() - > Method2() -> GetAttribute(for memthod1)
as the code only looks up one stack frame on top of GetAttribute to determine the calling method.


So we
1. Create method based
2. Declare TestProperty attributes with our test data. null, int, strings, etc
3. Pass in the object we are going to unit test and values get assigned in the method SetPropsToAttribValues
4. Run test
5. Assert return codes

/// <summary>
/// A unit test method providing a sample way to automatically assign
/// test attributes to Properties in an object
///</summary>
[TestMethod()]
[TestProperty("TestProperty1", "Something")]
[TestProperty("TestProperty2", "SomethingElse")]
public void SomeUnitTestMethod
{
    SomeCustomObject testObject = new SomeCustomObject();
    //This call will automatically set testObject.TestProperty1 to "Something" for ex.
    SetPropsToAttribValues (testObject);
    int retVal = testObject.SomeMethod();

    Assert.IsTrue(retVal==0, "Return value was unexpected:" + retVal.ToString());

}


/// <summary>
/// Uses reflection to look at the calling method's TestProperty attributes
/// and will set values on properties matching those attribute names with the attribute values.
/// Property names are not case sensitive for this implementation to help avoid unit test typos.
/// </summary>
/// <param name="obj"></param>
[MethodImpl(MethodImplOptions.NoInlining)]
private void SetPropsToAttribValues(object obj)
{
    //Get name of attribute. if it exists and value != null set it. 
    Type t = GetType();
    Type attributeType = typeof(TestPropertyAttribute);
    //hop up the stack frame to the caller so we can get its attributes
    object[] attributes = new System.Diagnostics.StackFrame(1, false).GetMethod().GetCustomAttributes(attributeType, false);

    for (int i = 0; i <= attributes.GetUpperBound(0); i++)
    {
        string propertyName = ((TestPropertyAttribute)attributes[i]).Name.ToLower();

        PropertyInfo[] properties = obj.GetType().GetProperties();
        foreach (PropertyInfo pi in properties)
        {
//Made to be not case sensitive to help avoid test script typos.
            if (pi.Name.ToLower() == propertyName)
            {
                pi.SetValue(obj, ((TestPropertyAttribute)attributes[i]).Value, null);
            }
        }
    }
}

/// <summary>
/// Looks up attribute values for a particular method. They are used here for sending in values for the test cases.
/// This reads all [TestProperty] attribute values
/// </summary>
/// <param name="attributeName"></param>
/// <returns>attribute value as specified in [TestProperty] attribute, throws an exception if not found.</returns>
[MethodImpl(MethodImplOptions.NoInlining)]
private object GetAttributeValue(string attributeName)
{

    Type t = GetType();
    Type attributeType = typeof(TestPropertyAttribute);
    //hop up the stack frame to the caller so we can get its attributes
    object[] attributes = new System.Diagnostics.StackFrame(1, false).GetMethod().GetCustomAttributes(attributeType, false);

    for (int i = 0; i <= attributes.GetUpperBound(0); i++)
    {
        if (((TestPropertyAttribute)attributes[i]).Name.ToLower() == attributeName.ToLower())
        {
            return ((TestPropertyAttribute)attributes[i]).Value;
        }
    }
    throw new Exception("Could not find attribute " + attributeName);
}

The Nagle algorithm - speeding up tcp, changing protocols, rewriting your app, or just dealing with it?

First - what is the nagle algorithm and its purpose?

The Nagle algorithm's basic idea is to queue up smaller amounts data and do not send it if there is current sent data waiting to be confirmed. If we are sending a large packet, then there is no effect. The idea is to prevent small packets and the header information on each packet from clogging the network if we can queue some of that data (since we're waiting for a send acknowledgment anyways) and send it once the pending send has been confirmed.

More details can be found at:

http://en.wikipedia.org/wiki/Nagle%27s_algorithm

The magic value here is MSS - Maximum Segment Size and this specifies the maximum size that can be transmitted. Upon establishing

There's a simple way to test the effects of Nagle and how it compares between using sockets and remoting. Remoting can wrap your data in headers, thus
RFC896
http://www.ietf.org/rfc/rfc0896.txt?number=896


I find myself at times saying "Nagle is bad in X scenario, RDP for instance where a user expects near real time response", so at times it does make sense to turn it off.

There is a great article on this subject at:

http://msmvps.com/blogs/alunj/archive/2006/05/08/94038.aspx


In it, Alun Jones makes a great point. Basically if you have gains because Nagle is turned off either your application is broke or the protocol is broke. Take RDP for example. A user expects a real time response. Since we are currently in a TCP/IP connected world, TCP/IP is basically a must for a remote connection. Therefore, the protocol is broke (for our required use.. although I hesitate to use the word broke, rather than unfitting)


A simple test to see the effects can be duplicated by the following scenario.


1. Use .Net remoting and make a simple method call that takes a string, but does nothing with it. (remoting in Singleton mode so you have one instance of the class responding to requests)
2. Simple socket call to send a small string (less than 20
bytes) to a method.

The remoting call is significantly faster (107 compared to
the sockets 58 requests per second in one test)

Sockets are simple on the receive end (and the send):

Socket ss = new
Socket(AddressFamily.InterNetwork,SocketType.Stream,
ProtocolType.Tcp);
ss.Bind(EndPoint);
ss.Listen(100); //100 in the backlog connection queue
while(true)
{
Socket sock = ss.Accept();
byte[] read = new byte[1024];
int bytes = sock.Receive(read, 1024, 0);
//Ideally you would spawn off another thread here to handle .. results are similiar, this is done here for simplicity
string messagefromclient = System.Text.Encoding.ASCII.GetString(read,0,bytes);
sock.Close();
}

Lets say in this scenario this is a mission critical authentication application that requires high throughput. The login packet may be extremely small. Having Nagle enabled can slow this down incredibly. Certain applications may say this is acceptable - for instance game development. A small movement on the screen usually equates to a small packet of data to be sent. Waiting for the maximum segment size to be reached before transmitting means data can just be queued up if a current acknowledgment is pending. In the fast paced realm of gaming where near instantaneous response is required, Nagle can slow the game down significantly. In this case it seems Alun Jones would say the protocol is broken, as disabling Nagle yields a good benefit. One could argue here then why even use TCP? Why not just use UDP? UDP after all doesn't concern us with packet size issues. It just sends. Theres no concept of connection, it just goes there. But.. will it arrive? Possibly. Do you know that you receive packets in order? NO!

You have to write something into your system to handle these issues, that's the tradeoffs with UDP, and mostly I'd say this pertains to games.

There is an excellent article on this for those that are interested in reading more (including more info on sequencing packets)
http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/


With all this said, if you have to have data integrity, order, and connections, then stay with tcp. If you cannot accept the performance hit, then either switch protocols or accept the performance hit you will take to guarantee your data's order (without writing your own management routines with udp as outlines in the above url)