How to identify what is leaking memory in a .NET application

A web api in production was leaking memory, lots of it. It was consuming over 27 gigs of memory where the usual consumption is around 200 Mbs.

I used this approach to pinpoint what was leaking since the memory leak was replicated again on a staging environment which I had access to. I needed to act quickly before the process chokes the server and becomes in need of a reboot.

  1. Create a process dump file. This can be done through task manager. The task manager you get from ctr+alt+del is a 64 bit version, this will generate a 64 bit dump. To generate a 32 bit dump use the 32 bit task manager (C:\Windows\SysWOW64\taskmgr.exe)TaskManager
  2. Download and install windbg.exe. The latest version of this can be found in the Windows 8.1 SDK. Make sure to use the 64 bit version when debugging 64 bit applications.
  3. Open the dump file in windbg.
  4. Go to File -> Symbol Path and add the debug symbols of your application. I have added the bin/debug directory as the symbol path.
  5. Open the command window (if not already open) from View -> Command.
  6. Type the following commands:
    1. .loadby sos clr (this loads the sos.dll to help in debugging managed applications)
    2. .symfix (automatically points to the Microsoft symbol store)
    3. !dumpheap (this may take some time to run depending on the dump size)

The last command will show you all the objects on the managed heaps, it will also show you the number of objects created for each type and the size of these instances. This is very helpful in narrowing the search radius for the leaky code. The dump looks something like this.

dump

The first column represents the memory address. The second column is the number of instances in memory. The third column is the size these instances are taking up.

What a developer should know

I’ve been doing quite a lot of interviewing lately for an ASP.NET developer position. I was surprised that a lot of the people I interviewed lacked the basic knowledge and understanding of trivial development and IT topics (most of them claimed to have multiple years working experience). So what are the things that I expect a developer (especially an ASP.NET developer) to know? And what characteristics should a developer have?

Object Orientation

You probably saw that one coming. It’s amazing (and a bit frightening) to know that some developers have managed to write and ship code without solid understanding of OOP. Most of the interviewees got lots of concepts confused. Difference between abstract classes and interfaces, static classes and the difference between overriding and overloading to name a few. Some know the concept but only as a textbook definition. Most fail to specify use cases for these concepts and some can’t state why object orientation was created in the first place.

Get to know your classes and objects, they are the building blocks you will use to create great software. This MSDN article explains the very basics, also check the “For more information” sections in the article.

Write “good” code

I expect from a developer with “over 5 years of experience” to transition from just writing code to writing good code. By good code I mean readable, extensible and maintainable code. The developer should know how to leverage OOP to loosely couple components and decrease the dependency between them. I also expect from an advanced developer to know how to organize source code to smaller reusable and manageable chunks. It saddens me to say that some “experienced” developers have never used or even heard of design patterns such as MVC.

The developer should also know how to make the best out of the development tools he/she uses. IDEs are there to make life easier, find out what they have to offer and how they can help you be more productive.

How the web works

No one can deny the importance of the internet in our lives, it’s everywhere. I expect from someone who writes applications for the web to actually know how the web works. Stuff like what happens when you request a page and what is the role of the browser should be no brainers. Every web developer should also know the basics of HTTP such as request and response formats and request types. I also expect from an ASP.NET developer to describe the page lifecycle, sessions, viewstate and caching, what IIS is and how it serves requests.

Oh and some ASP.NET developers seem to forget that ASP.NET is an abstraction layer so they don’t actually bother themselves with good old HTML.

Have some “under the hood” knowledge

This applies to every developer. If you develop .NET applications I expect some understanding about what .NET is, what it offers and what are its components. Things like CLR, MSIL and garbage collection should be known concepts to a .NET developer. Having this sort of under the hood knowledge will help you build better applications. You will understand the strengths and weaknesses of the technology you are using, things to use and things to avoid.

Get to know the terminology that is used for the technology stack you are using. This will help you when searching for resources on that particular area whether it’s for learning purposes or for when you’re in trouble and need someone’s help. I found that lots of developers can’t answer the question “what is an assembly in .NET” even though they use it all the time.

Also what I hate the most is a developer who doesn’t understand the basic underpinnings of computers and the operating system he/she is using. Stuff like setting a static IP address or formatting a drive should trivial (seriously I have seen developers who don’t know how to do that stuff).

Passion

I like being around passionate developers and I tend to take a passionate developer more seriously than someone who writes code just because it’s his job. You know a passionate developer from the way he talks about technology and code. A developer who is always up to date is also an indication of a passionate developer. I think a developer should be up to date when it comes to the technology stack he is working on. For example if you develop using Microsoft technologies you should know that Microsoft offers many ways to create services that can be consume over the web (WCF, WebApi … etc.). It doesn’t mean that you have to be a professional in all of them, but you should know what each one offers, what are its pros and cons, that helps you to choose the right tool for the task at hand.

“Not knowing” isn’t an excuse anymore, with internet access you can learn all you want. Nothing is further than a “search” away. Get to know professionals in the technology you are interested in, read their blog posts and follow them on social media. You will be amazed by the amount of stuff you can learn from other people. Happy coding!

VS2012 + .NET 4.5 broke something on my machine (Updated)

New development tools and frameworks get me excited, and since I’m the kind of person that likes my tools and frameworks to be up to date I downloaded VS2012. Among all the excitement I made a rather unwise decision to install it on my work machine.

I already had VS2010 installed and decided to install VS2012 side by side, the installation was smooth (although took quite some time). I fiddled around a bit with the new shiny tools and everything seemed to be excellent. When I wanted to do some real “work” work I started running into some issues.

One of the applications I’m working on is a colossal enterprise application. The application targets .NET 4.0. One of the projects is an ASP.NET Website. All the other projects in this solution build correctly (as expected) except the website project, the compiler spits out this error:

The best overloaded method match for ‘System.Web.UI.HtmlControls.HtmlTableRowCollection.Add(System.Web.UI.HtmlControls.HtmlTableRow)’ has some invalid arguments

That error got me suspicious since the solution used to build without issues before installing VS2012 and .NET 4.5. I decided to create a blank website project and try to reproduce the issue. I had a web page that contained this code:

<table runat="server" id="tbl1">
  <tbody>
    <tr>
      <td>
      </td>
    </tr>
  </tbody>
</table>

The above code gave me the compile error mentioned above. Removing the “runat” attribute OR the “tbody” tag made the project compile without problems. To make matters even more confusing the projects fails to build with the above error but publishes successfully! I searched the internet and no one seems to have run into a similar situation.

Also this issue doesn’t occur in ASP.NET WebApplication projects (exact same code!)

I tried re-installing VS2010 but nothing changed. I uninstalled VS2012 and .NET 4.5, that caused VS2010 to crash on startup and all I get is this log entry:

TFS Shell Extension event:\n Failed to start COMProvider while updating workspaces.

So I re-installed VS2012 again. That seemed to solve my issue and my projects now build without errors. What really bugged me is that I didn’t know the source of the problem (to be honest I was mostly concerned about restoring my work environment). I guess I’ll have to wait a bit until I can install VS2012 on my work machine.

Has anyone had issues running VS2012 and VS2010 side by side ?

Update 1:

Microsoft has acknowledged this issue and will soon release a hotfix. More info here 

Update 2:  1/9/2013

Today Microsoft release multiple fixes for .NET 4.5 and the issue was finally resolved. Update details.

My first use of the “dynamic” type in C#

Every once in a while when writing code I face a situation where I want to do something trivial, something that is usually done as a no brainer for a programmer. I start writing code and hit a wall immediately, it turns out it’s not as straightforward as it seems.

What I wanted to achieve

I am calling 4 different web services from 4 different methods. Each web service has a method that receives a request as an object and initiates a BizTalk orchestration. The request object is quite big, it has lots of properties that need to be filled. All 4 request objects have exactly the same properties inside (same names and types) but the objects differ in the class name and namespace. Have a look at the below class diagram.

Class Diagram
The properties are mainly primitive types

Each object is going to be filled in it’s respective method and since all classes have the same properties there will be redundant code in all 4 methods. I want to extract the code that fills the object into a separate method that will take the request object and all it’s values as parameters and assign the properties with their correct values. This method will be called from the 4 places where I want to fill the request object.

The problem

In order to access the properties of any of my request classes I must have an instance of that type. The issue is that the method I want to create doesn’t always receive the same type as a parameter. It could be any one of the 4 types shown in the diagram.

You might say why not change the parameter type to “object”. The  problem with this is that I have to cast this object to one of the above types and hold a reference to it in order to fill it’s properties.

The solution

The solution was changing the parameter type to “dynamic”. After all this parameter may have 4 possible types.  The caller of this method will then be responsible for casting back the dynamic object to the type it requires.

This is what the code looks like without using “dynamic”


Type1 type1 = new Type1();
type1.Prop1 = 1;
type1.Prop2 = 2;
type1.Prop3 = 3;
type1.Prop4 = 4;
type1.Prop5 = "a";
type1.Prop6 = "b";
type1.Prop7 = "c";
type1.Prop8 = "d";
type1.Prop9 = true;
type1.Prop10 = true;
type1.Prop11 = false;
type1.Prop12 = false;

Remember that I have more properties than this and that this code will be duplicated 4 times.

This is what the method that will fill the objects look like:

 public void FillData(ref dynamic request, int prop1, int prop2, int prop3, int prop4, string prop5,
string prop6, string prop7, string prop8, bool prop9, bool prop10, bool prop11, bool prop12)
{
request.Prop1 = prop1;
request.Prop2 = prop2;
request.Prop3 = prop3;
request.Prop4 = prop4;
request.Prop5 = prop5;
request.Prop6 = prop6;
request.Prop7 = prop7;
request.Prop8 = prop8;
request.Prop9 = prop9;
request.Prop10 = prop10;
request.Prop11 = prop11;
request.Prop12 = prop12;
}

The following code will call the above method, this code will be called in the 4 places where I need to fill my request object. Each time the appropriate type will be used.

Type1 type1 = new Type1();
dynamic dynamicType = (dynamic)type1;
FillData(ref dynamicType, 1, 2, 3, 4, "a", "b", "c", "d", true, true, true, true);
type1 = (Type1)dynamicType;

At the end

You’re probably saying to yourself that I am over thinking this and that it’s too much fuss just to avoid duplicating code. Perhaps you are right but small things like this annoy me. I am always trying to find the most optimized way.

I’d love to hear comments and  improvements or if I can achieve the above in a different way.

The Messaging Engine failed to register an adapter “SOAP”

I ran into this problem when trying to call a SOAP web service that starts a BizTalk orchestration, here is the error in detail:

The Messaging Engine failed to register an adapter “SOAP”. Details: “Registering multiple adapter types within the same process is not a supported configuration. For e.g. HTTP and SOAP receive adapters cannot co-exist in the same process”

The cause of the problem is that there are multiple virtual directories under the DefaultAppPool, all these virtual directories host WCF services except the virtual directory that hosts the SOAP web service I was trying to call. It turns out IIS can’t run multiple adapters under the same application pool (WCF adapter and SOAP adapter in my case).

Quick solution is to create an application pool for each adapter type.

Scribble app with HTML5

The “canvas” is one of the new elements in HTML5, it provides a drawing surface and APIs for yes you guessed it…drawing. The actual drawing is done through JavaScript. I’m going to show you a simple Scribble app that uses the canvas. It should be very straightforward and will help you get familiar with the canvas.

Before we begin I want to have a word about development tools for HTML5. Basically you can use any text editor to write HTML and JavaScript code. There are more sophisticated editors that provide syntax highlighting and basic debugging features such as Notepad++.

For this app I used Visual Studio 2010 SP1. The reason for my choice is that VS2010 SP1 now supports HTML5 and CSS3. That means you get syntax highlighting , Intellisense and some debugging capabalities (IE only). I know VS2010 is not free but you can get your hand on Microsoft’s Visual Web Developer 2010 Express which is free and also supports HTML5. Microsoft will be updating Visual Studio every 3 months to keep it up to date with the latest HTML5 standard changes.

Enough talking, lets dive into the code. First we will design the HTML page, on top we will have 2 drop down lists for selecting the colour and the line width along with a button to clear the canvas. Below these controls we will add the canvas. To achieve this I am using a simple table layout. Here is the HTML for the page:

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Scribble</title>
</head>
<body>
    <table>
        <tr>
            <td>
                Colour:
            </td>
            <td>
                <select id="ddlColour" onchange="ddlColour_onChange();">
                    <option value="#000000">Black</option>
                    <option value="#FF0000">Red</option>
                    <option value="#00FF00">Green</option>
                    <option value="#0000FF">Blue</option>
                </select>
            </td>
            <td>
                Line Width:
            </td>
            <td>
                <select id="ddlLineWidth" onchange="ddlLineWidth_onChange();">
                    <option value="1">1</option>
                    <option value="3">3</option>
                    <option value="5">5</option>
                    <option value="10">10</option>
                </select>
            </td>
            <td>
                <input id="btnClear" value="Clear" type="button" onclick="btnClear_onClick();" />
            </td>
        </tr>
    </table>
    <canvas id="mainCanvas" width="400" height="400" style="border: 1px solid #000;">
    </canvas>
</body>
</html>

There are few things I want to point out, the first line in the HTML file:

<!DOCTYPE html>

This line indicates that the HTML version is HTML5. You probably noticed the canvas element, there is nothing fancy about the markup for the canvas. The width and height are set and the canvas is given a border (so we actually know where we are drawing) and the canvas is given an ID so we can access it in JavaScript. I also added event handlers to the drop down lists and the button, in a bit we will see what these do.

If you open the HTML page in a browser (preferably Chrome) you will see a page that looks something like this:

At this point the app does nothing, it’s still not functional. We need to add the JavaScript Code that will contain the drawing logic.

We need some variables which we will use later.

var doDraw = false;
var mainCanvas, canvasContext;
var xCoords = new Array();
var yCoords = new Array();
var colour = new Array();
var isMouseDownEvent = new Array();
var lineWidth = new Array();

The first and most important function is the “window.onload” function

window.onload = function () {

    //Get canvas element
    mainCanvas = document.getElementById('mainCanvas');
    if (!mainCanvas) {
        alert('No canvas found !');
        return;
    }

    //Check for canvas support
    if (!mainCanvas.getContext) {
        alert('Sorry, your browser does not support the canvas element :(');
        return;
    }

    //Get canvas context
    canvasContext = mainCanvas.getContext('2d');

    //Attach the event handlers
    mainCanvas.addEventListener('mousemove', mainCanvas_mouseMove, false);
    mainCanvas.addEventListener('mousedown', mainCanvas_mouseDown, false);
    mainCanvas.addEventListener('mouseup', mainCanvas_mouseUp, false);
}

This function is responsible for 2 main things:

  1. Checking for canvas support in the browser. This is done by retrieving the canvas object and checking for the “getContext” method. If this method exists in the returned object then the browser supports the canvas element, otherwise the user should upgrade his browser.
  2. Attach the event handlers to the canvas element.

Notice that we are retrieving the “2D” context. The 2D context is responsible for everything 2D, it provides the APIs necessary for 2D drawing.

Now we want to change the line colour and width when the user changes the value from the drop down lists.

//Change the drawing colour
function ddlColour_onChange() {

    var ddlColour = document.getElementById('ddlColour');
    canvasContext.strokeStyle = ddlColour.options[ddlColour.selectedIndex].value;
}

//Change the line width
function ddlLineWidth_onChange() {

    var ddlLineWidth = document.getElementById('ddlLineWidth');
    canvasContext.lineWidth = ddlLineWidth.options[ddlLineWidth.selectedIndex].value;
}

There is nothing fancy about this code, we set the line colour by assigning a value to “canvasContext.strokeStyle” and we set the line width by assigning a value to “canvasContext.lineWidth”.

Now onto the drawing itself, we want to control what happens when the user presses down on a mouse button, what happens when the user moves the mouse and what happens when the user lifts up the mouse button.

When a mouse button is pressed down we want to record the click location, the current properties of the canvas.

//Start drawing when a mouse button is pressed
function mainCanvas_mouseDown(args) {
    xCoords.push(args.layerX);
    yCoords.push(args.layerY);
    colour.push(canvasContext.strokeStyle);
    lineWidth.push(canvasContext.lineWidth);
    isMouseDownEvent.push(true);
    doDraw = true;
}

The “doDraw” variable is set to true, this variable will be used to indicate that a mouse button is pressed. The reason for this is that we do not want to draw anything on the canvas if the mouse is moving without pressing down onto a button. Here is he variable in action in the “mainCanvas_mouseMove” function:

//The event handler that does the actual drawing
function mainCanvas_mouseMove(args) {

    if (doDraw) {

        xCoords.push(args.layerX);
        yCoords.push(args.layerY);
        colour.push(canvasContext.strokeStyle);
        lineWidth.push(canvasContext.lineWidth);
        isMouseDownEvent.push(false);
        redraw();
    }
}

If “doDraw” is set to “false” no drawing will take place when the mouse moves. Again we are saving the current mouse location and canvas properties. Then the “redraw” function is called (we’ll get to that in a moment).

When the mouse button is released we want to stop drawing on the canvas, “mainCanvas_mouseUp” does exactly that:

//Stop drawing when the mouse is released
function mainCanvas_mouseUp(args) {
    doDraw = false;
}

Onto the “redraw” function, this function is responsible for drawing the lines onto the canvas:

//Do the actual drawing
function redraw() {
    //Clear canvas
    mainCanvas.width = mainCanvas.width;

    for (i = 0; i < xCoords.length; i++) {

        canvasContext.beginPath();
        if (!i || isMouseDownEvent[i]) {
            canvasContext.moveTo(xCoords[i], yCoords[i]);
        }
        else {
            canvasContext.moveTo(xCoords[i - 1], yCoords[i - 1]);
        }

        canvasContext.lineTo(xCoords[i], yCoords[i]);

        canvasContext.closePath();

        //Stroke properties
        canvasContext.strokeStyle = colour[i];
        canvasContext.lineWidth = lineWidth[i];
        canvasContext.lineCap = 'round';
        canvasContext.lineJoin = 'round';
        canvasContext.stroke();
    }
}

First we want to clear the canvas, the easiest way to do this is to set the canvas width property to itself. “canvasContext.moveTo” moves the cursor to a given X,Y coordinate. “canvasContext.lineTo” draws a line to a given X,Y coordinate.

Surprisingly “canvasContext.lineTo” doesn’t render any drawing, it just draws invisible lines. To actually ink in the line we call the “canvasContext.stroke” function. Before calling it we tell the canvasContext how we want it to render our lines (strokeStyle, lineWidth…etc.)

When you string it all together you should have a working Scribble app 😀

That was a simple demo of the HTML5 canvas element. You can get the source code for the Scribble app here.

I’m sure there are more sophisticated effects that can be achieved using the canvas APIs but I’m still learning about HTML5 🙂 All comments are appreciated.

HTML5, whats it all about ?

HTML5 is a web standard in progress developed and maintained by W3C and WHAT (Web Hypertext Application Technologies working group). HTML5 is a collection of different things, different features. You need to grasp this idea to understand HTML5. Let me explain why.

There are different browsers out there that you can use, each browser having it’s pros and cons. The biggest difference between browsers is what happens under the hood, how they parse HTML and render it, how strictly have they implemented the standard. Since HTML5 is a standard (although not complete), each browser implements the standard differently, therefore you can find browsers that support most of the standard and some support few features of it.

So, what does HTML5 have to offer ? Here are the main features that make up HTML5:

  • Canvas: a canvas is exactly what the name implies, it’s a drawing surface. JavaScript is used to access the canvas APIs to do the actual drawing.
  • Canvas Text: the canvas text APIs are used to draw text on the canvas. Your browser might support the canvas API but not the canvas text APIs
  • Video: the <video> element is used  to embed video in your web page. This was previously impossible without a third party plug in.
  • Local Storage: local storage is used to store data on the client, this data can be used by your site. The concept is similar to cookies but the storage space is much larger.
  • Web Workers: this is one of the most exciting features, it allows your browser to run JavaScript in the background, and run multiple threads at the same time.
  • Offline Web Applications: HTML5 provides a way for your application to work offline. This is done by caching any resources your application need on the client.
  • Geolocation: the geolocation API allows you to retrieve the client’s location. You have to obtain the clients permission first.
These are some of the main features of HTML5. I recommend reading HTML5 Up and Running by Mark Pilgrim for more details on HTML5. The book is great and very much simplified.