Validation Considerations

The post stems from a conversation I had yesterday with a guy who is learning ASP.NET.  The question was when to validate client-side vs. when to validate server-side.  More over, when to validate in code vs. inside the database.

One of the cool things about .NET and SQL is that it lets you do things pretty much any way you want to suit your needs.  At the same time, it can leave people who are new to the technology feeling lost.

While there is no steadfast rule on how to do validation, I will run through a few scenarios where I might choose to do validation one way over another.  A good indication that you need to re-work your validation is if you're sending a lot of data to the client or making a lot of roundtrip calls to the server (be it a web server or database server, etc).  It seems like an innocent thing but there is a finite amount of things a computer can do in a given period of time and only so much space in which to transmit the inputs and outputs of that work.

Client Side Validation

Client side validation is most likely going to get to the client in the form of scripting, probably JavaScript.  That means a larger payload coming down the wire with the page.

In my opinion, the validation script should be small, relevant, and require no additional data from the server.  You might also argue that the script's execution shouldn't be critical to the success of the operation. What I mean by that is if the client has scripting disabled, it shouldn't prevent the user from being able to use the application.  I'll talk about what to do in this case momentarily.

Examples for good candidates for client side validation scripts:  verification of a birthdate (is it before today's date, are you older than 18 or 21), validation of SSN format (9 digits, all numeric), or validation of email address format (several characters and @ sign and a dot with a valid extension).

Notice all of these are validations of raw user input which is also a good indication that you might want to use client side scripting.

What to do if scripting is disabled?  In my opinion, client validation should assist but not replace validation within your business objects.  If scripting is disabled and invalid data is sent across, the business objects themselves shouldn't allow the data to be saved.

Two ways you could handle this...
1. Thrown an exception within the object, and provide a text area on the screen where you display error messages.  (Keep in mind you may only want to display these validation exceptions and make sure that any other exceptions display a generic "This operation cannot be completed at this time." message for other exceptions).
2. Create an IsValid method in your objects that return a bool value and take an out parameter for a list or string of errors.

Technically, you could also do some checking inside a stored proc before writing the data to the database.  However, I believe that the database should only do validation in limited scenarios which I've included below.

Server Side Validation

Keep in mind you can mix and match validation.  And while you can do all your validation server side it is an expensive operation to keep sending entire pages to be parsed, data to be evaluated, and new data to be sent across the wire.  From the time, load, and bandwith prespective, server side validation has really become less common in favor of client side validation.

I put AJAX in the server validation group because while you do have some scripts that come down like a client-side validation, you are really putting the burden on the server to do the work and then updating the client accordingly.  This is a valid way of doing validation but again for simple validation the client side validation may be the way to go.

I would definately use AJAX to do validation in cases where you don't have the data you need client side or the validation is intensive and/or time consuming and shouldn't bog down the user machine.

You might also do straight up server side validation (posting a page back to the server for processing) in cases where say you've completed a form and you can't really do anything else until the page data has been processed and some action(s) taken.

Examples for this scenario: A registration form or login page is a good example of this.  But as I said, you could use AJAX to silently do the submission and update the UI accordingly.  But more than likely, unless something fails, your going to want to redirect or render a new page which is a server side thing anyways.

You would also do server validation in cases where you have a lot of different pieces of data (objects) that don't make sense to push to the client. 

Example for this scenario: Say your application allows you to apply for a loan.  You're client script has validated the data being entered but now its time for you to do some credit checks.  It doesn't make sense pull the credit data, transfer it to the client, and have a script that processes the credit report.  You'd want to do that on the server, maybe save it to the database, and then return the results of that credit check back to the client either in the form of a new page or maybe AJAX is polling silently so it can display an APPROVED or REJECTED message.

SQL Validation

The last part of the validation puzzle is SQL itself.  Technically, you could do zero validation on the client or in your objects and throw errors from your SQL procs if data was invalid.  Its data, its going into a database, and the database should be responsible for making sure the data it contains is valid, right?

Yes, but to a point.  With one notable exception, the database should really only be validating data type, relationship, and constraint type information.  Basically, anything that would break the database schema.  Business rule/logic data should probably be outside the database; again, with one notable exception.

Take the case of the social security number.  The database should only care that the SSN meets the requirements of its field in a table (say a non-nullable integer), but it shouldn't parse the SSN to determine if the numbers are valid, if its the right number of digits, etc.  That is business logic!

Exception:  The notable exception on when to use the database for validation is in situations where its not feasible, practical, useful to do the validation outside the database.

Example:  Say you have a registration form and the username is an email.  You want to make sure the email doesn't already exist (IE, an account doesn't already exist).  You're not going to send all your account emails to the client to validate; that's not practical or secure and might take a long time.  So you you make an AJAX call to the server.  More than likely, you're not using server memory to store all the emails.  Plus, its not practical to load all the emails into an object and iterate through them in code just to see if one exists.  In this case, the database is really the prime candidate to do this work.  The database is born to these types of queries.  So you call a proc that would search the user table for that email and return a value to indicate if the email already exists or not.

Another flavor of this is cases where you are doing data manipulation that is more of a data operation than an execution of business rules.  There might be some business logic but primarily the goal of the operation is to generate some type of data.

Parsing Delimited Data

So I had to parse some delimited data today in Oracle and it turned out to be a little more complicated that it probably needed to be.  This is a great example of something being simple in one technology and difficult in another.

In .NET, we can simply delimited data string, and call the Split method and provide the delimiter and we're provided with an array of the tokenized elements.

Since what I was parsing was IP address data, I will carry that theme into my example.

In .NET...

string a;
a = "192.168.1.1";
string[] aParts = a.Split('.'); 
 
What I end up with is an array of 4 elements 192, 168,1,1.  I have parsed my IP address.  I can then loop through each string in the array or reference each element using its zero-based index.
 
In Oracle, this proved to be a bit more complicated.  A split type function doesn't exist in Oracle.  I could have written one but that was overkill since I'm writing this db procedure to be ran one time for a data migration of a few thousand records.
 
So let's take a look at what I did.  Keep in mind, I took of the FROM and WHERE clause from these queries because they don't really pertain to what I'm talking about.

We will assume the IPADDR value is 192.168.1.2.  Keep in mind that I'm doing the REPLACE on the entire SUBSTRING operation to remove the period (.) from the IP.
 
You'll notice I'm using substr and instr heavily in this example. 
 
INSTR returns the position of an character inside a string.  INSTR  has several overloads.  This overload I'm using takes 1) the string to examine, 2) the character(s) to find, 3) what position to start in the string, and 4) which instance of the character(s) to return the position of.
 
SUBSTRING returns a subset of characters inside a string.  The overload I'm using for it takes 1) the source string, 2) the position to start from, and 3) the number of characters to grab.
 
1. SELECT replace(substr(IPADDR, 1, instr(IPADDR, '.', 1,1)), '.', '') INTO mIP1

So for the "192" part of the IP ... I'm looking in my string, in the 1st position, and stopping at the location of the 1st period.    The instr returns 4 so I actually get back "192." and I just remove the period. 

You'll notice in Oracle, these strings are a 1-based index unlike C# which uses the 0-based.
2. SELECT replace(substr(IPADDR, instr(IPADDR, '.', 1,1), instr(IPADDR, '.', 1,2)- instr(IPADDR, '.', 1,1)), '.', '') INTO mIP2

So now I need the "168" part.  So I want to start after the 1st period and end at the 2nd period.  You'll notice that I substract the position of the 2nd period instr(IPADDR, '.', 1,2) from the location of the 1st period instr(IPADDR, '.', 1,1) to get the length between the two periods.

And again, I remove any periods with the outer replace.
3. SELECT replace(substr(IPADDR, instr(IPADDR, '.', 1,2), instr(IPADDR, '.', 1,3)- instr(IPADDR, '.', 1,2)), '.', '') INTO mIP3

Here I do basically the same thing to the "1" except I look for the location of the 2nd period and subtract the location of the 3rd period from the 2nd period to get my length.

4. SELECT replace(substr(IPADDR, instr(IPADDR, '.', 1,3), 4), '.', '') INTO mIP4

At the end, I start from the position of the 3rd period and go a hard coded 4 characters.  This is misleading because there may not be 4 characters but rather this just goes to the end of the string.  I didn't realize I'd done that until I look at it again for this blog.  But its not a big deal in this case.

So there you go.  Parsing an IP Address in C# vs. parsing it in Oracle.  I should point out that this same technique should work in MS SQL as-is or with minor modifications.  As a bonus lesson, you can see that if I had a complex data parsing situation, maybe using C# would be quicker and easier than trying to do the parsing in the database.

As a final note, I could have done a Regular Expression as well but that's probably more illegible than these queries were!  Plus I'm did this for a client and after I'm gone this should be easier to understand/maintain.

Consuming Web Services in .NET

http://msdn.microsoft.com/en-us/library/7h3ystb6(VS.80).aspxYou have several options for consuming a web service in .NET.   The easiest way is to add a web reference to your project.  However, there are some drawbacks to this method.

Why I don't like using "add web reference"

1. Your local machine has to have access to the web service in question.  Normally this is not a problem, but I've had issues before where due to proxy servers or permissions (IE, you don't have access to the production instance of the web service) prevent .NET from accessing the URL.  You can work around this by consuming say a DEV version of the web service and then manually editing the files to point to the instance you really want to access.

2. Say you want to switch between the instance you want to access?  You have to change your service URL which requires the objects created to be re-generated.

Solution

My preference is to use the WSDL tool.   This tool should be pre-installed as part of your .NET install.  I normally copy the WSDLexe and its config to a directory to make it easier.

To use the tool, you really only have to specify the URL of the web service but I go a step further and specify the language I want the proxy code to be generated with.

From the command prompt all you have to do is "WSDL /l:CS http://www.mydomain.com/myservice.asmx" and hit enter and you're done.  (Without the quotes, of course).  This will generate a class file that you can add to your project.   The /l indicates which language to generate the web service wrapper with.  You have the option of several of the popular languages C# (CS), VB (VB), JavaScript (JS), VJS (J#).  See additional info for more about the optional parameters of this tool.

To use the service, you simply declare an instance of the class and then call the web methods like you would a normal class.  The tool is also smart about creating object definitons if the service defines them instead of returning a simple data type result.

Additional Info

I normally only have to make 1 tweak to the generated class which is to go into the constructor and add a string input parameter which I then set the "this.url" property inside the class to.  By default, you will see the url of the web service is hard coded.  Now you can store your URL in a web.config  or app.config file so that it can be changed easily to hit the different instances (dev, test, production) of your web service!

You actually can do some of this and more through the optional parameters on the WSDL tool but I personally hate writing exteremely long commands through the command-line.  However, you might need to specify the login/password and/or the proxy login/password if applicable to your situation.

I've included the link to the MSDN page that covers these options in detail.

MSDN: WSDL Tool

Tools of the Trade

In addition to the Visual Studio IDE, there are several useful tools that I recommend to be a productive coder.

Red Gate Software's Reflector:  Reflector allows you to disassemble compiled .NET code (EXEs, DLLs, etc) and see the code behind them.  Keep in mind, some items may render using the MSEL syntax and not the normal language syntax you might be used to.  But for the most part, readable code the way the original developer(s) wrote it.

Uses:  My biggest use for this tool is to inspect production DLLs if there is an issue.  I've worked at IT shops in the past where you have one person doing builds and its hard to determine if your changes were pulled from the source code repository and deployed.  By using reflector, I can navigate to the method in my code inside a production DLL and verify if my changes were included.

Reflector can be found for free at http://www.red-gate.com/

Firebug:  For web development, Firebug for Firefox is a great tool to inspect the currently loaded HTML object.

Uses:  Checking the rendered HTML for anything goofy.  (Sometimes .NET outputs non-standard HTML that may not display correctly outside of IE.)  Additionally useful for checking the path of scripts and other resources.

NUnit: If you don't have any type of TDD software, I recommend NUnit.  Type NUnit in Bing or Google you'll find it.

NDoc: Greate tool for document your application.  NDoc can ready the meta data and uses it to give a good starting point for your documentation.

Really, these are the add-ons I use most often.

If you have additional tools you recommend, please add a comment so I (and other techies) can try them out!

DevConnections in Las Vegas - my take

I went to Las Vegas last week for the DevConnections event.  The event marked the release of Visual Studio 2010 and Silverlight 4.

A little short notice but I believe you still have until the end of the month to purchase Visual Studio 2010 and get 1 SKU higher for the same price.  That means if you bought Standard you'll get Professional.  If you get Professional then you'll get Ultimate.  That is a great deal if you're looking to go to VS2010.

My take

1. Data Binding and Templates

There were several courses that were almost exclusively about data binding and templates.  Obviously, data binding is binding of an object to a control.  The most common binding in a business application is 2-way binding where the UI and the object are updated as changes are made; traditionally through the use of the INotifyPropertyChanged interface. (Side note:  There is an alternative to INotifyProperty changed that I may address in another post).

With WPF and Silverlight 4 you get the same data binding functionality/experience that you're used to seeing if you've done any WinForms development.   There is also the IValueConverter interface that is very useful in data binding.  This interface provides a Convert and ConvertBack method where you can transform a data item as needed.

How is this useful?  Well, say you have a boolean Active that you want to bind to.  And instead of a check box, you want to bind to a radio button group which will act as a 'Yes'/'No' (or Active/Inactive).  So as you imagine, in the Convert method you would return a 'Yes' or 'No' string while in the ConvertBack you will set your object's True or False value.  A a bonus example, what if you wanted Active to turn the screen one color and Inactive to turn the screen another?  Same thing.  You bind your Active flag to say the background color property of a control and then use the Convert to set the correct brush color, etc.

More on IValueConverter

Lastly on templates is the ability you have in WPF and Silverlight to customize the look of your controls.  You can dramatically change the way a dropdown list or listbox looks by changing its item template.  A great example of this is an application called StaffLynx.

Billy Hollis, a speaker at DevConnections, worked on this application and shows it in many of his talks.  As one example, I've attached a screen shot of the application.  You'll notice on the left hand side are some sticky notes.  Those sticky notes are merely a list control that's item template has been tweaked to give the appearance of a sticky note. The buttons are also button controls that have been edited (probably using Blend) to rounded edges and some focus/lost focus animations.

It blew my mind a little and hopefully some of yours as well the power of templates.  For those of us doing web-based development we can stop making plain old forms and create rich robust responsive applications!  This is powerful stuff, and really not that hard to do.

FYI, you can find Billy Hollis at slmasters.net.

2. Silverlight 4

It is much more than Microsoft Flash or HTML5!

Silverlight had been off my radar for a while but this recent release is changing that.  This is a mature product now and is very feature rich.  If you used previous releases of Silverlight and hated having to edit XAML (pronounced Zam-el)  by hand... good news!  SL4 have a friendly IDE to help you out.

Also keep in mind that many people are using Expression Blend to do the visual tweaking of the UI controls.  You can do it in Visual Studio directly but Blend offers a design tool to compliment the VS IDE.

Keep in mind- Silverlight 4 doesn't have any ADO.NET capabilities.  So if you're doing things with data then you'll need to use WCF, RIA Services, etc... in order to handle data access.

3.  EntityFramework is still here. 

Personally, I thought the ORM concept would die out, but there is still support for it in VS2010 and if anything even MORE support for it.  If that is your cup of tea.   One of the presenters did do a simple performance test and a DataReader is about twice as fast as doing EntityFramework.  A DataSet, as you would expect, is also slower than a DataReader but faster than EntityFramework.  This is a general thing, in some select cases this may not hold true.  I will say the difference was 0.2ms vs 0.4ms in the example (about 1000 records) but for larger sets of data or high performance apps that might be a big deal.  Also keep in mind the first call is always slower time wise vs subsequent calls.  So, if you're doing your own benchmarking, be sure to do multiple runs and average the time and/or drop the first time.

4.  Azure

This is Microsoft's entry into cloud computing.  Most if not all the VS2010 products will come with some amount of free cloud computing time on Azure.

I really didn't see much on Azure so my information is limited.  It did look like you had some XML attributes and some other hoops to jump through on the code side to get your stuff to work in the Azure environment.  BUT... everything else in terms of using the IDE to develop your pages and using SQL Server over the cloud looked to be similar to doing "local" development.  During one of the key note talks I did hear you can run your Azure and Windows mobile applications locally for testing which is handy. Oh, and you can run SharePoint locally if that's your thing.  (Sorry, I'm not really into SharePoint but my exposure has been poorly implemented SharePoint solutions at several companies.)

5.  Oracle deprecation

The data access space for Oracle has been depcreated.  That means you can still use it but that method of data access will go away in a future release.

Microsoft recommends looking at a 3rd party data access object provider.
6. What's new in C# 4.0?

Not much different in 3.5 vs 4 in terms of the languages, but a lot in terms of the framework.  The "big" news is C# now supports dynamic (if you don't know what this means, read up on F# and dynamic languages).

If requested, I may go into all the difference and new features in a later post.  In terms of VB, the big news was you don't have to do the &_ to continue a string to another line.

7.  Better integration with 3rd party

Design time interpretation of scripting (JavaScript, jQuery, etc).  This allows for design time intellisense.  That's a very cool thing.

8.  Misc

Not a lot of talk about WinForms.  Probably safe to say WPF is going to replace WinForms entirely.  However, it seems that Silverlight and ASP.NET will coexist.

VS2010 is offering a lot of templates for popular design patterns.  MVC and MVVM being two of the main examples.

On a minor note, Passport is no longer supported.  It seemed like an intereting idea but I guess it didn't get the traction Microsoft was hoping for.

Las Vegas

Heading out to Las Vegas for Dev Connections. Can't wait. I'm hoping to learn some great inforamtion about .NET and make some industry contacts.