Iterate Iterate Iterate!

Adam Fridman writes about the "massive" downside of agile software development for Inc. magazine. He is right to acknowledge the risks involved in any custom software project. Without knowing how software projects of any size can fail, you can't modify scrum well because you don't see the difference between a catastrophic derailing vs an intelligent tweak. The way Adam assumes these risk are fatal flaws to agile seems to be throwing the baby out with the bathwater.
The articles biggest flaw is the way it propagates a Big Software Myth: clients know what they want.

Feedback is an essential part of every software project, and claiming that waterfall provides superior predictability doesn't hold water (see what I did there?) in reality. Waterfall's additional plans and requirements tomes provide the illusion of predictability but in reality are themselves the cause of much waste. The wise executive knows this and sets roadmaps of long and short-term goals, and adapts along the way as milestones are hit. Build something and inspect it. Adapt your plans accordingly. inspect adapt iteratively and you provide the highest value for the lowest cost.

SOLID principles and self reflection on over designing software

Rob Ashton, someone on the internet who codes, has opinions on the SOLID principles that resonate with the kind of decision making we are taught in school but isn't practical for the real world. I'm not debating whether or not there are measurable benefits to these changes, but the complexity added to the code is the unmeasured cost when you design in the ivory tower.

I've definitely made this mistake in over architecting, and so has Rob. He quotes a fictional self that resonates a little too close home:

What if somebody at some point wants to change this so they can have another behaviour for this value, I'd better use the strategy pattern here instead of this switch statement, but oh my word now I've done that what if somebody wants to use this code from some other system than this one, I'd better stick a command system in front of this and use double dispatch for handling them - but wait, what if other code needs to react from this and do something else, I'd better raise a pile of events, but what if those aren't enough I'd better make sure I split all this behaviours out into their own interfaces so they can be overridden and...
http://codeofrob.com/entries/my-relationship-with-solid---the-big-o.html

Everything in there I've done. Not quite in that order (I have a habit of implementing interfaces for everything)  but event handlers, command pattern, it so often ends up useless in typical LOB business software that I've been writing. You figure if it would matter anywhere it would be here, but not the case. Certainly overkill if we worked on one-off customer work. YAGNI and KISS are very applicable in one off projects or services engagements. Even in more recent engagements with software companies, "wiring" and "piping" can be extended after-the-fact to support your nonfunctional requirements (assuming you have decent test suites).

TL;DR: Don't be an architecture astronaut.

Relating to Ninja/Wizard/10x

Best article I’ve read in the Ninja/Wizard/10x discussion in a long while:

I’m more akin to a librarian, scientist, artist, and carpenter.

Unable to Reference classes in App_Code or other folder in Web Application Project

So if you google "unit testing on web application projects" you'll get discussion on using the "web application" project instead of "web site" project type. I won't retread that here, its been done elsewhere:
http://stackoverflow.com/questions/1198555/unit-testing-asp-net-web-site-project-code-stored-in-app-code?rq=1

The other common solution to the problem is changing the project's targeted .NET version.

But if that still doesn't work, it could be the content type on your files. This often happens when you have folders in your project (like App_Code) and add new items to those folders. Right click on those files in your solution explorer and select Properties.. --> change their type from "Content" to "Compile" and your reference should succeed:

Here is a link to more discussion and a post that had the answer I was looking for the whole time:
http://stackoverflow.com/questions/4228992/namespace-not-recognized-even-though-it-is-there

Google books lets you sample pages from any book

In "shouldn't-this-be-illegal?" news, google books lets you browse some limited number of pages of any book it indexes. That includes many technical books making this quite handy!

Here is one sample book, have fun.

Google lawyers have a FAQ: http://books.google.com/googlebooks/perspectives/facts.html

Fiction: If a book is still under copyright, scanning it without permission is illegal.

Fact: This is probably the most common misconception about Google Books, and about copyright law in general. The "fair use" provisions of U.S. copyright law (USC 17 107) describe the conditions under which someone may copy a work without the copyright holder's permission, like recording a TV show to watch later or quoting from an article in a blog post. Fair use is designed to safeguard copying that doesn't harm people's incentive or ability to produce and sell creative work, including books.

We've carefully designed Google Books to make sure our use of books is fair and fully consistent with the law. Copyright law is aimed at protecting and enhancing the value of creative works in order to encourage more of them–in this case, to ensure that authors write and publishers publish. We believe that by creating new opportunities for readers to find and buy books, we can help authors and publishers sell more of them. You can read more about fair use here.

Given the obvious risk of being able to reconstitute the book samples into the whole the established publishing regime must have disapproved. Google engineers must have seen that coming and had some reasonable arguments to defend the position. To be a fly on the wall there...

TDD is Dead Long Live Testing?

David Heinemeier Hansson has a great article on his blog titled TDD is Dead Long Live Testing. I wrote a similar article here in 2012: To Unit Test or not to Unit Test. TDD is a great programming technique and I cannot agree more with David that when it works, it brings a new confidence to the iterative development practice that is hard to match. But set in its proper context of the software and testing life cycle, TDD is an interesting technique that has costs often exceeding its benefits.

Software Training is All About Timing

The point of this post is simple -- software training cannot be successful until users have the live system in front of them with a goal in mind they want the software to help achieve.

TL;DR: Up front training is not successful for logical reasons. Use up front sessions to create user buy-in instead of training. Parallel testing is often the ideal time to train users.

I am not an expert when it comes to the fine details of software training. How best to conduct the sessions and how to present the content are things I respect but have little practical expertise in. I am, however, a witness to a great many people who learn how to use new software systems and I am (painfully) aware of the kinds of problems they have.

I am not naive about the difficulty of setting up the "software in front of your users with goal in mind" scenario but in most software deployments there is one convenient moment that puts your users in a position to succeed -- during parallel testing. A parallel test is no small task but it is the perfect opportunity for this scenario to exist for a wide range of users. Piggy backing on the parallel test for training is superior in efficiency of dollar spent per unit of user skill gained to traditional up-front user training because of the psychological engagement only real use cases can invoke. I strongly recommend parallel testing for any system roll-out for many other good reasons but utilizing the optimal training moment is one of the best side effects.

Psychologically, when users have "skin in the game" they can learn. And to be painfully realistic, very few are capable of learning before that point. We are busy, we have deadlines looming, we simply cannot pay attention to a system training for a system we will not touch for another 2 weeks and we've never seen before!

So I am recommending the abolishment of traditional up-front user training sessions? No. What I am recommending is a shifting of the goal of that up front meeting (and hopefully reducing its duration and expectations).  It can provide enough of a primer that the users think the system will be easy to use. Show high level things it can do. Describe in glorious detail how it will make their life easier. Set up scenarios and make the software sing. Its almost a full sales demo, but with the system's future users instead of its purchasers. Get them excited about it. That first training session is more about "buy-in" and emotional connection to the new reality. Make them want it by selling the system. That is about as good as you can hope for. This is a huge difference from the usual top-down approach that has a tendency to demoralize future users and encourage animosity towards support staff (notice how support teams are seperated from the trainers? Why are these are different teams? Support is very often "just in time" training!) negative feedback that finds its way back to the stakeholders as complaints about the "complexity of the system". That rationally stubborn user behaviour can be curbed by acknowledging the very human need for buy in in a sales-like kick off meeting. Of course there are limits to the success of this initiative and after that buy in meeting there may still be resistors, but at least you've identified them (or perhaps the solution doesnt solve their problems and thats a whole 'nother issue :)).

Web Deploy v2 and Web Application Packages

WebApplication.deploy.cmd -- when I run this command on my local machine I was getting this message:

To deploy this Web package, Web Deploy (msdeploy.exe) must be installed on the computer that runs the .cmd file. For information about how to install Web Deploy, see the following URL:
http://go.microsoft.com/?linkid=9278654

To install V2 of the MSDEPLOY.exe you MUST FIRST UNINSTALL V1! There is no failure message if you install V2 after V1, it just simply doesnt write the V2 files to the V2 directory if V1 is installed. Odd behaviour, but is what it is.

If this is not your issue, check out VS2010's SP1 fix to the WebApplication.deploy.cmd:
http://stackoverflow.com/questions/6595086/msdeploy-v2-doesnt-work-with-vs2010s-packages

CRM 2011 FYI for programmatic solution imports

The ImportSolutionRequest.PublishWorkFlows property also publishes SDKProcessingMessages! It doesnt say this in any documentation, but it does. It is explicit on a solution import through the GUI.

Rollup 12 Service Activity bug and Microsoft's Public Apology

Our users found an odd bug on our service activity form (where we have custom javascript to set a lookup field based on some business logic).

The error below started coming up after rollup 12. I explain steps to reproduce in more detail here:
http://stackoverflow.com/questions/14819416/bug-in-crm-service-appointment-forms-javascript-causing-form-save-error/14989557#14989557

We ended up getting an acknowledgement of the bug from Microsoft and they pushed a fix online, but will have to wait for rollup 13 for the fix on-premise. They are damaging their credibility with these bugs and I am not the only one to notice. It even prompted a public apology from Bob Stutz, a Microsoft VP. Apologies can't give me or my customers their time back.

>System.Web.HttpUnhandledException: Microsoft Dynamics CRM has experienced an error. Reference number for administrators or support: #52178855: System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.FormatException: Guid should contain 32 digits with 4 dashes (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).
>   at System.Guid.GuidResult.SetFailure(ParseFailureKind failure, String failureMessageID, Object failureMessageFormatArgument, String failureArgumentName, Exception innerException)
>   at System.Guid.TryParseGuidWithNoStyle(String guidString, GuidResult& result)
>   at System.Guid.TryParseGuid(String g, GuidStyles flags, GuidResult& result)
>   at System.Guid..ctor(String g)
>   at Microsoft.Crm.Application.Platform.AttributeCollection.CreateEnityReferenceFromLookupPropertyValue(String name, Object value, IOrganizationContext context)
>   at Microsoft.Crm.Application.Platform.AttributeCollection.SetEntityProperty(Entity entity, String name, String childAttributeName, Object parentProperty, AttributeMetadata attributeMetadata, Object value, IOrganizationContext context)
>   at Microsoft.Crm.Application.Platform.AttributeCollection.Insert(String name, Object value, Boolean throwIfKeyExists)
>   at Microsoft.Crm.Application.Platform.EntityProxy.SetLookupValueData(XmlNode node)
>   at Microsoft.Crm.Application.Platform.EntityProxy.SetData(XmlNode entityElement)
>   at Microsoft.Crm.Application.Platform.ApplicationEntityCollection.Deserialize(String entitiesXml, String entityType, IOrganizationContext context)
>   at Microsoft.Crm.Application.Platform.EntityProxy.SetData(XmlNode entityElement)
>   at Microsoft.Crm.Application.Forms.EndUserForm.RetrieveParametersForEventDefault()
>   at Microsoft.Crm.Application.Forms.EndUserForm.Initialize(Entity entity)
>   at Microsoft.Crm.Application.Forms.CustomizableForm.Execute(Entity entity, FormDescriptor fd)
>   at Microsoft.Crm.Application.Components.PageHandlers.SchedulableActivityBasePageHandler.ConfigureFormHandler()
>   at Microsoft.Crm.Application.Components.PageHandlers.ServiceAppointmentRecordPageHandler.ConfigureFormHandler()
>   at Microsoft.Crm.Application.Components.PageHandlers.RecordPageHandler.ConfigureFormWrapper()
>   at Microsoft.Crm.Application.Components.Utility.GenericEventProcessor.RaiseEvent(String eventName)
>   at Microsoft.Crm.Application.Controls.PageManager.OnPreRender(EventArgs e)
>   at System.Web.UI.Control.PreRenderRecursiveInternal()
>   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
>   at System.Web.UI.Page.HandleError(Exception e)
>   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
>   at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
>   at System.Web.UI.Page.ProcessRequest()
>   at System.Web.UI.Page.ProcessRequest(HttpContext context)
>   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
>   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Import CRM 2011 Solution Dependency Error for a Workflow

I have a workflow that creates a Case from an Opportunity. It maps fields like the title and price list and spits out a case, simple and tested in my development org. However, when I exported the unmanaged solution it resides in as managed and import that into a blank org, this error occurred:

The dependent component Attribute (Id=parentaccountid) does not exist.  Failure trying to associate it with Workflow (Id=13e950d4-8e77-e211-b5f0-080027cd26e8) as a dependency. Missing dependency lookup type = AttributeNameLookup.

Despite having the dependency (parentaccountid is a system field on the Opportunity Product entity!) in the target org. So right off the bat I know this is going to be a weird one. How can I not have a stock system field?

My own analysis looking into the XML for the workflow found one node with reference to parentaccountid for preferredserviceid. (perferredserviceid is a system field on the account). However, the workflow did not contain a reference to that preferredserviceid field! How odd. Its like CRM remembered at one point that it did have a reference to it and never removed it. That "memory" caused it to output the field as a dependency in the workflow XML but not in one of the workflow steps itself. How odd.

I attempted deleting the whole workflow and re-add it to see if that node exported in the new workflow and it did not. After deleting and painfully copying each detail from the original workflow into a new one the error was gone. Hope that helps someone else in this situation.

Execute FetchXML in CRM 2011

If you want to execute arbitrary fetchXML code, dont forget about the bids reporting extension.

While developing some custom code best suited to a fetchXML query, I wanted a rapid way to execute fetchXML for a complex query. A small test app would work, and there is even a simple codeplex project for exactly that here http://crm2011fetchexecute.codeplex.com/. However I recalled executing fetxhXML in the CRM 2011 bids reporting extension so if you have that installed simply open one of your CRM based reports and make a new dataset and use the query designer there to help debug those complex fetxhXML statements!

CRM 2011 Online Programatic Report Generation

CRM 2011 Online Programatic Report Generation isnt possible through the API.

Microsoft knows about the feature gap. They recommend we vote for it on connect. This is the highest voted item in connect for it: give it a vote with your live id?

https://connect.microsoft.com/dynamicssuggestions/feedback/details/740040/automatic-reporting

My scenario is for a customer who wants to auto create order reports and email them -- I have no solution other than recommending they move to on-premise.

UPDATE!

We want to generate Reports programmatically in a CRM-online environment (exported to PDF and/or word).

There appears to be a way to achieve this in a crm-online environment posted in this blog: http://xrmmatrix.blogspot.ca/2011/06/creating-report-as-pdf-attachment-in.html The logic behind it looks like it should work – since you can perform these actions via the web interface they must resolve into requests and responses so we can use XMLHttpRequests to get the data. The blog has many comments so it looks like the pitfalls have been addressed. 

CRM 2011 - Activity Party and SerializationException in plugins

I was working with the requiredattendees on the appointment entity and ran into a miserable time with a serialization exception. Funny thing was the exception is only thrown when I registered my plugin as isolation (aka sandbox) mode. No one seemed to have a post on the exact same issue so here's how I solved it:


I found an article about it that suggested removing all references to the activityparty svcutil generated class: http://nicolagrillo.blogspot.ca/2012/01/crm-2011-activityparty.html

That did not work, but I have not tried completely removing all reference to the class in my assembly as that is an unrealistic solution in my situation.

My error from the server trace: important part extracted for visibility:

System.Runtime.Serialization.SerializationException: Element 'http://schemas.microsoft.com/xrm/2011/Contracts:Entity' contains data from a type that maps to the name 'MyNamespace:ActivityParty'. The deserializer has no knowledge of any type that maps to this name. Consider changing the implementation of the ResolveName method on your DataContractResolver to return a non-null value for name 'ActivityParty' and namespace 'MyNamespace'.

>System.Runtime.Serialization.SerializationException: Microsoft Dynamics CRM has experienced an error. Reference number for administrators or support: #5BCA670D: System.Runtime.Serialization.SerializationException: Element 'http://schemas.microsoft.com/xrm/2011/Contracts:Entity' contains data from a type that maps to the name 'MyNamespace:ActivityParty'. The deserializer has no knowledge of any type that maps to this name. Consider changing the implementation of the ResolveName method on your DataContractResolver to return a non-null value for name 'ActivityParty' and namespace 'MyNamespace'.
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 id, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
>   at ReadArrayOfEntityFromXml(XmlReaderDelegator , XmlObjectSerializerReadContext , XmlDictionaryString , XmlDictionaryString , CollectionDataContract )
>   at System.Runtime.Serialization.CollectionDataContract.ReadXmlValue(XmlReaderDelegator xmlReader, XmlObjectSerializerReadContext context)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 id, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
>   at ReadKeyValuePairOfstringanyTypeFromXml(XmlReaderDelegator , XmlObjectSerializerReadContext , XmlDictionaryString[] , XmlDictionaryString[] )
>   at System.Runtime.Serialization.ClassDataContract.ReadXmlValue(XmlReaderDelegator xmlReader, XmlObjectSerializerReadContext context)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 id, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
>   at ReadAttributeCollectionFromXml(XmlReaderDelegator , XmlObjectSerializerReadContext , XmlDictionaryString , XmlDictionaryString , CollectionDataContract )
>   at System.Runtime.Serialization.CollectionDataContract.ReadXmlValue(XmlReaderDelegator xmlReader, XmlObjectSerializerReadContext context)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 id, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
>   at ReadEntityFromXml(XmlReaderDelegator , XmlObjectSerializerReadContext , XmlDictionaryString[] , XmlDictionaryString[] )
>   at System.Runtime.Serialization.ClassDataContract.ReadXmlValue(XmlReaderDelegator xmlReader, XmlObjectSerializerReadContext context)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 id, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
>   at ReadKeyValuePairOfstringanyTypeFromXml(XmlReaderDelegator , XmlObjectSerializerReadContext , XmlDictionaryString[] , XmlDictionaryString[] )
>   at System.Runtime.Serialization.ClassDataContract.ReadXmlValue(XmlReaderDelegator xmlReader, XmlObjectSerializerReadContext context)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Int32 id, RuntimeTypeHandle declaredTypeHandle, String name, String ns)
>   at ReadParameterCollectionFromXml(XmlReaderDelegator , XmlObjectSerializerReadContext , XmlDictionaryString , XmlDictionaryString , CollectionDataContract )
>   at System.Runtime.Serialization.CollectionDataContract.ReadXmlValue(XmlReaderDelegator xmlReader, XmlObjectSerializerReadContext context)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
>   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator xmlReader, Type declaredType, DataContract dataContract, String name, String ns)
>   at System.Runtime.Serialization.DataContractSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
>   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
>   at System.Runtime.Serialization.XmlObjectSerializer.ReadObject(XmlDictionaryReader reader)
>   at Microsoft.Crm.Sandbox.SandboxUtility.DeserializeDataContract[T](Byte[] serializedDataContract, Assembly proxyTypesAssembly)
>   at Microsoft.Crm.Sandbox.SandboxPluginExecutionContext.get_InputParameters()
>   at Microsoft.Crm.Sandbox.SandboxPluginExecutionContext.Merge(IPluginExecutionContext originalContext)
>   at Microsoft.Crm.Sandbox.SandboxPlugin.Execute(IServiceProvider serviceProvider)

I am very curious why it fails in isolation mode and works in non isolation. I hope the stack trace triggers a thought from someone.

Ultimately it turns out you cannot use the early bound classes when you want to assign to the target entity when you are in this context. I still don't complete understand what the context is that causes this error, because I use early bound classes in sandboxed plugin steps all the time, often assigning them into the target entity input parameter. Michael from microsoft has more detail here: http://social.microsoft.com/Forums/eu/crmdevelopment/thread/0b4d4d0f-0428-4964-9ce8-b86c89817974