'The Force Awakens' or How awake the communication channels on a development team?

Introduction

I’m actually not a follower of the Star Wars saga even when I saw all the seven movies – the last one as a cinema pirate copy – plus some episodes of animated series and also including some TV movies with the Ewoks when I was younger.

An Ewok

Who didn’t heard the story about Anakin Skywalker? The boy that was trained as Jedi by Obi-Wan Kenobi who eventually was turned – by his trend to the dark side of the force  and also influenced by a Sith – into Darth Vader, who also was the father of Luke – the new hope to get the balance of the force – and the princess Leia and so on.

Probably I missed something, some of these movies are too long and actually, some characters make me sick. But the true is that these movies were a turning point in the world of the visual effects.

Star Wars - The Force Awakens poster

I won’t talk about the movies but the title of the seven episode “The Force Awakens” inspired me to write a new post. 

The concept of the force, such energy field with source even genetic - cell with something call “midi-chlorian” - that empower the jedies knights  to move objects with his mind, get great reflex and anticipation, and also great skills as pilots, is actually unreal but in some scenarios could reveal its presence. 

Let me show you the way you can found an expression of the force as a result of an approach that allow you awake the team communication channel from a hypothetical scenario. 

Common communication channels

On software development teams the communication is a key factor. Not a few agile practice talk about it including informative workspace, release the plan, sit together, pair programming, ubiquitous language or even poker planning.  I have no doubt that every single agile practice is about to improve the communication in the team.

From a real life scenario communication channels could look like this way:

Common communication channels

The client tells to the product owner what he wants. The product owner can create a vision of the final product – optionally with a consultant - and share the expected result to the project lead in terms of requirements and also prioritize such requirements. Then project lead and the team should be able to create a roadmap in term of features and small increments to cover all client’s expectation.

The project lead must share systematically all his knowledge or the vision with the team looking for an effective and integrated solution. Notice that the project lead should be the only communication channel between each developer and the people that really knows what the client wants. 

But the world isn’t perfect and the project lead also should be ready to manage the changes on requirements including say NO or defers some requirements. There is a single rule, every single decision must be taken in collective with the team.

Actually, as a developer, I always expect an effective communication system. It could be done through public the iteration plan, plus daily meetings, retrospective, and demos, everything that allow the whole team get an integrated view of the thing that they are building. An informative workspace where they can share the issues or impediments and, of course, enjoy the progress.

Such approach should work and actually does.

But sometimes something happens that drive the team to a complete failure of the communication system.

Breaking the communication channels

Allow me to twist a bit the previous scenario.

Assume that suddenly the product owner and the external consultant start to manage the team in a very odd way. They do direct tasks and components assignments in private instead thought the project lead. From another side, but not less important the project lead always saying YES to each the product owner request without team consulting. At some point, the product lead also stops to communicate with the team at all, probably just a few emails or comments about some new client expectation, but not in real team dynamics.

The issue here is about product owner and consultant vision. They don't want to know about small increments. They want the final solution and production ready now. They don't know about the team dynamics or how they manage the source or about components integration. They also expect for some magical entity do the coordination work. But such thing doesn’t exist in software development.

The result: a completely broken communication system.

Broken communication channels

Awaking the communication channels

Could be spent some time before you really notice the disaster. The bug corrections lead to other incongruousness or to regressions. Even worse, people sitting together can't found a real way to coordinate their work. But something remains immutable, no one talk about the system beyond a few individuals' virtual chatting sessions. 

At this point, some practices could save them all. So, also assume that the team have a full dependency in a build automated system (just a build server), and have a well configured a deployment pipeline that run unit and integration tests and also run the deployment routines into quality assurance servers (including product owner's server).  

You as team member could ask yourself: Why don't use this to trigger the communication back?

So, you as a team member with integration needs can start to write a lot of integration test. For instance, for each disagreement, for each incongruousness, every assumption, every of non-final decisions, everything that anyone told aloud, you can translate it into an executable code that immediately will run on the build server, and eventually this approach will lead to stopping the deployment pipeline due by the tons of errors.

This is a point when a sort of energy field (a.k.a 'the force') should start to play its role. The product owner needs upgrades. But the upgrades are locked because the deployment pipeline is locked. So, eventually, someone would need to start talking again.

Conclusion

There aren’t nothing more effective to improve – not only to awake – the communication in your team than the integration tests. That is because coding is actually the communication channel that never dies. Even better if you really use practice like continuous integration you are already empowered – without “midi-chlorian” – to awake you communication channels at any time.

By the way, while I´m writing this lines, I noticed that Multivision channel - on Cuba - started to broadcast the saga again. So, if you never saw it, it's a good timing ;)

Star Wars Episode I: The Phantom Menace at Multivision

Remember: Your focus determines your reality. 
May the force be with you.

Self-disciplined Agile Monitoring

Note for readers: I wrote this post over a year ago (even more), but for some reason I forgot to post it, so here it is.

Introduction

Now days, we are improving our development process laying-out our organization strategies, development process and methodologies. 

Our experiences point to a mixed approach, with the self-organization from Scrum and self-discipline from eXtreme Programming (XP). But about this kind of “mixed martial arts” for software development approach I will talk in forthcoming blog post. 

As you can read in one of my previous blog post, we use Team Foundation Server (TFS) as issue tracker and thanks to Scrum for Team System v3 (STSv3) process template, we have an “excellent” implemented guidance to execute Scrum “as is”.

But the existing day by day sprint monitoring’s tools around this process template hides the real behavior of the team in the iteration. 

On the other hand, we also have some needs about monitoring. Indeed against the theoretical practice of monitor the whole team we have to track individuals.

Some projects leads (me included) want to track theirs developers work against the importance of monitor the whole team behavior. Sam Guckenheimer (with Juan J. Perez), in his book “Software Engineering with Microsoft Visual Studio Team System”, aims us to use descriptive metrics rather than prescriptive. But here such measurement method “doesn’t work”, we need more control and also need a single view. 

Tracking the daily work in STSv3

Scrum for Team System (v3) is a great process template. Its major advantage of this implementation for TFS is about the usage of the server side event notification API. This allows updating all of computable fields for instance: start and end time for sprints (summarized from team start and end sprint dates), remaining hours for sprints and team sprints, and so on.

Scrum for Team System also comes with a lot of reports, but about daily work sprint monitoring comes with only one (maybe two).  This report is known as Sprint Burn Down. 

The burn down metric

The Sprint Burn Down is good metric but it’s incomplete. Such single line indeed hides the real behavior of iteration. 

Please, try to answer these questions:

• How can you notice if some tasks where moved out of the current sprint?
• How can you notice if some user stories where moved out of the current sprint?
• How can you notice if some tasks where added to the current sprint?
• How can you notice if some user stories where added to the current sprint?

The fact is that as this chart doesn’t display the planned work so it hides these behaviors. 

Notice:

1. If you see an ideal chart, an straight down line from an amount of hours as remaining work (at some point of the sprint) to zero (at the end of the sprint), doesn’t mean that everything is fine. May some tasks were moved of the sprint.
2. If you see a horizontal line doesn’t mean that everything is wrong. May some tasks were added at the same time that others were actually done. 

The burn down metric

The fact is that you are not able to answer this question: What had really happened here?

Complementary tools to track the daily work

One of my favorites tools to track iterations, and also share the iteration status with the whole team is ScrumSprintMonitor. Yes, the multi-process template screen saver. 

The effect of the usage of this tool in the team’s focus is incredible and of course, more over the “guys in red” ;-).

Demo screenshot of ScrumSprintMonitor from codeplex and ScrumSprintMonitor in action but in planning mode in a lab

I like the Scrum Sprint Monitor and have been using it for years, even when I had no longer available the TV in the picture above. But again, the main metric is the Burn Down.

This tool includes a lot of info so I get inspired and wrote my own, just like I thought that it should be written ;-).

Self disciplined agile monitoring

Well, the thing actually started some years ago when I found this chart in the book “The Art of Agile Programming” from James Shore and Shane Warden. Its name is Burn Up. 

Basically consist of a couple of lines. One to show the total of planned work and the second to display the progress.

The burn up metric

In the context of a project to help my own organization to get the right way in terms of software development practices, I bought some time in order to port this metric into this monitoring application.

My own ScrumSprintMonitor in action with the burn up metric

As I said before it was inspired in Scrum Sprint Monitor but it was re-written from scratch to focus in the Burn Up metric. The current version it’s only compatible with STSv3.

Conclusion

1. Now we have a very cool monitoring application with a very meaningful metric. The Burn Up. 
2. In order to distribute as fast as possible this tool and its updates across my organization, I also implemented a draft of NuGet based automatic update system. After done, someone told me about something called Shimmer. I have to review it ;-).
3. I almost forget. This application is powered by Catel and Prism in combination (a.k.a. Catel.Extensions.Prism). 

Happy sprinting and monitoring ;-)

If you can’t defeat them, just “join” them – Part I

Introduction

I spend a lot of time using Subversion as the Version Control System for my development team. The fact is I love Subversion and strongly think that there is no a centralized version control system (CVCS) like Subversion. 

On the other hand, my organization promotes the usage of Microsoft Team Foundation Server (TFS) as Application Life-Cycle Management (ALM) and I also think TFS is a great integrated environment. However some of its services are in disadvantage mainly in terms of usability against some third party software.

But I’m not here to talk about TFS vs. Subversion, even when the user experience of the Visual SVN is better than Team Explorer (even in VS2012).

I just want to share how I committed to the indication of shutting down my local team Subversion, and move my entire sources to the centralized TFS, keeping all my “non-integrated” development environment’s cool features.

What features will I miss if I’ll move to TFS version control system?

I received a lot of critics because I don’t use the integration features of Visual Studio Team System. Actually I have a non-fully integrated environment for ALM (TFS and Subversion) but always keeping traces between sources and tasks. I know that TFS has built-in support for this and also have a check-in policy system to “force” some team disciplines.

On the other hand, I needed move my team into some of self-discipline development practices to control the existing “chaos”: uncommented commits, unplanned or non-well-reported work, and more. In such scenario, I was “forced” to ensure the Subversion and TFS integration and also “route” my team into the right direction. 

I thought that I would able to handle this situation starting from the Subversion with relative ease and I was right. I implemented a hook to enforce the usage of these practices and also solve some integration issues. The continuous integration server (Team City) also helps me in track task with sources, listing all related tasks from the TFS. The fact is I also modified the existing TFS integration plugin to work as I expected, but this is part of another history.

Therefore with a simple Subversion hook I get a centralized or server side “check-in” polices system to:

Avoid empty commit messages: Every single commit must contain a description.

Avoid unplanned work: Every single commit message must contain the id of the work item of type “Sprint Backlog Item” in “In Progress” state.

Automatically track the task history: As every single commit message contains a work item id then all commit messages are attached as history of the work item. 

Accept some relevant commit comments or messages: A "relevant" commit comment is about a commit comment that indicates a feature completion. It should be related with addition, modification, removal or bug fixing, but just when those goals are actually "Done, Done". The usage of this pattern also turn automatically to the “Done” the state of the related work items. This messages can also use to generate the release notes document for a specific version

Migrating from SVN to TFS version control system

My support team started with some existing migration tools but the outcome was incomplete (with runtime errors included). On the other hand, I actually thought in move from SVN to GIT, because the next version of TFS comes with built-in support for GIT repositories but apparently “they” can’t wait and yes, I have to move my source to TFS.

Therefore, with the GIT idea in my mind, I came across a couple of GIT extensions and I started to type the follow commands:

> git svn clone http://svn.mydivsion:3690/repository
> git tfs init http://tfs.mycompany:8080/tfs/DefaultCollection/MyDivision $/MyDivision 
> git tfs fetch 
> git rebase /remotes/default/tfs master 
> git tfs rcheckin –authors="authors.txt"

Everything works like charm. The full migration can be done. Now, after years of discussions, I lose and “they" win. Now, I have to start over again.

The fact is that I think TFS was built on top of one of the best Business Intelligence Platform (BI) ever made and can be configured to produce a lot of multidimensional metrics very useful for ALM. But this migration to TFS version control system are backwards steps, at least for me.

Hook the TFS commits from the server side

As I wrote before, TFS has its own check-in policy system. Actually it is a “client side evaluation” check-in policy system, and I don’t like them. Such policies can also be overridden by the developers. TFS check-in policy system looks more like a warnings system.

But TFS comes with a server side event handling system and can be plugged-in deploying an assembly into the web service plugins directory of the application tier of the current installation of TFS.

I started to re-write all my useful check-in polices, referencing the right TFS assemblies.

For instance, the “Avoid empty commit messages” policy looks like this:

public class AvoidEmptyCommitMessagePolicy : ISubscriber
{
    public string Name
    {
        get { return " AvoidEmptyCommitMessagePolicy"; }
    }

    public SubscriberPriority Priority
    {    
        get { return SubscriberPriority.Normal; }
    }
    
    public EventNotificationStatus ProcessEvent(TeamFoundationRequestContext requestContext, NotificationType notificationType, object notificationEventArgs, out int statusCode, out string statusMessage, out ExceptionPropertyCollection properties)
    {
        statusCode = 0;
        properties = null;
        statusMessage = string.Empty;
        var eventNotificationStatus = EventNotificationStatus.ActionPermitted;
        if (notificationType == NotificationType.DecisionPoint)
        {
            var checkinNotification = notificationEventArgs as CheckinNotification;
            if (checkinNotification != null && string.IsNullOrEmpty(checkinNotification.Comment))
            {
                statusMessage = "Your commit has been blocked because you didn't give any log message.\n Please write a log message describing the purpose of your changes and \n then try committing again.";
                eventNotificationStatus = EventNotificationStatus.ActionDenied;
            }
        }

        return eventNotificationStatus;
    }
}

The “Avoid unplanned work” looks like this:

public class AvoidUnplannedWorkPolicy : ISubscriber
{
        /*...*/
        if (checkinNotification != null)
        {
                CheckinNotificationInfo notificationInfo = checkinNotification.NotificationInfo;
                CheckinNotificationWorkItemInfo[] workItemInfos = notificationInfo.WorkItemInfo;
                if (notificationType == NotificationType.DecisionPoint)
                {
                    if (workItemInfos == null || workItemInfos.Length == 0)
                    {
                        statusMessage = "Your commit has been blocked because you didn't give any work item linked to this changeset.";
                        eventNotificationStatus = EventNotificationStatus.ActionDenied;
                    }
                    
                    var teamFoundationLocationService = requestContext.GetService<TeamFoundationLocationService>();
                    var uri = new Uri(teamFoundationLocationService.ServerAccessMapping.AccessPoint + "/" + requestContext.ServiceHost.Name);
                    TfsTeamProjectCollection tfsTeamProjectCollection =  TfsTeamProjectCollectionFactory.GetTeamProjectCollection(uri);
                    var workItemStore = tfsTeamProjectCollection.GetService<WorkItemStore>();
                    if (workItemInfos.Exists(workItem => workItem.Type != "Sprint Backlog Item" ||  workItem.State != "In Progress"))
                    {
                        statusMessage = "Your commit has been blocked because you didn't give any “Sprint Backlog Item” in state "In Progress" linked to this changeset.";
                        eventNotificationStatus = EventNotificationStatus.ActionDenied;
                    }
                }
       }
        /*...*/
}

and so on.

The AvoidUnplannedWorkPolicy is very Scrum for Team System process template related. It can be done more generic and cross process template support, but it’s just an example. 

That fact is I’m now thinking about a lot of policies to improve my team practices, but if tell you I have to kill you  ;-)

Conclusions

Seems like I can live with such transition (to TFS version control system), but it isn’t all happiness. Now I have to wait for an approval process to deploy my server side policies in the main TFS server. But if “they” don’t like such intrusion then probably I will also back into the “chaos”.

In the meantime I just wrote this blog post.

PD: Yes I'm back in the game again, but now without Subversion, with my ankle ligaments broken as outcome of a basketball game, and with my team Villa Clara as the new Champion of the Cuban National Baseball League ;-)

Release notes generation puzzle

Introduction

Why a puzzle? It is just about interaction. I want to share my knowledge with you and hope you want to share your knowledge with me. I know you as a reader of this blog have lots of wisdom I can learn from.

I know that I'm "copying" the style of Java specialists with this kind of a puzzle but I like it, so here we go...

 

The problem

 

I will talk about a very simple and useful practice, which I have formalized thanks to the combination of Catel team practices and my own development process automation needs. It isn't a new one and probably you have used it before.

Let's start taking a look at the Catel history file, located at %workspace%\doc\history.txt. I just show you a summary for simplification and, why not, I also give you a preview of the next version of Catel.

Catel history
=============
(+) Added
(*) Changed
(-) Removed
(x) Error / bug (fix)

...
===========
Version 3.2
===========
Release date:
=============
2012/xx/xx

Added/fixed:
============
...
(+) Added IViewModelFactory so the instantiation of view models can now be fully controlled and customized
(+) Added Description property to IMementoSupport to allow a description to the end-user for memento classes
(*) The IOpenFileService now supports multiple files and in WPF it also supports a title
...
(*) Updated System.Windows.Interactivity.dll for SL5

(x) Fixed issue that views and view models could not be resolved when a naming convention contained 2 or more [UP] constants
...
===========
Version 3.1
===========
...

I always dreamed to produce this kind of release notes. But all my previous attempts gave me a strong headache because sometimes the issue tracker doesn't contain all features/fixes that were born from the "creative" developer process or the version control commit comments just said something like "Merged".

So, where does the history file added/fixed lines come from?

Read the history file again. You should notice a legend and the very simple technique that allows us to tag the changes and document them over time.

But it seems like an overwhelming task and requires self-discipline to keep this kind of log up-to-date. Actually we (the Catel team) update this file manually, actually, Geert do most of these updates,  and we also know many teams that do it manually as well.

But this doesn't need to be updated manually. This is the problem that I want to solve and I hope you will share your experience and knowledge with me. The solution is pretty simple but it just become clear to me when I ported the "relevant" commit comment technique (the Catel team practice) to one of my "real live" projects.

Is it to possible generate release notes with a combination of "relevant" commit comment techniques, of course, a version control system (VCS), an issue tracker system (ITS), and a continuous integration server (CIS)?

The clues

 

1)  What is a "relevant" commit comment?

    A "relevant" commit comment is about a commit comment that indicates a feature completion.  It should be related to addition, modification, removal, or bug fixing, but just when those goals are actually "Done, Done". Yes, you read well, I wrote: "Done, Done". A feature is "Done, Done" when is tested, coded, designed or refactored, integrated, it builds, etc. For details read more "The Art of Agile Development".

2) Take a look at the source control page.

 

The solution

There are several scenarios; the fact is that there are tons of version control systems, issue trackers, and continuous integration servers. So, I will wait for your solutions for a couple of weeks, you just have to comment here about it.

I want to share my solution with Subversion (VCS), Team Foundation Server (ITS) with Scrum for Team System process template, and Team City (CIS) but you will need to wait for my next blog post. The truth is that I still have to implement it ;).

What do you think? Can this problem be solved? I know, you have the answer and I am waiting for it.

Remember a couple of weeks. Please don't hesitate just comment on your solution here.