Should You Design Upfront When Scaling Development?

Welcome to the Amazing Applications podcast -- for Microsoft Business Applications builders who want to create amazing applications that everyone will love.

Hi, I'm your host Neil Benson. My goal on this show is to help you slash your project budgets, reduce your delivery timelines, mitigate technical risks and create amazing, agile Microsoft Dynamics 365 and Power Platform applications.

The question in this episode comes from Michael Nunes. Michael is a Dynamics 365 functional consultant at sa.global from Peterborough in the UK. Alexa, please read Michael's question.

[Hi Neil, this is Michael Nunes. When multiple systems are being delivered by different parties, I find that we have to design the details upfront, then use agile for the delivery of the Dynamics 3 65 systems. Is this your experience too?]

Thanks for your question, Michael. Your question has a couple of layers, there's a scaling layer about working together with other teams and handling interdependencies between the parties. There's another layer about upfront design using a traditional approach with a design document then an agile approach for the delivery.

You'll find show notes for this episode at customery.com/021. The show notes include a transcript from the episode and links to any of the resources mentioned.

Let's take the upfront design layer first.

When we're building a complex product, especially one in a situation like yours, Michael, there's a tendency to attempt to reduce risk by gathering all the requirements upfront and then designing all the components of a system so that developers have a clear, unambiguous specification to work with.

Years ago, Microsoft's SureStep methodology taught us to follow this plan-based approach. Start with an Analysis phase to capture the requirements in a functional requirements document. Once the requirements document has been signed off by the users, then in the Design phase, design all the features of the systems in the functional design document and the technical design document. Once those are all signed off, we can start the Development phase.

This approach is based on a couple of completely false assumptions, and it also has some massive drawbacks.

Here are the assumptions:

We know all the requirements of a complex system upfront. Our users, despite never having used Dynamics 365, a complex Power App or a Power BI self-service dashboard will nonetheless be able to eloquently, unambiguously and definitely express their requirements. We've got system shall statements, use case and state diagrams. We'll be all right, mate!

We can design all the components of a complex system upfront. Our architects are not just gifted technical wizards, but they are mind-reading savants too. They can interpret those requirements specifications and know exactly what the users were thinking when they told the business analyst what they thought they needed. And we can design complex, interdependent components on paper and be sure that they'll perform exactly as expected.

From the moment we start Analysis to the moment we live in production, there are going to be no material changes to the requirements, to the design or to the plan. Everything will work out fine, just as we planned it. After all, we've got a Gantt chart, mate.

Those assumptions cause some massive drawbacks:

All that analysis and design can take months. Sometimes more than a year. At the end of it, we've got lots of documents. But we've created not one single penny of value. We have no working software yet. Nothing to show for our troubles. If the project gets cancelled tomorrow, we've turned tens or hundreds of thousands of pounds or dollars into technical specifications. I'd rather have the money.

There's no opportunity for improvement. The only time we learn anything is when we sit back at the end of the project and conduct a post-implementation review. Given the lack of success in many plan-based projects, this is usually an exercise in trying to pin the blame on the donkey.

And we try and make it impossible to change our minds. We don't show the users any working software until after development is complete. Then ta-da! If it isn't quite what they wanted then they can propose a change request which we'll use to show them exactly how expensive it is not to get your requirements right in the first place.

Am I a fan of upfront requirements specifications and designs.? Heck, no.

Here's an alternative idea. Get your product owner, some users, and developers in a workshop (and when I same developers, I'm referring to anyone who might build the application, including analysts, architects, developers and testers). If there are multiple teams involved, invite representatives from each team. Have them discuss, question and analyse the problem. Brainstorm a few possible solutions. Choose one and write it up. The design document should be a wiki page and ideally so short that it doesn't need a scroll bar.

Build a prototype, show it to your product owner and, if she likes it, see what the user think. Then keep iterating on that until it's ready to release into production.

Michael, that's my approach to emergent design. We do it as late as we can in our project. When we're as close as possible to development. By this time, our users know a little bit more about our application. They've seen it, they've participated in designing it. They know what a table is and what a choices field is (unless we've changed the names again). We've done the least amount of design work necessary to build some working software so test its value and its fit to the requirements. Instead of spending a lot of effort doing upfront design, we refocus that effort on as-built design -- documenting how the system actually works so that it can be enhanced, extended, maintained and supported by teams that come after us.

I hope that's given you something to think about for the layer in your question about documenting your design. Let's address the first layer about scaling teams and handling interdependencies.

When I've worked as part of large projects that had multiple teams working on components that fit together to deliver a complex business application, we tended to organise ourselves by technology speciality. There was the Dynamics 365 application team, the Portals team, the Data Migration team, the Azure services team, the legacy application API team. If the users in the Insurance department needed an insurance premium calculation, we'd develop the components in Dynamics 365, sometimes design the Portal user interface, call an Azure service such as a function or Logic App to call the Insurance application's premium calculation API and return the value to Dynamics 365 and we'd ensure that the legacy data for existing policies were migrated into the Dynamics app.

All this work was done by different teams who had to carefully coordinate their work and struggle through the interdependencies. The Dynamics 365 application team couldn't show a working feature until the Azure team had built the integration services, and they couldn't build their services until the legacy Insurance app team had built an API to support premium calculations.

We scaled using a scaled Scrum framework called Nexus. In Nexus, we have multiple Scrum teams each practising Scrum as we know it today, and we have a Nexus Integration Team, some additional events called the Nexus Sprint Planning, Nexus Daily Scrum, Nexus Sprint Review (which replaces the Scrum Sprint Review), the Nexus Sprint Retrospective and Refinement. We have a new artifacts called the Nexus Sprint Backlog that highlight all the items that have cross-team dependencies.

As I learn more about applying the Nexus framework to enterprise business applications projects, the one thing I've learned is that teams could have been much better organised to reduce cross-team dependencies.

Instead of organising our teams by technology speciality it would have significantly reduced our cross-team dependencies if we had organised our scrum teams so that they had developers specialising in Dynamics, portals, Azure, the legacy application and data migration in each scrum team. This would have made each scrum team more cross-functional and less dependent on other teams.

If you're interested more in how to organise your teams in a large-scale project, I recommend a book, the Nexus Framework for Scaling Scrum by Kurt Bitner, Patricia Kong and Dave West. I'll include a link to it in the show notes, which you can find at customery.com/021

I hope that helps answer your question, Michael. Try and avoid upfront design and organising your teams by technology speciality and you'll deliver a lot more value faster with a scaled approach to building your business applications.

That's it for another episode of Amazing Applications. And that's a wrap on this Q&A season of the podcast. Thanks to everyone who submitted a question:

• Neal Carty asked about product owners and Dynamics 365
• Mark Llewellyn asked how can we be sure we've captured all the requirements
• Anand Kushalnagar asked how we can adopt Scrum for Business Central projects
• David Mochrie asked if we should have a standard set of user stories for Dynamics 365 applications
• Carolle Bruce asked why we should have a scrum master when we have an agile project manager instead
• Matt Venamore asked how we can handle data migration on an agile business applications project
• Stephen Price asked how we manage scope creep in agile projects
• We covered the six roles every enterprise business applications project should have which was a question asked by one of my Microsoft partner coaching clients
• Andreas Dutz asked if Microsoft Business Applications teams should all follow the Manifesto for Agile Software Development
• Dan Madden asked how to bridge the gap between business and IT in agile projects
• Daniel Kerridge asked how to adopt Scrum on Power Platform projects with one developer
• And thanks to Michael Nunes who whether we should design components upfront when we've got multiple teams involved in our business applications projects

Thanks to everyone who asked a question. And hope that they, and you, found the episodes thought-provoking and insightful. If you did, please do me a favour and share your favourite episode on LinkedIn. You'll find them all listed under the Amazing Applications company page. You can leave a comment or a reaction and get involved.

If you'd like to have your question answered in the next Q&A season of the Amazing Applications show, please visit customery.com and click on the Send Voicemail button. Remember to include your name, role and where your from and your question. Click Submit when you're happy with it and we'll weave your question into a future show.

Until then, keep sprinting.