English is Purple and One Dimensional

and why "Source Code" is a terrible place to put Software

Everyone agrees that a Picture is Worth a Thousand Words, but have you really stopped to think about why? 

An obvious reason is that pictures are 2 dimensional, and can make use of all the colors of the rainbow, while if I were to assign English a color and a shape it would have to be strictly Purple and quite Linear in nature.  Allow me to explain... 

It is purple because while we think of English as a Primary Color, English is in fact composed of two completely separate and distinct things, just like purple is actually a mixture of blue and red. 

What I mean is that English can be disambiguated into two completely separate and distinct parts.  A Blue Abstract Model representing the underlying facts and data about the idea being expressed, along with a Red Algorithm, that can interpret the meaning embedded in the underlying facts and generate the purple English that us human beings want to read, when given the Blue Abstract Model as an input.  We'll dig into exactly how this works below.

Additionally, the shape of an English document is very linear and one dimensional in it's nature.  In other words, when we read a book we read all the words in a big long line of words, starting with the first word, and then the next word, and the next, sequentially until "The End." 

We can read faster or slower.  We can skip over parts and move forward and backwards along the line - but in general, the meaning of language is conferred through it's vocabulary, syntax and grammar, rather than through it's physical location on the page or within a book.

These two facts about English, and about language in general are what make it such a terrible place to put software, technical systems, protocols, and complex ideas in general.

Instead, the English technical specification that human beings need and expect for any given project should actually be a report, written against an abstract model of the idea - which can be shared amongst all languages, including English, French, German, Spanish, C#, Python, SQL, etc.  All of these languages are purple, and one dimensional, and they are not a good format to assign as the "source" for complex ideas.

What's the Alternative?

At this point, you're probably asking... "Well, that's great EJ - but what's the alternative?", and that's a completely reasonable question.

While we might start with a Purple English description of an idea, literally the next step should be to extract a Blue, Data-Based, Abstract Model of the idea from the purple language (as described below,) that includes enough fidelity to unambiguously represent the strict facts about the underlying concepts.

Step 3 is then to validate that we have successfully accomplished this by creating a Red Algorithm (basically a "report") that reassembles the Blue Model back into the original Purple English.  Doing this proves, by it's very existence, that the model has enough internal detail to accurately represent the original idea.

The key is that by pulling apart the Purple into these two separate parts, a different "report", i.e. a different Red Algorithm, can take the same underlying facts from the model - but construct longer, possibly more detailed report about the idea... possibly in French, or German.  It would use the same underlying facts.  The same underlying elements and details.  The same information being communicated - but arranged into a different syntax, grammar - and possibly using a different vocabulary, i.e. possibly a completely different language.

Each of these different reports however hopefully represents exactly the same idea - the same underlying facts - which should be Language Independent.  Truth should be language independent.

What is "Truth"

Answering this question may seem academic, but having a shared understanding of what makes something "true" is essential for meaningfully differentiating between a Linguistic Description of a system and a Blue Model or "Digital Twin" of the system as described below.

The best description I've heard for the definition of truth is that...

Something is True if it Comports with "Reality".

The problem with this definition is that of course everyone creates their own version of reality. 

So, while trying to communicate an idea with Bob, Alice will say some words, which hopefully have the same meaning to Bob as they do to her.  And then, based on Bob's understanding of Alice's purple words, he will say words back to her to convey what he think's she meant. 

And if Bob seems to be thinking of the same thing as Alice, then she may agree.  And if not, she might disagree and say more purple words back to Bob, in order to attempt to update his understanding to more accurately match her understanding of the idea that they're trying to communicate about. 

But this is all clearly an exercise futility,  even when both parties are speaking the same language, because "reality" is completely Subjective.  And all of this becomes dramatically more difficult when the parties are speaking different languages, like English and Python or C#, for example.

By contrast, the blue model described above can actually serve as "digital twin" (more below) of the idea - and then all we each have to do is agree that it accurately represents our understanding of the idea in question. 

This opens the door to simply define this "digital twin" as being "Reality" - at which point the truthiness of literally any linguistic statement, can be Objectively Tested by simply checking if it "comports with reality" - where reality is defined as our Digital Twin. 

Creating a "Digital Twin" for the underlying idea

Language's Purple and One Dimensional nature make it an undesirable candidate to be the "Source" encoding of complex ideas.  It is so inefficient at communicating complexity because all it can do is dance around an idea, relying on the parties consuming the idea to share the same understanding of every word and inference of the language being used.

The Blue Model described above, by contrast, is not a linguistic description of the idea.  It is not a language at all, in fact.  Instead, it is digital instantiation of the idea.  A digital example of the idea that really serves as the platonic ideal of the idea that we are ultimately trying to capture with Language, and as a result, provides us with the the opportunity to agree on a shared "reality". 

This "digital twin" can be created in virtually any no-code tool, from databases, to spreadsheets, to no-code services like Airtable, Tray.io, Bubble.io or others.  The only requirement is that it is not "code".  I.e. that it is just the decisions about how a system should behave, and should generally be exportable to json, xml, csv or a similar data-based, non-linguistic format.

This multi-dimensional data structure literally forms a physical picture in space.  Not a description, but a digital instantiation of the idea being discussed..

With this digital twin in hand, all the project stakeholders can agree that it is an accurate representation of the idea.  No words are needed.  They can simply look at the Digital Twin - and it if looks, and acts, and behaves as expected - everyone can give a simple thumbs up or a thumbs down. 

You could literally have 10 people approve the Digital Twin - and they could each literally speak a different language - and never actually communicate with each other directly in any way.  Instead, they all simply look at the model and give a thumbs up or a thumbs down.

Once everyone involved agrees, at an abstract level, that the digital twin accurately represents the system, protocol, or software that we are trying to actually build - everything else gets dramatically easier.

Mail Merge

Think of a simple mail merge, where we want to send the following email to Mary a Graphic Designer, along with Bob and Juanita who are applying to work in Sales.  

Dear Mary,
Thank you for your recent application to work with us as a Graphic Designer.  We will review your application and be in touch shortly.  Sincerely, HR Manager Ellen.

So while we want the same basic content within each of the emails, we actually need 3 different versions of the email, each one including candidate specific details like their name and job.  And, this email is just one of many different things that we need to do with this list of candidates.

We could do this by creating an email to Mary, and then copying and pasting it twice more, replacing Mary with Bob and Graphic Designer with Salesperson - and this is largely how software get's written, even in 2022.

Instead, however, we could also take the lessons learned in the 1980's, and split that 3 page purple content into it's two constituent parts. 

A list of candidates, along with the job they are applying for,   (a blue model)

Applicant  Phone     Job               Address...

SSoT vs a "Traditional" Development Model

 In a Traditional Development Model, there is an order to how things must be developed in order to efficiently develop code – and it is critical that this order be followed, precisely, or things get really expensive, fast.

Traditional Project Management

In a traditional development flow, the process unfolds as follows.

1. Someone comes up with an Idea and describes it to the Dev Team
2. They develop a “Specification”, this specification might include:
   1. Business Requirements
   2. Technical Requirements
   3. Wireframes
   4. User Stories
   5. Data Model (schema) designs
   6. etc
3. With all parties having been consulted, the spec is given to the development team who begin to “write the code”
4. Developers create Back-End systems, API’s, Client Applications, Documentation, Unit Tests and more
5. Testers go through and find problems – and product stability is achieved.
6. Now the “Project Owner” (customer) is brought in to review.  Typically changes are needed/requested
7. Change Requests are made, and the project scope slips.
8. The change requests then effectively go back to step 2 – and thus we “iterate”.

Often at this point, the project might be 30-50% of the way through the budget on both Time and Money – and frequently, big changes are needed at this point.

As it turns out, this is a very expensive way to write code.  It’s expensive for a number of different reasons which we’ll explore below.

Order of Operations is Critical

In a “Traditional Development Model” It is absolutely key and essential that the original specification be as complete as possible before a single line of code is written.  The reason for this is that it is not uncommon for literally 100’s of man hours to be invested in making the specification into something real.  So if it’s wrong, it can literally cost 100’s of man hours before a shift is possible.

If it’s wrong, and change request is needed, it is critical that the exact nature of the change is fully understood – and that a complete description is provided or changes are costly as well.

Costs Increase with Size

Another cascading problem of the “Traditional” approach is that as the project proceeds, things tend to get more and more out of Sync.  This problem is exacerbated as it goes over budget, because there’s less and less time to fully implement changes consistently across the stack.  In other words, a change is requested, and the back end get’s updated – but the Client App is still out of date.  Or the API get’s updated, but they have to wait for the back end folks to implement their changes before they can proceed.

These problems also get worse are the code-base grows.

Having an SSoT changes Everything

With an SSoT – everything works differently.  For one thing, “the specification”, “the code” and “the docs” are all “the same” in a project based on an SSoT.  In other words, when the SSoT changes, everything changes – as set.

As a result – on day one, when only an hour has been spent on specifying what is actually needed – with a stack based on an SSoT, it is not uncommon to have a fully functional implementation of that description, end to end – right out of the gate.

It’s not “done” – and it will change again and again and again, but it exists consistently across the full technology stack on Day 1, Day 10, Day 100, etc.

So – after a short discussion, we have a sketchy implementation which can actually be used, and the Project Owners (the customers) can use the tool and immediately provide feedback on what works, what doesn’t work – and the thus the iterations can begin.

So – Agile is to Waterfall as development based on an SSoT is to Agile.

Speed, Price Quality - Pick All 3!

 

SSoT.me – the ‘Single Source of Truth’ Toolbox.

---

They say, “Fast, Good, Cheap… Pick 2” – but SSoT.me let’s you pick all three.

Creating software based on a Single Source of Truth allows you to do so in a fraction of the time time vs a “Traditional” development model. This speed translates into significantly reducing the overall cost of a project.

As a side affect benefit, this approach also results in dramatically better code along a variety of dimensions.

Us Humans

First and foremost though, Derivative Code can always “explain itself” in Natural Language English. This is the most important layer for all of us Humans!

This “Derivate Documentation” always (and this is important) matches “the tech” which it is describing, because is are is derived. Because is is not “source code”.

Derivative Code

“Derivative Code” is code derived from a Single Source of Truth or SSoT. When compared to code not developed with an authoritative SSoT, Derivative Code tends to be more:

1. Complete
2. Stable
3. Flexible
4. Durable
5. Understandible
6. Internally-consistent
7. And Responsive

The reason that the Derived Documentation always matches the tech it’s describing is that neither it, nore the derived-tech it is is describing is the “Source” description. Instead, both are “derivative code” – both from one, central, authoritative single source of truth. When the SSoT changes – both the documentation and the code change, because they are are derived, because they are not “source code”.

“Source Code” or “Hand Code”

“Source code” is expensive. “Source Code” has to be maintained by hand. Human Hands. And Human Hands are expensive.

The goal of ever project should be to reduce the total amount of source code, since 1,000,000 lines of source code will inevitably cost more to create, maintain, upgrade, manage, etc than a 1,000 line app all else being equal.

And one way to dramatically reduce the amount of “source code” which you have to manage is to turn 50, 60, even 90% or more of it into “Derivative Code”. What that means is that when the SSoT changes – 80% of the code in the project automatically updates itself to match the new design.

To be clear, the “source code” would still have to be updated “by hand”. So the remaining 20% of the changes needed would still need to be done by human beings – but hopefully “the interesting bits” in most cases. In other words….

What is the “Low Hanging Fruit?”

The 80% that is most likely to be turned into derived code first, is “the plumbing”. The … well … “derivative” … parts. i.e. The parts of the code that are not too interesting, and are primarily following a well established pattern. This includes any code which, if pointed to, is 1-of-n. Are there 2? Maybe it’s derivative. Are there 3? Or 10? Definitely derivative.

What we’ve found is that most developers are surprised at how much of the code in their project can be isolated and turned into derivative code.

Get Started >